From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.user/17823 Path: news.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.gnus.user Subject: Re: Finding message by ID fails when message not visible Date: Sun, 20 Dec 2015 11:28:06 +0800 Message-ID: <8760ztre7t.fsf@ericabrahamsen.net> References: <87wpsfb5jh.fsf@ericabrahamsen.net> <877fkdvnc0.fsf@ericabrahamsen.net> <878u4rrwvb.fsf@ericabrahamsen.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1450582141 27873 80.91.229.3 (20 Dec 2015 03:29:01 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 20 Dec 2015 03:29:01 +0000 (UTC) To: info-gnus-english@gnu.org Original-X-From: info-gnus-english-bounces+gegu-info-gnus-english=m.gmane.org@gnu.org Sun Dec 20 04:28:51 2015 Return-path: Envelope-to: gegu-info-gnus-english@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aAUfz-0002hW-AQ for gegu-info-gnus-english@m.gmane.org; Sun, 20 Dec 2015 04:28:51 +0100 Original-Received: from localhost ([::1]:39498 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aAUfy-0008Ms-ML for gegu-info-gnus-english@m.gmane.org; Sat, 19 Dec 2015 22:28:50 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48804) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aAUft-0008ML-Mu for info-gnus-english@gnu.org; Sat, 19 Dec 2015 22:28:49 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aAUfp-0000R3-5r for info-gnus-english@gnu.org; Sat, 19 Dec 2015 22:28:45 -0500 Original-Received: from plane.gmane.org ([80.91.229.3]:60496) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aAUfo-0000Ql-R5 for info-gnus-english@gnu.org; Sat, 19 Dec 2015 22:28:41 -0500 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1aAUfk-0002H2-J8 for info-gnus-english@gnu.org; Sun, 20 Dec 2015 04:28:36 +0100 Original-Received: from 218.94.59.34 ([218.94.59.34]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 20 Dec 2015 04:28:36 +0100 Original-Received: from eric by 218.94.59.34 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 20 Dec 2015 04:28:36 +0100 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 283 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 218.94.59.34 User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1.50 (gnu/linux) Cancel-Lock: sha1:bAuxwBTZ0ADL8XLTbQwGjyvHHig= X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 80.91.229.3 X-BeenThere: info-gnus-english@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Announcements and discussions for GNUS, the GNU Emacs Usenet newsreader \(in English\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: info-gnus-english-bounces+gegu-info-gnus-english=m.gmane.org@gnu.org Original-Sender: info-gnus-english-bounces+gegu-info-gnus-english=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.gnus.user:17823 Archived-At: Kai von Fintel writes: > Eric Abrahamsen writes: > >> Kai von Fintel writes: >> >>> Eric Abrahamsen writes: >>> >>>> Kai von Fintel writes: >>>> >>>>> Kai von Fintel writes: >>>>> >>>>>> Eric Abrahamsen writes: >>>>>> >>>>>>> Kai von Fintel writes: >>>>>>> >>>>>>>> I'm puzzled by the following behavior: if I search in an >>>>>>>> nnimap group >>>>>>>> for a message with a particular message-ID, the search fails >>>>>>>> unless the >>>>>>>> message is visible. If for example, only new messages are >>>>>>>> shown, the >>>>>>>> search for an old message with a particular ID fails but the >>>>>>>> same >>>>>>>> message is found when "/o" has made the old messages appear. >>>>>>>> >>>>>>>> This is relevant because I use links to gnus messages from >>>>>>>> org-mode and >>>>>>>> more often than not the link fails: I get taken to the nnimap >>>>>>>> group that >>>>>>>> contains the message, but I get the error "No such article >>>>>>>> (may have >>>>>>>> expired or been canceled)", even though the message is in fact >>>>>>>> in the >>>>>>>> group. >>>>>>>> >>>>>>>> I suppose org-gnus.el might have to be amended to work around >>>>>>>> this, but >>>>>>>> first I'd like to understand whether this is expected behavior >>>>>>>> in gnus. >>>>>>>> >>>>>>>> Can anybody enlighten me or help me trouble-shoot this? >>>>>>> >>>>>>> No, it's not expected, and I haven't seen it before -- >>>>>>> sorry for the >>>>>>> unhelpful reply! I just tried it (dovecot imap server), and it >>>>>>> worked >>>>>>> both with and without angle brackets on the ID (sometimes >>>>>>> brackets can >>>>>>> be an issue). >>>>>>> >>>>>>> To be sure everyone's on the same page, how exactly are you >>>>>>> searching? >>>>>>> If you're in the *Group* buffer and you hit "G G" on the group, >>>>>>> and >>>>>>> paste in the message ID, is the message found? If you're >>>>>>> already in the >>>>>>> *Summary* buffer, there isn't really any way to "search" as >>>>>>> such, >>>>>>> there's only "limiting" what messages are shown by some >>>>>>> criteria -- in >>>>>>> those cases it's true you won't be able to get to unseen >>>>>>> messages. >>>>>>> >>>>>>> But that should never affect following links from org mode... >>>>>>> >>>>>>> Eric >>>>>> >>>>>> Thanks for the reply, Eric. More details then. >>>>>> >>>>>> 1. GG in the group buffer works (after a brief delay, displaying >>>>>> "Opening server FastmailLocal" and then "Searching >>>>>> nnimap+FastmailLocal:Archive...done"). >>>>>> >>>>>> 2. M-^ ("Fetch article with id ...) in the summary buffer (a) >>>>>> works when >>>>>> the message is visible, (b) fails when the message isn't visible >>>>>> (with >>>>>> "No such article (may have expired or been canceled)"). >>>>>> >>>>>> 3. When I follow the org link, say >>>>>> "gnus:nnimap+FastmailLocal:Archive#56614DE9.5030300@upf.edu", I >>>>>> do get >>>>>> taken to the right nnimap group in gnus but the article is not >>>>>> displayed >>>>>> (again, with "No such article (may have expired or been >>>>>> canceled)"). >>>>>> >>>>>> One funny thing is that when the "search" fails (in both cases >>>>>> 2b and >>>>>> 3), the article summary actually is visible (see screenshot: >>>>>> https://www.dropbox.com/s/b1b9mxi7bl0a51j/missing-article.jpg?dl=0) >>>>>> >>>>>> If 1 and 2 are as expected, then I guess the issue is with >>>>>> org-gnus and >>>>>> how it interacts with my IMAP set-up (offlineimap + dovecot + >>>>>> nnimap)? >>>>>> >>>>>> -- Kai. >>>>> >>>>> Some more follow-up. I tried to understand the code in >>>>> org-gnus.el and >>>>> org-sum.el. It appears that the functions "org-gnus-open" and >>>>> "org-gnus-follow-link" call the function >>>>> "gnus-summary-goto-article" >>>>> from gnus-sum.el. The latter is called with the optional >>>>> arguments "nil" >>>>> and "t". The last one is supposed to force that all articles are >>>>> loaded, >>>>> but that doesn't seem to happen. >>>>> >>>>> When I call (gnus-summary-goto-article "56614DE9.5030300@upf.edu" >>>>> nil t) >>>>> in the summary buffer of the relevant group, it finds the message >>>>> when >>>>> it is listed and doesn't find it if the buffer has only the first >>>>> 200 >>>>> articles (loading all articles with "/o" then makes the function >>>>> succeed). >>>>> >>>>> I added (gnus-large-newsgroup nil) to the group parameters, which >>>>> then >>>>> loads all articles when entering the group, and >>>>> (gnus-summary-goto-article "56614DE9.5030300@upf.edu" nil t) then >>>>> succeeds in the summary buffer. >>>>> >>>>> However, the gnus link still doesn't work from outside gnus. It's >>>>> as if >>>>> the FORCE optional argument that is given to >>>>> "gnus-summary-goto-article" >>>>> has no effect. >>>> >>>> I've got the same setup as you (Gnus and local dovecot), but >>>> perhaps the >>>> versions are different? Can you tell us your versions for Gnus and >>>> Org? >>>> >>>> When I step through `gnus-summary-goto-article', searching for a >>>> message >>>> ID that is *not* currently display, the "force" argument never >>>> even >>>> comes into play. I end up in this branch: >>>> >>>> (if (and (stringp article) >>>> (string-match "@\\|%40" article)) >>>> (gnus-summary-refer-article article) >>>> >>>> So then on to `gnus-summary-refer-article' (no force argument >>>> used), and >>>> in that function I end up in the "t" branch of the "cond" later >>>> on, and >>>> eventually into "(gnus-summary-insert-subject message-id)". Then >>>> that >>>> leads to "(gnus-read-header id)", and all is well. >>>> >>>> I know this is tedious, but would you mind stepping through with >>>> edebug >>>> and seeing where things fall apart? My off-the-cuff guess is that >>>> you're >>>> using an older version of Gnus where the nnimap backend isn't >>>> quite all >>>> put together... >>>> >>>> If you aren't comfortable with edebug let me know and I can >>>> provide >>>> exact instructions. It's a very useful thing to know! >>>> >>>> Eric >>> >>> #### System Info >>> - OS: darwin >>> - Emacs: 24.5.1 >>> - Gnus: v5.13 >>> >>> Well, that was fun ... sort of. I found something I can tweak to >>> make it >>> work, but I'm not clear on why this issue arises. >>> >>> At the relevant point, the article id is >>> "<56614DE9.5030300@upf.edu>" >>> (the <> having been added by gnus-summary-refer-article) and the >>> group >>> is "nnimap+FastmailLocal:Archive". >>> >>> In `gnus-request-head (article group)', a funcall to >>> `nnimap-request-head' >>> is constructed. Because `gnus-override-method' at that point >>> contains a long >>> construct `(nnimap "FastmailLocal" ...)', the group argument of the >>> call >>> to nnimap-request-head is set to nil rather than to "Archive". I >>> take >>> it, the idea is that nnimap should be able to return the correct >>> group >>> on its own? >>> >>> The relevant code (in gnus-int.el) is >>> >>> (setq res (funcall head article >>> (and (not gnus-override-method) >>> (gnus-group-real-name group)) >>> (nth 1 gnus-command-method))) >>> >>> Now, `nnimap-request-head (article nil "FastmailLocal")' returns >>> "(nil . >>> 277)". I believe the "nil" in that cons should be the group >>> ("Archive"). >>> Maybe that's a problem in nnimap.el but looking at the code I don't >>> see >>> how it could ever return anything other than nil. >>> >>> From that point on, the group info seems lost and the message can't >>> be >>> targeted. >>> >>> If I revert the commit to gnus-int.el that introduced the override >>> method thingy >>> (http://git.gnus.org/cgit/gnus.git/commit/?id=8ba1cd0b96466a96265ec5336728519aa6030d83), >>> to the following instead of what's above: >>> >>> (setq res (funcall head article (gnus-group-real-name group) >>> (nth 1 gnus-command-method)))) >>> >>> my messages get found because the actual group is passed on to >>> nnimap-request-head. >>> >>> So, I have things working to a certain degree, but I'm utterly >>> confused >>> about why this is happening. >>> >>> Can you see from this what the problem is? >> >> My guess is it might just be time to update Gnus. Nnimap has >> undergone a >> lot of changes over the past couple of years, as has the rest of >> Gnus, >> and while the versioning system has always confused me, I'm pretty >> sure >> v5.13 is several years old. The code branches you seem to be >> following >> don't match up with what I'm seeing (in git master, aka Ma Gnus), >> and >> it's probably not worth trying to figure out exactly what's going >> on. I >> find Ma Gnus pretty stable -- would you be willing to run from git? >> >> Eric > > OK, I switched to Ma Gnus from git master (`gnu-version' now reports > "Ma > Gnus v0.14)"). > > No luck, same issue. And reverting `gnus-request-head' in gnus-int.el > to > the version without gnus-override-method makes the links work. I will > run through it all again with edebug (which unfortunately is > super-slow > here) over the holiday break and see whether I can determine more > precisely where my installation fails. I'm very puzzled that your > pretty > much identical set-up works. Well, let me know how that goes. I just stepped through a couple of link-following processes, and can tell you that the call to gnus-request-head/nnimap-request-head also returns a cons of (nil . 12345) here. By that point we're already in the proper Summary buffer, so I don't think it matters that the group name isn't in there. I notice that gnus-summary-insert-subject actually gets called twice, with the same "id" argument, and only inserts the correct article on the second pass, but I haven't gone so far as to figure out why that happens. In the let* construct at the top of the function, this branch: (t (gnus-read-header id)) Returns a full header vector (with article number, and the various Subject/From headers) for the article I'm looking for. Anyway, there are a few more points of data to work with, hope you can sort it out... Eric