From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/67296 Path: news.gmane.org!not-for-mail From: David Engster Newsgroups: gmane.emacs.gnus.general Subject: Re: gnus and imap Date: Mon, 25 Aug 2008 21:50:29 +0200 Message-ID: <87prnwvpq2.fsf@randomsample.de> References: <877iabwtjx.fsf@randomsample.de> <87abf51c4m.fsf@marauder.physik.uni-ulm.de> <874p5dm35l.fsf@randomsample.de> <87iqttq7ja.fsf@randomsample.de> <87d4k0t7ga.fsf@randomsample.de> <87myj3wzp7.fsf@randomsample.de> <878wumqo7z.fsf@randomsample.de> <8763pq6k0d.fsf@rimspace.net> <868wulotw7.fsf@lifelogs.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1219693895 3791 80.91.229.12 (25 Aug 2008 19:51:35 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 25 Aug 2008 19:51:35 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M15747@lists.math.uh.edu Mon Aug 25 21:52:20 2008 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1KXi6Y-0007S6-Qn for ding-account@gmane.org; Mon, 25 Aug 2008 21:51:59 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1KXi5K-0003d2-2U; Mon, 25 Aug 2008 14:50:42 -0500 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1KXi5H-0003cf-VE for ding@lists.math.uh.edu; Mon, 25 Aug 2008 14:50:39 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtp (Exim 4.69) (envelope-from ) id 1KXi5D-0006HX-Mc for ding@lists.math.uh.edu; Mon, 25 Aug 2008 14:50:39 -0500 Original-Received: from m61s02.vlinux.de ([83.151.21.164]) by quimby.gnus.org with esmtp (Exim 3.36 #1 (Debian)) id 1KXi5F-0003aV-00 for ; Mon, 25 Aug 2008 21:50:37 +0200 Original-Received: from dslb-082-083-032-075.pools.arcor-ip.net ([82.83.32.75] helo=honk) by m61s02.vlinux.de with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1KXi5e-0000UK-Ge for ding@gnus.org; Mon, 25 Aug 2008 21:51:03 +0200 Mail-Copies-To: never Mail-Followup-To: ding@gnus.org In-Reply-To: <868wulotw7.fsf@lifelogs.com> (Ted Zlatanov's message of "Mon, 25 Aug 2008 13:02:16 -0500") User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/22.2.90 (gnu/linux) X-Spam-Score: -2.6 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:67296 Archived-At: Ted Zlatanov writes: > DE> Thanks for your explanation. While this behavior of Zimbra might be > DE> peculiar, it's in accordance with the IMAP RFC, which only demands UIDs > DE> to be unique and ascending. So I'm afraid it is Gnus which will have to > DE> be fixed here, which - being originally only a nntp client - still > DE> assumes that article numbers are more or less contiguous. > > I would appreciate a quick primer: > > What elements of the Gnus data formats enforce this assumption currently? > > What backend functions make this assumption? Which functions outside > the backends have knowledge of the data format internals? As far as I see, Gnus almost always tries to determine article counts from the active data and the stuff that is saved in the group's info[1]. However, all you can get from this data are upper bounds. It will only be exact in the case of strictly contiguous article numbers. For example: In `gnus-articles-to-read', the total number of articles in a group is simply the length of (setq articles (gnus-uncompress-range (gnus-active group))) With 'select' being the number of articles you want to retrieve from a group, Gnus queries the backend with the following article list: ;; Select the N most recent articles. (setq articles (nthcdr (- number select) articles)))) (where 'number' is the length of 'articles'). Similarly, unread articles are determined in `gnus-list-of-unread-articles' by doing the following: ;; This function returns a list of article numbers based on the ;; difference between the ranges of read articles in this group and ;; the range of active articles. In the case of nnimap, this estimate is currently "fixed" (in fact overridden) through an imap-search for UNSEEN, which is done when closing a nnimap group and saved in a separate hash table. Article counts are done in the backend interface. AFAIK, the backends itself don't assume much about article numbers. They just have to be unique there. I agree with Vitaly that the best way to solve this is to give the backend interface complete knowledge of the available article numbers in a group. We could use an additional backend function for this, nnchoke-request-group-articles, which already exists for nntp, but somehow isn't used further in Gnus. Based on what nntp-request-group-articles returns, the function for nnimap would roughly look like this (deffoo nnimap-request-group-articles (group &optional server) (nnimap-possibly-change-group group server) (let ((articles (with-current-buffer nnimap-server-buffer (imap-search "ALL")))) (if articles (with-current-buffer nntp-server-buffer (erase-buffer) (insert (format "211 %d %d %d %s\n" (length articles) (car articles) (car (last articles)) group)) (insert (mapconcat 'number-to-string articles "\n")) (insert "\n.\n") t) nil))) Using this function will make backend access slower, so it should be optional, maybe configurable through an additional server variable. As Vitaly mentioned, this information should be cached so that we don't have to get it every time a group is accessed. -David