From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/52720 Path: main.gmane.org!not-for-mail From: kai.grossjohann@gmx.net (=?iso-8859-1?q?Kai_Gro=DFjohann?=) Newsgroups: gmane.emacs.gnus.general Subject: Re: which gnus files used to hold gnus state? (for manual) Date: Fri, 16 May 2003 21:03:34 +0200 Organization: University of Duisburg, Germany Sender: ding-owner@lists.math.uh.edu Message-ID: <84d6iin27t.fsf@lucy.is.informatik.uni-duisburg.de> References: NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1053112042 2268 80.91.224.249 (16 May 2003 19:07:22 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Fri, 16 May 2003 19:07:22 +0000 (UTC) Original-X-From: ding-owner+M1264@lists.math.uh.edu Fri May 16 21:07:19 2003 Return-path: Original-Received: from malifon.math.uh.edu ([129.7.128.13]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 19GkUq-0000Hn-00 for ; Fri, 16 May 2003 21:04:00 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by malifon.math.uh.edu with smtp (Exim 3.20 #1) id 19GkUl-00020T-00; Fri, 16 May 2003 14:03:55 -0500 Original-Received: from sclp3.sclp.com ([64.157.176.121]) by malifon.math.uh.edu with smtp (Exim 3.20 #1) id 19GkUc-00020N-00 for ding@lists.math.uh.edu; Fri, 16 May 2003 14:03:46 -0500 Original-Received: (qmail 49795 invoked by alias); 16 May 2003 19:03:45 -0000 Original-Received: (qmail 49789 invoked from network); 16 May 2003 19:03:45 -0000 Original-Received: from quimby.gnus.org (80.91.224.244) by sclp3.sclp.com with SMTP; 16 May 2003 19:03:45 -0000 Original-Received: from news by quimby.gnus.org with local (Exim 3.12 #1 (Debian)) id 19GkbN-0007vj-00 for ; Fri, 16 May 2003 21:10:45 +0200 Original-To: ding@gnus.org Original-Path: not-for-mail Original-Newsgroups: gnus.ding Original-Lines: 194 Original-NNTP-Posting-Host: pd9e1e0f0.dip.t-dialin.net Original-X-Trace: quimby.gnus.org 1053112245 30486 217.225.224.240 (16 May 2003 19:10:45 GMT) Original-X-Complaints-To: usenet@quimby.gnus.org Original-NNTP-Posting-Date: 16 May 2003 19:10:45 GMT Mail-Copies-To: never User-Agent: Gnus/5.1003 (Gnus v5.10.3) Emacs/21.3.50 (gnu/linux) Cancel-Lock: sha1:yneB6BLy53ZRKFYTDtdEEqPUoOE= Precedence: bulk Xref: main.gmane.org gmane.emacs.gnus.general:52720 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:52720 John Owens writes: > I'd like to get a list of the files used to hold gnus state. This is > for potential inclusion in the manual (after discussion with Lars) > and would be important for users who are interested in synching the > gnus state of two machines (say, a laptop and a desktop). Sounds like a real cool project. Alas, my knowledge is rather limited. But I'll share what I know. > 1) general gnus state files that apply to any backend - .newsrc, > .newsrc.eld, .emacs, .gnus. Is that the complete list? This could go > in manual section 1.8, "Startup Files". You could add site-start.el and default.el, if you mention .emacs. But those are general Emacs files, not Gnus-specific files. Maybe you should mention which files are Gnus specific and which aren't. .newsrc, .newsrc.eld, .gnus are Gnus specific. The rest isn't. > 2) gnus state files that apply to each backend. I am particularly > concerned with nnimap, but I'm not sure there are any state files in > gnus that are associated with it (?) because maybe the imap server > keeps track of that internally? Evidently nnmail uses .nnmail-cache? Maybe you should group the backends in some way -- nnmail is not a backend per se, it's a group of backends. There is also gnus-registry.el which comes into play. But maybe Ted should say something about it. > I'm aware there are News and Mail folders but I'm not sure what is > supposed to live in them. How does this interact with nnimap - where > are imap folders located in the directory structure and how does this > work with gnus? > > This could go in section 6.3 where the backends are discussed. We have ~/Mail. It contains the file ~/Mail/active for an nnmail-based backend. It also contains files and/or directories for the groups from nnmail-based backends. So people who are using nnml will see the list of nnml groups in ~/Mail/active, and the nnml:foo.bar group lives in a directory called ~/Mail/foo/bar or ~/Mail/foo.bar, depending on nnmail-use-long-file-names. Then ~/Mail contains Incoming* files which are used by the mail-sources mechanism to fetch mail. All mails fetched by mail-sources are first dumped into such an ~/Incoming* file, and from there they are distributed into the groups (this is called splitting). It seems that these files are usually deleted right after use, but mail-source-delete-incoming can be frobbed to change this. Then we have ~/.gnus-mail-crashbox or similar (forget the exact name) which plays a role similar to the ~/Mail/Incoming* files, but I don't know details. Any experts out there who can help? Now let's talk about the ~/News directory. It contains a number of files and subdirs, some related to news, some related to mail. It's rather confusing, I'm afraid. First of all, we have ~/News/drafts which contains the nndraft:drafts and nndraft:queue groups, and also the nndraft:delayed group if you use that. (Some people are surprised to see a directory called ~/News/drafts/drafts -- that's just the nndraft:drafts group.) Secondly, we have a lot of *.SCORE and *.ADAPT files which are used for scoring and adaptive scoring, you guessed it. ~/News/agent and ~/News/cache will be discussed below. > 3) gnus state files that apply to the Agent. Are there any? > (~/News/agent should contain ... ?) How about the cache? Where do > cached files live? (~/News/cache, but what files?) If I delete all the > cached files will they be refetched by the Agent? I'm, again, > particularly concerned with the imap interaction with the agent, and > want to know how files work with it. (Ideally I can have > semi-up-to-date IMAP mail folders on my laptop and can work with > those, and when I plug in they will be auto-updated manually? or > automatically?) > > This could go in section 6.8 which deals with the Agent. Please note that ~/News/cache has nothing to do with the agent, though there has been talk about unifying the cache and the agent. The talk comes from the fact that they have a similar purpose. The cache lives in ~/News/cache and the agent lives in ~/News/agent. The cache is the older part of Gnus. The cache is meant to be used manually by the user while browsing: the user hits some key and the article is entered into the cache. This means that the article will stay around even after the server has expired it. So, normally, the cache is only useful for nntp, and perhaps nnimap. But of course you can count on creative abuses of the cache even for nnml and its ilk ;-) ~/News/cache contains an active file (listing all cached groups) and a directory for each group. (My directories have names like ~/News/cache/nntp+dfn:comp.emacs, but I gather that it might be ~/News/cache/nntp/dfn/comp/emacs or the like if gnus-use-long-file-name is nil.) Each group directory contains a .overview file and the articles. Now, for the agent. There is a subdir ~/News/agent/lib which contains the categories and the servers files. Further subdirs are for servers/groups. (Presumably the exact directory layout depends on some *-use-long-file-name(s) variable.) A directory for a server contains a subdir agent.lib with the file agent.lib/active. All other subdirs are groups (in foo.bar or foo/bar format). Each group contains a file .agentview and a file .overview and the messages. What does .agentview contain? --> Anyone? If you delete cached files, they will NOT be refetched by the agent: you need to cache a message to insert it into the cache. The agent has nothing to do with this. Yes, you can have an ~/News/agent directory which contains all the messages from your IMAP server. Well, probably not ALL messages. And as long as you're unplugged, Gnus will use those local files. There are a number of commands for updating the state. Some commands read information from the server and change the local directory to match the situation on the server, other commands work in the other direction. In particular, `J s' fetches messages from the server (according to criteria you specify), and `J S' sends queued outgoing messages, and `J Y' (?) sends marks changes to the server. Marks changes means it tells the server which messages you have read or ticked etc. Actually, `J Y' happens semi-automatically -- after connecting to the server Gnus asks you whether to send the marks changes. > The overall goal would be to know exactly what files to sync if I want > to mirror my gnus setup on computer B from computer A. Hopefully in > the next generation of gnus there will be a list of state-containing > files and an automated way to sync them (gnus-sync-state-files with a > configurable sync command like "rsync -e ssh ..."). This could be > useful for users who use a laptop and want to sync before going on > the road. Nifty. If you want to replicate the cache, replicate ~/News/cache. Similar for the agent. And for mail, replicate ~/Mail. It is difficult (but doable) to replicate just *parts* of the cache, or parts of the agent, or parts of your mail. For the cache and the agent, the problem is the active files. You need to have the right lines from the active files. (I'm not sure if it works to replicate the whole active file but only some groups. Anyone?) For mail, there are some backends where it is fairly painless to replicate just some groups. This is true for nnml and nnfolder: you just replicate the right directory, plus the .marks file (nnml) or the *.mrk file (nnfolder), and Bob's your uncle. It is also probably true for nnmaildir, but I'm no expert. Anyone? (Actually, the .marks file lives inside the directory for the nnml group, so just the directory for the nnml group is good enough. For nnfolder groups, you replicate two files -- the group file itself, and the *.mrk file.) But in these cases, you also have the active file problem. Hm. But it is very difficult to replicate just some groups for other mail backends. For example, nnmh does not have a .marks file and thus the read/ticked/dormant/... marks are ONLY stored in the .newsrc.eld file, and if you transfer that without also transferring all the groups -- hm, strange things may happen. Is there anyone who has tried? Hm. Now that I think of it, the .newsrc.eld problem also may happen with nnml and nnfolder: if you transfer .newsrc.eld, then you're in trouble. Hm. Hm. Hm. So, it boils down to this: if you transfer .newsrc.eld, you got to transfer the whole hog. If you abstain from .newsrc.eld, then you can do the some-groups trick alluded to for nnml and nnfolder (and probably nnmaildir). Stuff from cache and agent aren't mentioned in .newsrc.eld, so there is no problem. Above, when I say .newsrc.eld, I always also mean .newsrc. -- This line is not blank.