From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/66031 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general Subject: Re: file storage in IMAP (eventually for Tramp) working and needs testing Date: Thu, 03 Jan 2008 07:21:42 -0600 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <86prwjxcwp.fsf@lifelogs.com> References: <86k5n1443g.fsf@lifelogs.com> <87fxxjgohd.fsf@gmx.de> <873atjgf6m.fsf@gmx.de> <86r6h2x6fv.fsf@lifelogs.com> <87y7bag519.fsf@gmx.de> <86ejd2wyej.fsf@lifelogs.com> <87odc6ft4e.fsf@gmx.de> <86myroyn3v.fsf@lifelogs.com> <877iir4win.fsf@gmx.de> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1199366504 9158 80.91.229.12 (3 Jan 2008 13:21:44 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 3 Jan 2008 13:21:44 +0000 (UTC) Cc: ding@gnus.org, Tramp Development List To: Michael Albinus Original-X-From: ding-owner+M14524@lists.math.uh.edu Thu Jan 03 14:22:00 2008 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1JAQ1H-0007j7-7s for ding-account@gmane.org; Thu, 03 Jan 2008 14:21:59 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1JAQ0c-0008R4-RA; Thu, 03 Jan 2008 07:21:18 -0600 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1JAQ0a-0008Qo-VD for ding@lists.math.uh.edu; Thu, 03 Jan 2008 07:21:16 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.67) (envelope-from ) id 1JAQ0Z-00051u-6J for ding@lists.math.uh.edu; Thu, 03 Jan 2008 07:21:16 -0600 Original-Received: from blockstar.com ([170.224.69.95] helo=mail.blockstar.com) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1JAQ0Y-0007NQ-00 for ; Thu, 03 Jan 2008 14:21:15 +0100 Original-Received: from tzlatanov-ubuntu-desktop.jumptrading.com (unknown [38.98.147.130]) by mail.blockstar.com (Postfix) with ESMTP id A9CFD418979; Thu, 3 Jan 2008 05:51:18 -0800 (PST) X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" X-Hashcash: 1:20:080103:michael.albinus@gmx.de::uM/NRr/4gtrMQvV4:0000000000000000000000000000000000000001ngW X-Hashcash: 1:20:080103:tramp-devel@gnu.org::gOpBzyftzHSx0Zm1:0000000000000000000000000000000000000000000m3U X-Hashcash: 1:20:080103:ding@gnus.org::vG7OsUfByU2VAWPh:000058bx In-Reply-To: <877iir4win.fsf@gmx.de> (Michael Albinus's message of "Thu, 03 Jan 2008 00:50:56 +0100") User-Agent: Gnus/5.110007 (No Gnus v0.7) Emacs/23.0.50 (gnu/linux) X-Spam-Score: -2.6 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:66031 Archived-At: On Thu, 03 Jan 2008 00:50:56 +0100 Michael Albinus wrote: MA> Ted Zlatanov writes: MA> I'm a little bit lost. Does it mean you don't want to offer *all* MA> messages in a mailbox? >> >> Correct, otherwise it's hard to handle invalid messages: are they >> invalid files or something else? I wanted also to treat duplicates as >> automatic backups but if you don't like that idea then I'll drop it. MA> Maybe we should see real life examples. I don't know whether it is MA> always good to present selected contents only. If there are technical MA> restrictions - that's another game. (I'm leaving Ding on the CC in case anyone has comments) Say you have these article UIDs and subjects, and UIDs are ordered by date. Note the ENCODED_FILENAME will be the filename encoded in some way and /a/b/c is the filename without any risky characters; Gnus has the subject encoding functionality already. The subject format is specifically to allow easy searching. 1 "tramp-imap-marker /a/b/c ENCODED_ABC" 2 "tramp-imap-marker /a/b/c ENCODED_ABC" 3 "tramp-imap-marker /a/b/c ENCODED_ABC" 4 "hello world" Do we A) allow for three identically named files, dynamically renaming 2 and 3 with the Emacs <1> and <2> convention? B) ignore 1 and 2, never create them intentionally, and (optionally) erase duplicates when a file is saved? This is probably easiest to implement in code. C) treat 1 and 2 as backups (there will be a tramp-imap-backups parameter to control how many are allowed per filename, and older ones over that limit are erased when a file is saved)? We'll need a mechanism to revert to a backup, which I don't think Tramp has built-in at the moment. Second question: is message 4 ignored? I would prefer to do so, to allow coexistence of tramp-imap.el with other messages (or even later versions of tramp-imap.el). I think it's fine to use message UIDs as the true file name, but we need to decide what to do in the cases above. Third question: namespaces. I feel that it's much better for the user to store all the files in a single mailbox: INBOX.storage holds /a/b/c, /d/e/f, and /g/h/i I believe you proposed that instead we should auto-create: INBOX.storage.a.b to hold /a/b/c INBOX.storage.d.e to hold /d/e/f INBOX.storage.g.h to hold /g/h/i Did I understand you correctly? Maybe we could do both, allowing for a "root mailbox" and a "root prefix". In the first case, those would be INBOX.storage / and in the second case, they would be INBOX.storage.a.b /a/b INBOX.storage.d.e /d/e INBOX.storage.g.h /g/h And maybe the user can configure those mappings exactly. I still think it's too complicated for 99% of the users and they'll never need more than one mailbox per virtual filesystem, but if you disagree I can do the extra work. I'll assume we've picked the single mailbox approach for the rest of the message. The implementations will change quite a bit as far as directories are concerned if we use multiple mailboxes. MA> As starting point you might look at tramp-smb.el or tramp-gw.el. Both MA> are addons, like tramp-imap.el could be. tramp-fish.el might be examined MA> as well, but this method isn't used anywhere I believe - it was merely a MA> proof of concept I didn't want to throw away. And I never ever got a bug MA> report about ... OK, I'll look. tramp-gw.el doesn't have any Emacs primitives implemented at first glance, so I'll look at tramp-smb.el which defines all those mappings nicely in tramp-smb-file-name-handler-alist. That list has quite a few methods; are they all necessary or do the default handlers for some of them suffice? Here's the list (omitting the commented-out items from tramp-smb.el) with some questions and comments. Sorry if most of this is obvious, this is my first time diving into the Tramp code. What function would handle the truename-to-visualname translation? add-name-to-file: could be a special "link message" or just a copy, like in Windows copy-file: implemented as an APPEND delete-directory: implemented with a search+delete for all matching messages delete-file: search+delete of all matching messages directory-file-name: tramp-handle-directory-file-name? directory-files: search for matches directory-files-and-attributes: search for matches, attributes always 777 dired-call-process: ignore dired-compress-file: ignore file-accessible-directory-p: always t file-attributes: always 777 file-directory-p: needs a search, but we could have a file name that conflicts with a directory name file-executable-p: always nil file-exists-p: needs a search file-local-copy: ? file-remote-p: tramp-handle-file-remote-p file-modes: tramp-handle-file-modes or hard-code file-name-all-completions: needs a search file-name-completion: tramp-handle-file-name-completion file-name-directory: tramp-handle-file-name-directory file-name-nondirectory: tramp-handle-file-name-nondirectory file-newer-than-file-p: needs a search, plus backups may matter file-ownership-preserved-p: ignore file-readable-p: always t file-regular-p: always t file-symlink-p: always nil file-truename: returns UID file-writable-p: always t find-backup-file-name: we need to decide insert-directory: ? insert-file-contents: search+retrieve load: tramp-handle-load make-directory, make-directory-internal: we need to decide what this should do make-symbolic-link: ignore or implement as a special message rename-file: append with new subject, delete original and its backups if we decide to do backups set-file-modes: ignore set-visited-file-modtime: we could use a header for this shell-command: ignore? substitute-in-file-name: ? unhandled-file-name-directory: tramp-handle-unhandled-file-name-directory? vc-registered: always nil verify-visited-file-modtime: ? write-region: needs to do an append+delete of original+backups as needed; IMAP can't rewrite a message Thanks Ted