From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/68448 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general,gmane.emacs.tramp Subject: Re: file storage in IMAP (eventually for Tramp) working and needs testing Date: Tue, 14 Apr 2009 11:22:56 -0500 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <86vdp7qk67.fsf@lifelogs.com> References: <86k5n1443g.fsf@lifelogs.com> <86k55otuwi.fsf@lifelogs.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1239725960 6621 80.91.229.12 (14 Apr 2009 16:19:20 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 14 Apr 2009 16:19:20 +0000 (UTC) Cc: tramp-devel@mail.freesoftware.fsf.org To: ding@gnus.org Original-X-From: ding-owner+M16884@lists.math.uh.edu Tue Apr 14 18:20:39 2009 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1LtlN8-0002Cp-TN for ding-account@gmane.org; Tue, 14 Apr 2009 18:20:31 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1LtlLg-0007xC-5p; Tue, 14 Apr 2009 11:19:00 -0500 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1LtlLe-0007we-1u for ding@lists.math.uh.edu; Tue, 14 Apr 2009 11:18:58 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.69) (envelope-from ) id 1LtlLY-0004Vx-10 for ding@lists.math.uh.edu; Tue, 14 Apr 2009 11:18:57 -0500 Original-Received: from main.gmane.org ([80.91.229.2] helo=ciao.gmane.org) by quimby.gnus.org with esmtp (Exim 3.36 #1 (Debian)) id 1LtlLx-00066I-00 for ; Tue, 14 Apr 2009 18:19:17 +0200 Original-Received: from list by ciao.gmane.org with local (Exim 4.43) id 1LtlLX-0005N5-0d for ding@gnus.org; Tue, 14 Apr 2009 16:18:51 +0000 Original-Received: from 38.98.147.130 ([38.98.147.130]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 14 Apr 2009 16:18:51 +0000 Original-Received: from tzz by 38.98.147.130 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 14 Apr 2009 16:18:51 +0000 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 361 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 38.98.147.130 X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.0.91 (gnu/linux) Cancel-Lock: sha1:gv1vuNcw9qj7+KRp2zm4ehDETeM= X-Spam-Score: -1.5 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:68448 gmane.emacs.tramp:6762 Archived-At: --=-=-= On Tue, 14 Apr 2009 10:44:56 +0200 Michael Albinus wrote: MA> Ted Zlatanov writes: >> It's all yours if you want it. I will gladly help with testing and >> whatever else you need. MA> As a first step, you could add the current version to the Tramp CVS MA> repository. This might ease to work on it. It's really, really embarassing to check that messy pile of code into Tramp. Most of it is just TODO information. I am attaching it for yours and Magnus' review. The encrypt-* calls should be replaced by auth-source-* calls, as well. If you really want me to check it in, I will (and if this is a sneaky way to get me to fix it up and write the missing code, it's working :) Ted --=-=-= Content-Type: application/emacs-lisp Content-Disposition: attachment; filename=tramp-imap.el Content-Transfer-Encoding: quoted-printable ;;; tramp-imap.el --- Tramp interface to IMAP through imap.el ;; Copyright (C) 2007 Teodor Zlatanov ;; Author: Teodor Zlatanov ;; Keywords: mail, comm ;; This file is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation; either version 2, or (at your option) ;; any later version. ;; This file is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with GNU Emacs; see the file COPYING. If not, write to ;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, ;; Boston, MA 02110-1301, USA. ;;; Commentary: ;; Package to provide Tramp over IMAP ;;; TODO: ;; From: Michael Albinus ;; Subject: Re: file storage in IMAP (eventually for Tramp) working and nee= ds testing ;; To: Ted Zlatanov ;; Cc: ding@gnus.org, Tramp Development List ;; Date: Fri, 04 Jan 2008 12:00:08 +0100 ;; X-Sent: 4 weeks, 4 days, 9 hours, 41 minutes, 25 seconds ago ;; Ted Zlatanov writes: ;; > Say you have these article UIDs and subjects, and UIDs are ordered by ;; > date. Note the ENCODED_FILENAME will be the filename encoded in some ;; > way and /a/b/c is the filename without any risky characters; Gnus has ;; > the subject encoding functionality already. The subject format is ;; > specifically to allow easy searching. ;; > ;; > 1 "tramp-imap-marker /a/b/c ENCODED_ABC" ;; > 2 "tramp-imap-marker /a/b/c ENCODED_ABC" ;; > 3 "tramp-imap-marker /a/b/c ENCODED_ABC" ;; > 4 "hello world" ;; > ;; > Do we=20 ;; > ;; > A) allow for three identically named files, dynamically renaming 2 and= 3 ;; > with the Emacs <1> and <2> convention? ;; > ;; > B) ignore 1 and 2, never create them intentionally, and (optionally) ;; > erase duplicates when a file is saved? This is probably easiest to ;; > implement in code. ;; > ;; > C) treat 1 and 2 as backups (there will be a tramp-imap-backups ;; > parameter to control how many are allowed per filename, and older ones ;; > over that limit are erased when a file is saved)? We'll need a ;; > mechanism to revert to a backup, which I don't think Tramp has built-in ;; > at the moment. ;; If 1, 2 and 3 are created only by tramp-imap, then C might be the ;; preferred choice. B doesn't look good to me, I believe all files must ;; be accessible. ;; > Second question: is message 4 ignored? I would prefer to do so, to ;; > allow coexistence of tramp-imap.el with other messages (or even later ;; > versions of tramp-imap.el). ;; Maybe. But then tramp-imap would only access files it has created ;; itself, this might be a restriction. ;; Of course it wouldn't be good if tramp-imap would change subject lines ;; of existing messages. Do we need to encode the filename in the subject ;; line? Couldn't it be any other header, like X-Tramp-Filename? ;; > Third question: namespaces. I feel that it's much better ;; > for the user to store all the files in a single mailbox: ;; [...] ;; Honestly, I'm not so familar with IMAP conventions. I have no ;; precedence for this, you might decide yourself. ;; > Here's the list (omitting the commented-out items from tramp-smb.el) ;; > with some questions and comments. Sorry if most of this is obvious, ;; > this is my first time diving into the Tramp code. What function would ;; > handle the truename-to-visualname translation? ;; There is an "atomic" function tramp-smb-send-command, which makes the ;; communication with the server. Similar functions are ;; tramp-send-command and tramp-send-fish-command in their respective ;; packages. I guess you'll need a tramp-imap-send-command or so; this ;; could be a central place for the mapping. Other places might be needed ;; as well when *interpreting* the result of a sent command. ;; [...] ;; > add-name-to-file: could be a special "link message" or just a copy, li= ke in Windows ;; Both is possible. Btw, on my todo is implementing a real link for ;; Windows, but this has no priority. ;; > copy-file: implemented as an APPEND ;; > delete-directory: implemented with a search+delete for all matching me= ssages ;; OK. ;; > delete-file: search+delete of all matching messages ;; It depends how we decide with your first question. If there is ;; uniqueness in mapping file names to true names, than there is only one ;; message to be deleted. ;; > directory-file-name: tramp-handle-directory-file-name? ;; > directory-files: search for matches ;; OK. ;; > directory-files-and-attributes: search for matches, attributes always = 777 ;; It returns more than just file permissions. ;; > dired-call-process: ignore ;; OK. ;; > dired-compress-file: ignore ;; Likely yes, but there might be an implementation depending on how a ;; message is stored. ;; > file-accessible-directory-p: always t ;; No. Use default implementation (which checks both file-directory-p and ;; file-executable-p). ;; > file-attributes: always 777 ;; No. file-attributes returns a list of 12 different attributes. Must be ;; implemented. ;; > file-directory-p: needs a search, but we could have a file name that c= onflicts with a directory name ;; We shall avoid such conflicts. A file name shall be unique. ;; > file-executable-p: always nil ;; Better is t (see file-accessible-directory-p). It doesn't hurt, ;; because we don't support processes. ;; > file-exists-p: needs a search ;; OK. ;; > file-local-copy: ? ;; That is the basic function for retrieving a file from the server. It ;; stores the file in the temp directory; some other functions use it ;; then (like insert-file-contents etc). ;; > file-remote-p: tramp-handle-file-remote-p ;; OK ;; > file-modes: tramp-handle-file-modes or hard-code ;; Both is possible. tramp-handle-file-modes is shorter to write. ;; > file-name-all-completions: needs a search ;; > file-name-completion: tramp-handle-file-name-completion ;; > file-name-directory: tramp-handle-file-name-directory ;; > file-name-nondirectory: tramp-handle-file-name-nondirectory ;; > file-newer-than-file-p: needs a search, plus backups may matter ;; > file-ownership-preserved-p: ignore ;; OK ;; > file-readable-p: always t ;; > file-regular-p: always t ;; No. You must return nil when the file does not exist. ;; > file-symlink-p: always nil ;; > file-truename: returns UID ;; OK ;; > file-writable-p: always t ;; No, again the case of non existing file names. ;; > find-backup-file-name: we need to decide ;; > insert-directory: ? ;; Both can be postponed, they are not basic functions. ;; > insert-file-contents: search+retrieve ;; file-local-copy and insert-file-contents of that local copy. ;; > load: tramp-handle-load ;; > make-directory, make-directory-internal: we need to decide what this s= hould do ;; > make-symbolic-link: ignore or implement as a special message ;; > rename-file: append with new subject, delete original and its backups = if ;; > we decide to do backups ;; > set-file-modes: ignore ;; > set-visited-file-modtime: we could use a header for this ;; OK ;; > shell-command: ignore? ;; Yes ;; > substitute-in-file-name: ? ;; tramp-handle-substitute-in-file-name ;; > unhandled-file-name-directory: tramp-handle-unhandled-file-name-direct= ory? ;; > vc-registered: always nil ;; OK ;; > verify-visited-file-modtime: ? ;; Maybe ignore for the time being, it's not core functionality. ;; > write-region: needs to do an append+delete of original+backups as need= ed; IMAP can't rewrite a message ;; OK ;; I would start with file-local-copy and write-region, these are the ;; functions for retrieve and store a file. ;; > Thanks ;; > Ted ;; Best regards, Michael. ;;; Code: (require 'imap) (require 'encrypt) (defgroup tramp-imap nil "Tramp over IMAP configuration." :group 'applications) (defcustom tramp-imap-buffer-name "*tramp-imap-buffer" "The buffer name that Tramp-Imap will use." :type 'string :group 'tramp-imap) (defcustom tramp-imap-subject-marker "tramp-imap-subject-marker" "The buffer name that Tramp-Imap will use." :type 'string :group 'tramp-imap) (defun tramp-imap-test () (let ((user "tzz") (server "imap2.lifelogs.com") (password (encrypt-get-passphrase-if-needed "IMAP flow" 'IMAP "ssl" t)) (file-to-insert "/tmp/t") (mailbox "INBOX.test")) (with-current-buffer (imap-open server nil 'ssl) ;;; (debug (imap-search "ALL")) ; see all the messages (imap-authenticate user password) (imap-mailbox-select mailbox) ;;; (debug (tramp-imap-put-file mailbox file-to-insert)) ; insert the cont= ents of file-to-insert in a message (dolist (msg (imap-search=20 (format "SUBJECT %s" tramp-imap-subject-marker))) (debug msg=20 (tramp-imap-get-file msg) )) (imap-close)))) (defun tramp-imap-put-file (mailbox filename) (imap-message-append=20 mailbox (tramp-imap-message-buffer tramp-imap-buffer-name filename))) (defun tramp-imap-get-file (msg) (let ((contents (nth 2 (nth 0 (imap-fetch msg "BODY.PEEK[1]" 'BODYDETAIL)= )))) (with-temp-buffer (insert contents) (encrypt-gpg-decode-buffer=20 (encrypt-get-passphrase-if-needed "Tramp-Imap flow" 'GPG "AES" t)=20 "AES")))) ;;; (tramp-imap-collapse-name "a b c / where ; strange ! characters $ aboun= d")=20 ;;; =3D> "abc/wherestrangecharactersabound" (defun tramp-imap-collapse-name (name) "Return NAME with only [A-Za-z0-9] characters" (replace-regexp-in-string "[^A-Za-z0-9/]" "" name)) ;;;(tramp-imap-message-buffer "tzz-article-buffer" "/tmp/t") (defun tramp-imap-message-buffer (name filename) "Return a new buffer called NAME with encrypted FILENAME" (with-current-buffer (get-buffer-create name) (erase-buffer) (insert (format "From: Tramp-Imap \nSubj= ect: %s %s\n\n" tramp-imap-subject-marker (tramp-imap-collapse-name filename))) =20=20=20=20=20=20 (insert (with-temp-buffer (insert-file-contents filename) (encrypt-gpg-encode-buffer=20 (encrypt-get-passphrase-if-needed "Tramp-Imap flow" 'GPG "AES" t) "AES")))=20=20=20=20 ;;; (debug (buffer-string)) (current-buffer))) (provide 'tramp-imap) ;;; tramp-imap.el ends here --=-=-=--