* file storage in IMAP (eventually for Tramp) working and needs testing @ 2007-12-26 20:02 Ted Zlatanov 2007-12-29 22:34 ` David 2009-04-12 13:04 ` Magnus Henoch 0 siblings, 2 replies; 58+ messages in thread From: Ted Zlatanov @ 2007-12-26 20:02 UTC (permalink / raw) To: Ding Mailing List; +Cc: Simon Josefsson [-- Attachment #1: Type: text/plain, Size: 1923 bytes --] Using imap.el and encrypt.el, I was able to store files in IMAP. imap.el is in the Emacs core now, and encrypt.el can be replaced by pgg.el or something similar if necessary (I personally find it convenient to use encrypt.el but it's not the only way). So this package doesn't depend on Gnus functionality, although it can definitely be used in Gnus. The attached trimp.el (name is not final, but I like it) will eventually provide a Tramp over IMAP interface. For now it's simply a way to put/get files from an IMAP mailbox. Store a file (uses a file name): (trimp-put-file mailbox file-to-insert) Get a file (returns just contents): ; assumes mailbox is already selected (trimp-get-file message-number) The messages are build with a hardcoded headers string for now. There's no message ID header, for instance. Duplicates are allowed for now (this is useful IMO, since it provides a sort of version history, but there will probably be a limit on the number of duplicates). The files are stored as GPG-encrypted (AES cipher) data (encrypt.el allows much more choice, this is just hard-coded for testing). This encryption is, IMO, much easier than the hassle of MIME over IMAP, plus it gives us file encryption. The sacrifice of having to store 1 file per message is worthwhile, again IMO. This is very experimental work, but it does work for me to a Courier IMAP server. If anyone wants to test this out, look at the function trimp-test which encapsulates the current state of my testing. I think trimp.el could be very useful for storing things like marks and passwords on an IMAP server for Gnus; the applications are literally unlimited once it works with Tramp. I need to make sure the IMAP basics are solid before I get to that point, though. That's why I'm not cc-ing the Tramp devel list yet. Simon and other Gnus users, if you could take a look and comment, I would appreciate it. Thanks Ted [-- Attachment #2: trimp.el --] [-- Type: application/emacs-lisp, Size: 3396 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-26 20:02 file storage in IMAP (eventually for Tramp) working and needs testing Ted Zlatanov @ 2007-12-29 22:34 ` David 2007-12-30 20:35 ` Ted Zlatanov 2009-04-12 13:04 ` Magnus Henoch 1 sibling, 1 reply; 58+ messages in thread From: David @ 2007-12-29 22:34 UTC (permalink / raw) To: ding Ted Zlatanov <tzz@lifelogs.com> writes: > Using imap.el and encrypt.el, I was able to store files in IMAP. [...] > I think trimp.el could be very useful for storing things like marks and > passwords on an IMAP server for Gnus; the applications are literally > unlimited once it works with Tramp. I need to make sure the IMAP basics > are solid before I get to that point, though. That's why I'm not cc-ing > the Tramp devel list yet. > > Simon and other Gnus users, if you could take a look and comment, I > would appreciate it. It works for me with the Dovecot IMAP server. Aside from the applications you already mentioned, I think this could be particularly useful for storing drafts on the IMAP server. IMHO this is one feature which Gnus sorely lacks at the moment. As far as I understood, one would have to implement an IMAP file handler so that e.g. auto-saving of drafts can work. Another application would be to save files like .bbdb and .gnus on the IMAP server, which is useful for syncing different Gnus installations. -David ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-29 22:34 ` David @ 2007-12-30 20:35 ` Ted Zlatanov 2007-12-31 10:14 ` Michael Albinus 2007-12-31 17:09 ` David 0 siblings, 2 replies; 58+ messages in thread From: Ted Zlatanov @ 2007-12-30 20:35 UTC (permalink / raw) To: David; +Cc: ding On Sat, 29 Dec 2007 23:34:02 +0100 David <de_bb@arcor.de> wrote: D> Ted Zlatanov <tzz@lifelogs.com> writes: >> Using imap.el and encrypt.el, I was able to store files in IMAP. D> [...] >> I think trimp.el could be very useful for storing things like marks and >> passwords on an IMAP server for Gnus; the applications are literally >> unlimited once it works with Tramp. I need to make sure the IMAP basics >> are solid before I get to that point, though. That's why I'm not cc-ing >> the Tramp devel list yet. >> >> Simon and other Gnus users, if you could take a look and comment, I >> would appreciate it. D> It works for me with the Dovecot IMAP server. Aside from the D> applications you already mentioned, I think this could be particularly D> useful for storing drafts on the IMAP server. IMHO this is one feature D> which Gnus sorely lacks at the moment. As far as I understood, one would D> have to implement an IMAP file handler so that e.g. auto-saving of D> drafts can work. I had not thought about using drafts over Tramp, but I think it's better to support drafts to an IMAP server directly, so you can use them with any other IMAP client (as Thunderbird does by default, for example). D> Another application would be to save files like .bbdb and .gnus on D> the IMAP server, which is useful for syncing different Gnus D> installations. Yes, those would be nice. Splitting rules are another one, as a special case of the .gnus storage. The Gnus registry file is another one, as long as it's not too big. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-30 20:35 ` Ted Zlatanov @ 2007-12-31 10:14 ` Michael Albinus 2007-12-31 12:30 ` Ted Zlatanov 2007-12-31 17:09 ` David 1 sibling, 1 reply; 58+ messages in thread From: Michael Albinus @ 2007-12-31 10:14 UTC (permalink / raw) To: ding; +Cc: David Ted Zlatanov <tzz@lifelogs.com> writes: > I had not thought about using drafts over Tramp, but I think it's better > to support drafts to an IMAP server directly, so you can use them with > any other IMAP client (as Thunderbird does by default, for example). I don't get the point. Tramp is good for storing files somewhere else, this could be a draft folder of an IMAP server(1). How would it prevent other IMAP clients using it? (1) It would need an imap method implementation, of course. > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-31 10:14 ` Michael Albinus @ 2007-12-31 12:30 ` Ted Zlatanov 2007-12-31 13:35 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2007-12-31 12:30 UTC (permalink / raw) To: Michael Albinus; +Cc: ding, David On Mon, 31 Dec 2007 11:14:38 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Ted Zlatanov <tzz@lifelogs.com> writes: >> I had not thought about using drafts over Tramp, but I think it's better >> to support drafts to an IMAP server directly, so you can use them with >> any other IMAP client (as Thunderbird does by default, for example). MA> I don't get the point. Tramp is good for storing files somewhere else, MA> this could be a draft folder of an IMAP server(1). How would it prevent MA> other IMAP clients using it? MA> (1) It would need an imap method implementation, of course. I'm throwing the whole file, PGP-encrypted, into an article, and using minimal headers plus a hacked-up Subject header to make it work. I'm not doing this with interoperability in mind, and I don't think as it stands now it's usable by other IMAP clients. If there's a standard for file storage over IMAP (and I don't mean MIME) I don't know about it. It needs to support deletion, overwriting, unambiguous naming, etc. Gnus, whether accidentally or by design, will actually understand these primitive articles and decode them without Trimp's help, but that was a surprise to me :) Incidentally since I mentioned "unambiguous naming" above, the header currently has the file name stripped of any characters other than [a-zA-Z/] which is nice for searching but will cause conflicts with many file names. I am thinking of using Subject: trimp-subject-marker stripped-filename subject-encoded-filename So the full filename is also visible without getting the article, but the stripped filename lets us do IMAP searches easily. The encoding could be UTF-8. As I said, there's no standard for this AFAIK, so any suggestions are welcome. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-31 12:30 ` Ted Zlatanov @ 2007-12-31 13:35 ` Michael Albinus 2007-12-31 14:52 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Michael Albinus @ 2007-12-31 13:35 UTC (permalink / raw) To: ding; +Cc: David Ted Zlatanov <tzz@lifelogs.com> writes: > MA> I don't get the point. Tramp is good for storing files somewhere else, > MA> this could be a draft folder of an IMAP server(1). How would it prevent > MA> other IMAP clients using it? > > MA> (1) It would need an imap method implementation, of course. > > I'm throwing the whole file, PGP-encrypted, into an article, and using > minimal headers plus a hacked-up Subject header to make it work. I'm > not doing this with interoperability in mind, and I don't think as it > stands now it's usable by other IMAP clients. If there's a standard for > file storage over IMAP (and I don't mean MIME) I don't know about it. > It needs to support deletion, overwriting, unambiguous naming, etc. I don't know the IMAP protocol. But a short scan of RFC 3501 (IMAPv4) gives me the impression one could use the SELECT, EXAMINE, CREATE, DELETE, LIST, RENAME etc commands for mailbox operations. APPEND, EXPUNGE, FETCH, STORE, COPY work over messages of a mailbox. If you regard mailbox as a directory, and a message inside as a file, I see no reason why you cannot apply Emacs' primitive file name handler routines. Of course, the *contents* of the file needs some massage during the transfer as you have said above. > Incidentally since I mentioned "unambiguous naming" above, the header > currently has the file name stripped of any characters other than > [a-zA-Z/] which is nice for searching but will cause conflicts with many > file names. I am thinking of using > > Subject: trimp-subject-marker stripped-filename subject-encoded-filename > > So the full filename is also visible without getting the article, but > the stripped filename lets us do IMAP searches easily. The encoding > could be UTF-8. As I said, there's no standard for this AFAIK, so any > suggestions are welcome. A very simple approach would be to use the message UID or the message sequence number as filename. That might introduce restrictions for searching etc I don't know. But it would prevent ambiguous names. > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-31 13:35 ` Michael Albinus @ 2007-12-31 14:52 ` Ted Zlatanov 2007-12-31 17:14 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2007-12-31 14:52 UTC (permalink / raw) To: Michael Albinus; +Cc: ding On Mon, 31 Dec 2007 14:35:29 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Ted Zlatanov <tzz@lifelogs.com> writes: MA> I don't get the point. Tramp is good for storing files somewhere else, MA> this could be a draft folder of an IMAP server(1). How would it prevent MA> other IMAP clients using it? >> MA> (1) It would need an imap method implementation, of course. >> >> I'm throwing the whole file, PGP-encrypted, into an article, and using >> minimal headers plus a hacked-up Subject header to make it work. I'm >> not doing this with interoperability in mind, and I don't think as it >> stands now it's usable by other IMAP clients. If there's a standard for >> file storage over IMAP (and I don't mean MIME) I don't know about it. >> It needs to support deletion, overwriting, unambiguous naming, etc. MA> I don't know the IMAP protocol. But a short scan of RFC 3501 (IMAPv4) MA> gives me the impression one could use the SELECT, EXAMINE, CREATE, MA> DELETE, LIST, RENAME etc commands for mailbox operations. APPEND, MA> EXPUNGE, FETCH, STORE, COPY work over messages of a mailbox. If you MA> regard mailbox as a directory, and a message inside as a file, I see no MA> reason why you cannot apply Emacs' primitive file name handler MA> routines. Of course, the *contents* of the file needs some massage MA> during the transfer as you have said above. Yes, that's my plan basically. I'm just making up the protocol instead of using something already standardized :) Also I don't want the mailbox to be a directory, you'd end up with a lot of mailboxes. Better to have the IMAP server+folder be the root, and everything under it goes in a single mailbox. >> Incidentally since I mentioned "unambiguous naming" above, the header >> currently has the file name stripped of any characters other than >> [a-zA-Z/] which is nice for searching but will cause conflicts with many >> file names. I am thinking of using >> >> Subject: trimp-subject-marker stripped-filename subject-encoded-filename >> >> So the full filename is also visible without getting the article, but >> the stripped filename lets us do IMAP searches easily. The encoding >> could be UTF-8. As I said, there's no standard for this AFAIK, so any >> suggestions are welcome. MA> A very simple approach would be to use the message UID or the message MA> sequence number as filename. That might introduce restrictions for MA> searching etc I don't know. But it would prevent ambiguous names. That won't work, unfortunately. IMAP UIDs and message sequence IDs can change (if I understand the IMAP protocol correctly) on subsequent sessions. I know at least one IMAP server will do that. Also you can compact a folder and renumber all the messages, which is sometimes necessary. Finally, if the filename is the message UID, that limits us to numerical filenames. I think we need to store metadata in the subject line and do a search+retrieve every time instead of a straight retrieve to grab a single message. Fortunately IMAP is fast on searches so this is a decent compromise. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-31 14:52 ` Ted Zlatanov @ 2007-12-31 17:14 ` Michael Albinus 2007-12-31 17:45 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Michael Albinus @ 2007-12-31 17:14 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding Ted Zlatanov <tzz@lifelogs.com> writes: > Yes, that's my plan basically. I'm just making up the protocol instead > of using something already standardized :) Also I don't want the > mailbox to be a directory, you'd end up with a lot of mailboxes. Better > to have the IMAP server+folder be the root, and everything under it goes > in a single mailbox. Forgive me my ignorance, but what is the difference beetween a folder and a mailbox on an IMAP server? I thought it is the same ... RFC 3501 says: "IMAP4rev1 permits manipulation of mailboxes (remote message folders) in a way that is functionally equivalent to local folders". > MA> A very simple approach would be to use the message UID or the message > MA> sequence number as filename. That might introduce restrictions for > MA> searching etc I don't know. But it would prevent ambiguous names. > > That won't work, unfortunately. IMAP UIDs and message sequence IDs can > change (if I understand the IMAP protocol correctly) on subsequent > sessions. RFC 3501: "The unique identifier of a message MUST NOT change during the session, and SHOULD NOT change between sessions. Any change of unique identifiers between sessions MUST be detectable using the UIDVALIDITY mechanism discussed below. Persistent unique identifiers are required for a client to resynchronize its state from a previous session with the server (e.g., disconnected or offline access clients); this is discussed further in [IMAP-DISC]." OK, if there is a new IMAP session during Emacs' lifetime, opened files must be retrieved again when the IMAP server reports a change. But this shall be possible to handle, maybe with revert-buffer or so. Unsaved files would be a problem, but they would be a problem anyway when an IMAP session has been closed. > I know at least one IMAP server will do that. Also you can > compact a folder and renumber all the messages, which is sometimes > necessary. Finally, if the filename is the message UID, that limits us > to numerical filenames. Not really. You could create a virtual filename in the metadata of the subject line. This name is kept in buffer-file-name. In buffer-file-truename, you could keep the UID of the corresponding message. Reading/writing is performed by your IMAP file name handler functions, which know what to do. You would need kind of file name unicity per mailbox. But this task could be given to Emacs: first, you create the buffer with the virtual file name "Foo" from the subject line. If there is already such a buffer, Emacs will use "Foo<1>" etc. The corresponding buffer-file-name for that buffer is then (concat default-directory "/" (buffer-name)) > I think we need to store metadata in the subject line and do a > search+retrieve every time instead of a straight retrieve to grab a > single message. Fortunately IMAP is fast on searches so this is a > decent compromise. Yes. > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-31 17:14 ` Michael Albinus @ 2007-12-31 17:45 ` Ted Zlatanov 2007-12-31 21:32 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2007-12-31 17:45 UTC (permalink / raw) To: Michael Albinus; +Cc: ding On Mon, 31 Dec 2007 18:14:42 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Ted Zlatanov <tzz@lifelogs.com> writes: >> Yes, that's my plan basically. I'm just making up the protocol instead >> of using something already standardized :) Also I don't want the >> mailbox to be a directory, you'd end up with a lot of mailboxes. Better >> to have the IMAP server+folder be the root, and everything under it goes >> in a single mailbox. MA> Forgive me my ignorance, but what is the difference beetween a folder MA> and a mailbox on an IMAP server? I thought it is the same ... Sorry, my terminology is off. Yes they are the same and I should have been consistent. MA> RFC 3501 says: "IMAP4rev1 permits manipulation of mailboxes (remote MA> message folders) in a way that is functionally equivalent to local MA> folders". I know, but it's ugly (IMO) to do INBOX.a.b.c to store file /a/b/c/d so I'd rather have INBOX.storage effectively act as a filesystem, and /a/b/c/d or /e/f/g/h would both live inside INBOX.storage. OTOH doing mailboxes as directories would simplify file name handling and be a bit more obvious to the users... I'm not sure what's best. My vote is for a single mailbox. MA> A very simple approach would be to use the message UID or the message MA> sequence number as filename. That might introduce restrictions for MA> searching etc I don't know. But it would prevent ambiguous names. >> >> That won't work, unfortunately. IMAP UIDs and message sequence IDs can >> change (if I understand the IMAP protocol correctly) on subsequent >> sessions. MA> RFC 3501: "The unique identifier of a message MUST NOT change during the MA> session, and SHOULD NOT change between sessions. Any change of unique MA> identifiers between sessions MUST be detectable using the UIDVALIDITY MA> mechanism discussed below. Persistent unique identifiers are required MA> for a client to resynchronize its state from a previous session with the MA> server (e.g., disconnected or offline access clients); this is discussed MA> further in [IMAP-DISC]." MA> OK, if there is a new IMAP session during Emacs' lifetime, opened files MA> must be retrieved again when the IMAP server reports a change. But this MA> shall be possible to handle, maybe with revert-buffer or so. Unsaved MA> files would be a problem, but they would be a problem anyway when an MA> IMAP session has been closed. It seems like a lot of work. I don't know imap.el well enough to tell if it can do the dirty work for me and give me a unique identifier. Can anyone tell me? I see UID and ID mixed in the imap.el functions and in the documentation, but don't know how they relate. Right now I'm just using what imap.el gives me as message numbers without knowing their persistence. As I said, at least one IMAP server (Zimbra I think) does strange things with the UIDVALIDITY and Gnus has to resynchronize every session. Is it really a win to use UIDs compared to a SEARCH+RETRIEVE? Why not make the file name be the unique ID of the message? See below for the approach I am proposing. >> I know at least one IMAP server will do that. Also you can >> compact a folder and renumber all the messages, which is sometimes >> necessary. Finally, if the filename is the message UID, that limits us >> to numerical filenames. MA> Not really. You could create a virtual filename in the metadata of the MA> subject line. This name is kept in buffer-file-name. In MA> buffer-file-truename, you could keep the UID of the corresponding MA> message. Reading/writing is performed by your IMAP file name handler MA> functions, which know what to do. MA> You would need kind of file name unicity per mailbox. But this task MA> could be given to Emacs: first, you create the buffer with the virtual MA> file name "Foo" from the subject line. If there is already such a buffer, MA> Emacs will use "Foo<1>" etc. The corresponding buffer-file-name for that MA> buffer is then (concat default-directory "/" (buffer-name)) Why not just use the file name as the identifier, embedded in the Subject header? At worst we'll end up with 2 or more articles with the same file name, which is OK: we just work with the latest one, and the others are backups that can be automatically removed if necessary. Even the Message-ID is not really necessary to identify the article of interest, we just need to do a Subject search for the collapsed name and check the matching articles' Subject headers against the full expansion of the file name. We use the latest article (by the Date header) no matter how many matches there are, which protects us from older versions. I don't know Tramp too well, so if you think the UID approach is better for Tramp integration, I'll go along. Thank you very much for the comments and suggestions. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-31 17:45 ` Ted Zlatanov @ 2007-12-31 21:32 ` Michael Albinus 2008-01-02 20:43 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Michael Albinus @ 2007-12-31 21:32 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding Ted Zlatanov <tzz@lifelogs.com> writes: > MA> RFC 3501 says: "IMAP4rev1 permits manipulation of mailboxes (remote > MA> message folders) in a way that is functionally equivalent to local > MA> folders". > > I know, but it's ugly (IMO) to do INBOX.a.b.c to store file /a/b/c/d so > I'd rather have INBOX.storage effectively act as a filesystem, and > /a/b/c/d or /e/f/g/h would both live inside INBOX.storage. OTOH doing > mailboxes as directories would simplify file name handling and be a bit > more obvious to the users... I'm not sure what's best. My vote is for > a single mailbox. What about using "INBOX.a.b.c" as first level directory from root? This would result in flat directories like "/INBOX.a.b.c/", "/INBOX.d.e.f/" and "/DRAFT.x.y.z/" and no more subdirectories, but so what? Users would recognize their mailbox names. > MA> Not really. You could create a virtual filename in the metadata of the > MA> subject line. This name is kept in buffer-file-name. In > MA> buffer-file-truename, you could keep the UID of the corresponding > MA> message. Reading/writing is performed by your IMAP file name handler > MA> functions, which know what to do. > > MA> You would need kind of file name unicity per mailbox. But this task > MA> could be given to Emacs: first, you create the buffer with the virtual > MA> file name "Foo" from the subject line. If there is already such a buffer, > MA> Emacs will use "Foo<1>" etc. The corresponding buffer-file-name for that > MA> buffer is then (concat default-directory "/" (buffer-name)) > > Why not just use the file name as the identifier, embedded in the > Subject header? At worst we'll end up with 2 or more articles with the > same file name, which is OK: we just work with the latest one, and the > others are backups that can be automatically removed if necessary. Even > the Message-ID is not really necessary to identify the article of > interest, we just need to do a Subject search for the collapsed name and > check the matching articles' Subject headers against the full expansion > of the file name. We use the latest article (by the Date header) no > matter how many matches there are, which protects us from older > versions. I'm a little bit lost. Does it mean you don't want to offer *all* messages in a mailbox? > I don't know Tramp too well, so if you think the UID approach is better > for Tramp integration, I'll go along. Oh, there is no Tramp specific meaning how a file name should look like. In fact I've only tried to sketch a scenario where file names visible to the user are derived from the virtual file name kept in the subject header line of the message, and the "file name" you speak with IMAP is the UID. Solving also the problem of file name unicity. The more serious work with Tramp is that all file operations towards IMAP must be mapped to Emacs' file name handler primitives like file-local-copy or write-region. If you want, I could support you a little bit(1) in writing a tramp-imap.el. > Ted Best regards, Michael. (1): As usual, I'm busy with other projects. Currently the D-Bus integration into Emacs. But you've attracted me, somehow. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-31 21:32 ` Michael Albinus @ 2008-01-02 20:43 ` Ted Zlatanov 2008-01-02 21:40 ` Reiner Steib 2008-01-02 23:50 ` Michael Albinus 0 siblings, 2 replies; 58+ messages in thread From: Ted Zlatanov @ 2008-01-02 20:43 UTC (permalink / raw) To: Michael Albinus; +Cc: ding On Mon, 31 Dec 2007 22:32:01 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Ted Zlatanov <tzz@lifelogs.com> writes: MA> RFC 3501 says: "IMAP4rev1 permits manipulation of mailboxes (remote MA> message folders) in a way that is functionally equivalent to local MA> folders". >> >> I know, but it's ugly (IMO) to do INBOX.a.b.c to store file /a/b/c/d so >> I'd rather have INBOX.storage effectively act as a filesystem, and >> /a/b/c/d or /e/f/g/h would both live inside INBOX.storage. OTOH doing >> mailboxes as directories would simplify file name handling and be a bit >> more obvious to the users... I'm not sure what's best. My vote is for >> a single mailbox. MA> What about using "INBOX.a.b.c" as first level directory from root? This MA> would result in flat directories like "/INBOX.a.b.c/", "/INBOX.d.e.f/" MA> and "/DRAFT.x.y.z/" and no more subdirectories, but so what? Users would MA> recognize their mailbox names. I'm not sure what you're saying, sorry. MA> I'm a little bit lost. Does it mean you don't want to offer *all* MA> messages in a mailbox? Correct, otherwise it's hard to handle invalid messages: are they invalid files or something else? I wanted also to treat duplicates as automatic backups but if you don't like that idea then I'll drop it. MA> The more serious work with Tramp is that all file operations towards MA> IMAP must be mapped to Emacs' file name handler primitives like MA> file-local-copy or write-region. If you want, I could support you a MA> little bit(1) in writing a tramp-imap.el. Sure. In fact trimp.el is tramp-imap.el, shortened :) The name is not important, if you want the latter. Should it live in Tramp instead of Gnus? If so, imap.el will be needed, and it's only in Emacs after 22 I believe. Any opinions (Reiner?) are welcome. Let me know what pieces you need from me, and if there's an example in Tramp I should follow to implement this. MA> (1): As usual, I'm busy with other projects. Currently the D-Bus MA> integration into Emacs. But you've attracted me, somehow. I appreciate your interest, and it's certainly nice to have your help. Thanks Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-02 20:43 ` Ted Zlatanov @ 2008-01-02 21:40 ` Reiner Steib 2008-01-02 22:15 ` Ted Zlatanov 2008-01-02 23:50 ` Michael Albinus 1 sibling, 1 reply; 58+ messages in thread From: Reiner Steib @ 2008-01-02 21:40 UTC (permalink / raw) To: Ted Zlatanov; +Cc: Michael Albinus, ding [ The following message is a courtesy copy of an article that has been posted to news:gmane.emacs.gnus.general as well. ] On Wed, Jan 02 2008, Ted Zlatanov wrote: > Michael Albinus <michael.albinus@gmx.de> wrote: > MA> The more serious work with Tramp is that all file operations towards > MA> IMAP must be mapped to Emacs' file name handler primitives like > MA> file-local-copy or write-region. If you want, I could support you a > MA> little bit(1) in writing a tramp-imap.el. > > Sure. In fact trimp.el is tramp-imap.el, shortened :) The name is not > important, if you want the latter. Should it live in Tramp instead of > Gnus? If so, imap.el will be needed, and it's only in Emacs after 22 I > believe. Any opinions (Reiner?) are welcome. Emacs 21 (e.g. Gnus 5.9) already has imap.el. Of course not a current version, so I don't know if it would be sufficient for trimp.el/tramp-imap.el. But I don't think this matters much. A library for "file storage in IMAP" should probably be included in Emacs, either as part of tramp or independently, but not as part of Gnus (making it possible to save drafts on IMAP would be a very nice feature for Gnus, of course!). So you should discuss it on emacs-devel sooner or later. Bye, Reiner. -- ,,, (o o) ---ooO-(_)-Ooo--- | PGP key available | http://rsteib.home.pages.de/ ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-02 21:40 ` Reiner Steib @ 2008-01-02 22:15 ` Ted Zlatanov 0 siblings, 0 replies; 58+ messages in thread From: Ted Zlatanov @ 2008-01-02 22:15 UTC (permalink / raw) To: ding On Wed, 02 Jan 2008 22:40:12 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: RS> Emacs 21 (e.g. Gnus 5.9) already has imap.el. Of course not a RS> current version, so I don't know if it would be sufficient for RS> trimp.el/tramp-imap.el. But I don't think this matters much. A RS> library for "file storage in IMAP" should probably be included in RS> Emacs, either as part of tramp or independently, but not as part of RS> Gnus (making it possible to save drafts on IMAP would be a very nice RS> feature for Gnus, of course!). So you should discuss it on RS> emacs-devel sooner or later. All right, I'll work with Michael Albinus to put this functionality in Tramp and from there it will go into Emacs. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-02 20:43 ` Ted Zlatanov 2008-01-02 21:40 ` Reiner Steib @ 2008-01-02 23:50 ` Michael Albinus 2008-01-03 11:32 ` Reiner Steib 2008-01-03 13:21 ` Ted Zlatanov 1 sibling, 2 replies; 58+ messages in thread From: Michael Albinus @ 2008-01-02 23:50 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, Tramp Development List Ted Zlatanov <tzz@lifelogs.com> writes: > MA> I'm a little bit lost. Does it mean you don't want to offer *all* > MA> messages in a mailbox? > > Correct, otherwise it's hard to handle invalid messages: are they > invalid files or something else? I wanted also to treat duplicates as > automatic backups but if you don't like that idea then I'll drop it. Maybe we should see real life examples. I don't know whether it is always good to present selected contents only. If there are technical restrictions - that's another game. > MA> The more serious work with Tramp is that all file operations towards > MA> IMAP must be mapped to Emacs' file name handler primitives like > MA> file-local-copy or write-region. If you want, I could support you a > MA> little bit(1) in writing a tramp-imap.el. > > Sure. In fact trimp.el is tramp-imap.el, shortened :) The name is not > important, if you want the latter. Should it live in Tramp instead of > Gnus? If so, imap.el will be needed, and it's only in Emacs after 22 I > believe. Any opinions (Reiner?) are welcome. > > Let me know what pieces you need from me, and if there's an example in > Tramp I should follow to implement this. It would be nice if it could live in the Tramp repository. I'm not fundamentalistic about names, but so far we have just tramp*.el files there. The only "piece" I would need from you are legal papers from the FSF for Tramp and for Emacs. Guess you have it partly already because of gnus. Once this is done, nothing will prevent you from writing in the Tramp CVS. As starting point you might look at tramp-smb.el or tramp-gw.el. Both are addons, like tramp-imap.el could be. tramp-fish.el might be examined as well, but this method isn't used anywhere I believe - it was merely a proof of concept I didn't want to throw away. And I never ever got a bug report about ... It wouldn't be a problem if imap.el is not present in all (X)Emacs versions to be supported. tramp-gw.el lives with similar restrictions; it is loaded only if some additional (external) packages are available. I've added tramp-devel to the Cc, in order to answer the question from your other message. > Thanks > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-02 23:50 ` Michael Albinus @ 2008-01-03 11:32 ` Reiner Steib 2008-01-03 12:43 ` Ted Zlatanov 2008-01-03 13:21 ` Ted Zlatanov 1 sibling, 1 reply; 58+ messages in thread From: Reiner Steib @ 2008-01-03 11:32 UTC (permalink / raw) To: Michael Albinus; +Cc: Ted Zlatanov, ding, Tramp Development List On Thu, Jan 03 2008, Michael Albinus wrote: > The only "piece" I would need from you are legal papers from the FSF for > Tramp and for Emacs. Guess you have it partly already because of gnus. Ted (Teodor Zlatanov) already has an assignment for Emacs on file. I'd guess this covers Tramp as well, doesn't it? (Like an assignment for Emacs covers Gnus.) Bye, Reiner. -- ,,, (o o) ---ooO-(_)-Ooo--- | PGP key available | http://rsteib.home.pages.de/ ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-03 11:32 ` Reiner Steib @ 2008-01-03 12:43 ` Ted Zlatanov 2008-01-03 12:56 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2008-01-03 12:43 UTC (permalink / raw) To: Reiner Steib; +Cc: Michael Albinus, ding, Tramp Development List On Thu, 03 Jan 2008 12:32:44 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: RS> On Thu, Jan 03 2008, Michael Albinus wrote: >> The only "piece" I would need from you are legal papers from the FSF for >> Tramp and for Emacs. Guess you have it partly already because of gnus. RS> Ted (Teodor Zlatanov) already has an assignment for Emacs on file. RS> I'd guess this covers Tramp as well, doesn't it? RS> (Like an assignment for Emacs covers Gnus.) I've submitted Emacs papers and Gnus papers, so no worries either way. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-03 12:43 ` Ted Zlatanov @ 2008-01-03 12:56 ` Michael Albinus 0 siblings, 0 replies; 58+ messages in thread From: Michael Albinus @ 2008-01-03 12:56 UTC (permalink / raw) To: Ted Zlatanov; +Cc: Tramp Development List, ding, Reiner Steib Ted Zlatanov <tzz@lifelogs.com> writes: > I've submitted Emacs papers and Gnus papers, so no worries either way. Thanks. I have no idea whether an additional Tramp paper is needed then, we will ask the FSF. However, it is reasonable for me to give you write access to the Tramp CVS repository. I've just added you on savannah. > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-02 23:50 ` Michael Albinus 2008-01-03 11:32 ` Reiner Steib @ 2008-01-03 13:21 ` Ted Zlatanov 2008-01-03 13:27 ` Ted Zlatanov ` (2 more replies) 1 sibling, 3 replies; 58+ messages in thread From: Ted Zlatanov @ 2008-01-03 13:21 UTC (permalink / raw) To: Michael Albinus; +Cc: ding, Tramp Development List On Thu, 03 Jan 2008 00:50:56 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Ted Zlatanov <tzz@lifelogs.com> writes: MA> I'm a little bit lost. Does it mean you don't want to offer *all* MA> messages in a mailbox? >> >> Correct, otherwise it's hard to handle invalid messages: are they >> invalid files or something else? I wanted also to treat duplicates as >> automatic backups but if you don't like that idea then I'll drop it. MA> Maybe we should see real life examples. I don't know whether it is MA> always good to present selected contents only. If there are technical MA> restrictions - that's another game. (I'm leaving Ding on the CC in case anyone has comments) Say you have these article UIDs and subjects, and UIDs are ordered by date. Note the ENCODED_FILENAME will be the filename encoded in some way and /a/b/c is the filename without any risky characters; Gnus has the subject encoding functionality already. The subject format is specifically to allow easy searching. 1 "tramp-imap-marker /a/b/c ENCODED_ABC" 2 "tramp-imap-marker /a/b/c ENCODED_ABC" 3 "tramp-imap-marker /a/b/c ENCODED_ABC" 4 "hello world" Do we A) allow for three identically named files, dynamically renaming 2 and 3 with the Emacs <1> and <2> convention? B) ignore 1 and 2, never create them intentionally, and (optionally) erase duplicates when a file is saved? This is probably easiest to implement in code. C) treat 1 and 2 as backups (there will be a tramp-imap-backups parameter to control how many are allowed per filename, and older ones over that limit are erased when a file is saved)? We'll need a mechanism to revert to a backup, which I don't think Tramp has built-in at the moment. Second question: is message 4 ignored? I would prefer to do so, to allow coexistence of tramp-imap.el with other messages (or even later versions of tramp-imap.el). I think it's fine to use message UIDs as the true file name, but we need to decide what to do in the cases above. Third question: namespaces. I feel that it's much better for the user to store all the files in a single mailbox: INBOX.storage holds /a/b/c, /d/e/f, and /g/h/i I believe you proposed that instead we should auto-create: INBOX.storage.a.b to hold /a/b/c INBOX.storage.d.e to hold /d/e/f INBOX.storage.g.h to hold /g/h/i Did I understand you correctly? Maybe we could do both, allowing for a "root mailbox" and a "root prefix". In the first case, those would be INBOX.storage / and in the second case, they would be INBOX.storage.a.b /a/b INBOX.storage.d.e /d/e INBOX.storage.g.h /g/h And maybe the user can configure those mappings exactly. I still think it's too complicated for 99% of the users and they'll never need more than one mailbox per virtual filesystem, but if you disagree I can do the extra work. I'll assume we've picked the single mailbox approach for the rest of the message. The implementations will change quite a bit as far as directories are concerned if we use multiple mailboxes. MA> As starting point you might look at tramp-smb.el or tramp-gw.el. Both MA> are addons, like tramp-imap.el could be. tramp-fish.el might be examined MA> as well, but this method isn't used anywhere I believe - it was merely a MA> proof of concept I didn't want to throw away. And I never ever got a bug MA> report about ... OK, I'll look. tramp-gw.el doesn't have any Emacs primitives implemented at first glance, so I'll look at tramp-smb.el which defines all those mappings nicely in tramp-smb-file-name-handler-alist. That list has quite a few methods; are they all necessary or do the default handlers for some of them suffice? Here's the list (omitting the commented-out items from tramp-smb.el) with some questions and comments. Sorry if most of this is obvious, this is my first time diving into the Tramp code. What function would handle the truename-to-visualname translation? add-name-to-file: could be a special "link message" or just a copy, like in Windows copy-file: implemented as an APPEND delete-directory: implemented with a search+delete for all matching messages delete-file: search+delete of all matching messages directory-file-name: tramp-handle-directory-file-name? directory-files: search for matches directory-files-and-attributes: search for matches, attributes always 777 dired-call-process: ignore dired-compress-file: ignore file-accessible-directory-p: always t file-attributes: always 777 file-directory-p: needs a search, but we could have a file name that conflicts with a directory name file-executable-p: always nil file-exists-p: needs a search file-local-copy: ? file-remote-p: tramp-handle-file-remote-p file-modes: tramp-handle-file-modes or hard-code file-name-all-completions: needs a search file-name-completion: tramp-handle-file-name-completion file-name-directory: tramp-handle-file-name-directory file-name-nondirectory: tramp-handle-file-name-nondirectory file-newer-than-file-p: needs a search, plus backups may matter file-ownership-preserved-p: ignore file-readable-p: always t file-regular-p: always t file-symlink-p: always nil file-truename: returns UID file-writable-p: always t find-backup-file-name: we need to decide insert-directory: ? insert-file-contents: search+retrieve load: tramp-handle-load make-directory, make-directory-internal: we need to decide what this should do make-symbolic-link: ignore or implement as a special message rename-file: append with new subject, delete original and its backups if we decide to do backups set-file-modes: ignore set-visited-file-modtime: we could use a header for this shell-command: ignore? substitute-in-file-name: ? unhandled-file-name-directory: tramp-handle-unhandled-file-name-directory? vc-registered: always nil verify-visited-file-modtime: ? write-region: needs to do an append+delete of original+backups as needed; IMAP can't rewrite a message Thanks Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-03 13:21 ` Ted Zlatanov @ 2008-01-03 13:27 ` Ted Zlatanov 2008-01-03 17:31 ` James Cloos 2008-01-04 11:00 ` Michael Albinus 2 siblings, 0 replies; 58+ messages in thread From: Ted Zlatanov @ 2008-01-03 13:27 UTC (permalink / raw) To: ding, Tramp Development List On Thu, 03 Jan 2008 07:21:42 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: TZ> Third question: namespaces. I feel that it's much better TZ> for the user to store all the files in a single mailbox: TZ> INBOX.storage holds /a/b/c, /d/e/f, and /g/h/i TZ> I believe you proposed that instead we should auto-create: TZ> INBOX.storage.a.b to hold /a/b/c TZ> INBOX.storage.d.e to hold /d/e/f TZ> INBOX.storage.g.h to hold /g/h/i I realized there is a third option: don't allow directories. Everything is at the top level /, and make-directory fails. That would simplify the code tremendously though users may not like it. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-03 13:21 ` Ted Zlatanov 2008-01-03 13:27 ` Ted Zlatanov @ 2008-01-03 17:31 ` James Cloos 2008-01-03 17:42 ` Ted Zlatanov 2008-01-04 11:00 ` Michael Albinus 2 siblings, 1 reply; 58+ messages in thread From: James Cloos @ 2008-01-03 17:31 UTC (permalink / raw) To: Ted Zlatanov; +Cc: Michael Albinus, ding, Tramp Development List On namespaces in IMAP, do note that not every imapd uses . (full stop) as the hierarchy separator character; many use / (virgule). The LIST command will show what that server uses. Also, some (many?) users will probably not like the idea of turning INBOX into a parent. That convention seems to be a uw thing, AFAICT. Were I to use this I’d want to put it all under something like Emacs/Tramp/ or so. -JimC -- James Cloos <cloos@jhcloos.com> OpenPGP: 1024D/ED7DAEA6 ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-03 17:31 ` James Cloos @ 2008-01-03 17:42 ` Ted Zlatanov 0 siblings, 0 replies; 58+ messages in thread From: Ted Zlatanov @ 2008-01-03 17:42 UTC (permalink / raw) To: ding, Tramp Development List On Thu, 03 Jan 2008 17:31:14 +0000 James Cloos <cloos+math_uh-ding@jhcloos.com> wrote: JC> On namespaces in IMAP, do note that not every imapd uses . (full stop) JC> as the hierarchy separator character; many use / (virgule). The LIST JC> command will show what that server uses. Right, I did not mention that but knew it :) JC> Also, some (many?) users will probably not like the idea of turning JC> INBOX into a parent. That convention seems to be a uw thing, AFAICT. We'll definitely have any mailbox be the top parent (not just INBOX), the question is whether other mailboxes under it will implement a directory hierarchy, or if the hierarchy will be entirely inside a single mailbox, or if we won't have directories at all. Thanks Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2008-01-03 13:21 ` Ted Zlatanov 2008-01-03 13:27 ` Ted Zlatanov 2008-01-03 17:31 ` James Cloos @ 2008-01-04 11:00 ` Michael Albinus 2 siblings, 0 replies; 58+ messages in thread From: Michael Albinus @ 2008-01-04 11:00 UTC (permalink / raw) To: Ted Zlatanov; +Cc: Tramp Development List, ding Ted Zlatanov <tzz@lifelogs.com> writes: > Say you have these article UIDs and subjects, and UIDs are ordered by > date. Note the ENCODED_FILENAME will be the filename encoded in some > way and /a/b/c is the filename without any risky characters; Gnus has > the subject encoding functionality already. The subject format is > specifically to allow easy searching. > > 1 "tramp-imap-marker /a/b/c ENCODED_ABC" > 2 "tramp-imap-marker /a/b/c ENCODED_ABC" > 3 "tramp-imap-marker /a/b/c ENCODED_ABC" > 4 "hello world" > > Do we > > A) allow for three identically named files, dynamically renaming 2 and 3 > with the Emacs <1> and <2> convention? > > B) ignore 1 and 2, never create them intentionally, and (optionally) > erase duplicates when a file is saved? This is probably easiest to > implement in code. > > C) treat 1 and 2 as backups (there will be a tramp-imap-backups > parameter to control how many are allowed per filename, and older ones > over that limit are erased when a file is saved)? We'll need a > mechanism to revert to a backup, which I don't think Tramp has built-in > at the moment. If 1, 2 and 3 are created only by tramp-imap, then C might be the preferred choice. B doesn't look good to me, I believe all files must be accessible. > Second question: is message 4 ignored? I would prefer to do so, to > allow coexistence of tramp-imap.el with other messages (or even later > versions of tramp-imap.el). Maybe. But then tramp-imap would only access files it has created itself, this might be a restriction. Of course it wouldn't be good if tramp-imap would change subject lines of existing messages. Do we need to encode the filename in the subject line? Couldn't it be any other header, like X-Tramp-Filename? > Third question: namespaces. I feel that it's much better > for the user to store all the files in a single mailbox: [...] Honestly, I'm not so familar with IMAP conventions. I have no precedence for this, you might decide yourself. > Here's the list (omitting the commented-out items from tramp-smb.el) > with some questions and comments. Sorry if most of this is obvious, > this is my first time diving into the Tramp code. What function would > handle the truename-to-visualname translation? There is an "atomic" function tramp-smb-send-command, which makes the communication with the server. Similar functions are tramp-send-command and tramp-send-fish-command in their respective packages. I guess you'll need a tramp-imap-send-command or so; this could be a central place for the mapping. Other places might be needed as well when *interpreting* the result of a sent command. [...] > add-name-to-file: could be a special "link message" or just a copy, like in Windows Both is possible. Btw, on my todo is implementing a real link for Windows, but this has no priority. > copy-file: implemented as an APPEND > delete-directory: implemented with a search+delete for all matching messages OK. > delete-file: search+delete of all matching messages It depends how we decide with your first question. If there is uniqueness in mapping file names to true names, than there is only one message to be deleted. > directory-file-name: tramp-handle-directory-file-name? > directory-files: search for matches OK. > directory-files-and-attributes: search for matches, attributes always 777 It returns more than just file permissions. > dired-call-process: ignore OK. > dired-compress-file: ignore Likely yes, but there might be an implementation depending on how a message is stored. > file-accessible-directory-p: always t No. Use default implementation (which checks both file-directory-p and file-executable-p). > file-attributes: always 777 No. file-attributes returns a list of 12 different attributes. Must be implemented. > file-directory-p: needs a search, but we could have a file name that conflicts with a directory name We shall avoid such conflicts. A file name shall be unique. > file-executable-p: always nil Better is t (see file-accessible-directory-p). It doesn't hurt, because we don't support processes. > file-exists-p: needs a search OK. > file-local-copy: ? That is the basic function for retrieving a file from the server. It stores the file in the temp directory; some other functions use it then (like insert-file-contents etc). > file-remote-p: tramp-handle-file-remote-p OK > file-modes: tramp-handle-file-modes or hard-code Both is possible. tramp-handle-file-modes is shorter to write. > file-name-all-completions: needs a search > file-name-completion: tramp-handle-file-name-completion > file-name-directory: tramp-handle-file-name-directory > file-name-nondirectory: tramp-handle-file-name-nondirectory > file-newer-than-file-p: needs a search, plus backups may matter > file-ownership-preserved-p: ignore OK > file-readable-p: always t > file-regular-p: always t No. You must return nil when the file does not exist. > file-symlink-p: always nil > file-truename: returns UID OK > file-writable-p: always t No, again the case of non existing file names. > find-backup-file-name: we need to decide > insert-directory: ? Both can be postponed, they are not basic functions. > insert-file-contents: search+retrieve file-local-copy and insert-file-contents of that local copy. > load: tramp-handle-load > make-directory, make-directory-internal: we need to decide what this should do > make-symbolic-link: ignore or implement as a special message > rename-file: append with new subject, delete original and its backups if > we decide to do backups > set-file-modes: ignore > set-visited-file-modtime: we could use a header for this OK > shell-command: ignore? Yes > substitute-in-file-name: ? tramp-handle-substitute-in-file-name > unhandled-file-name-directory: tramp-handle-unhandled-file-name-directory? > vc-registered: always nil OK > verify-visited-file-modtime: ? Maybe ignore for the time being, it's not core functionality. > write-region: needs to do an append+delete of original+backups as needed; IMAP can't rewrite a message OK I would start with file-local-copy and write-region, these are the functions for retrieve and store a file. > Thanks > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-30 20:35 ` Ted Zlatanov 2007-12-31 10:14 ` Michael Albinus @ 2007-12-31 17:09 ` David 1 sibling, 0 replies; 58+ messages in thread From: David @ 2007-12-31 17:09 UTC (permalink / raw) To: ding Ted Zlatanov <tzz@lifelogs.com> writes: > I had not thought about using drafts over Tramp, but I think it's better > to support drafts to an IMAP server directly, so you can use them with > any other IMAP client (as Thunderbird does by default, for example). Yes, that would the best thing to do, but so far no one has coded this. I only skimmed through nndraft.el and I don't really understand how this all integrates with the Emacs file handlers, but since no one has made nndraft.el to work over IMAP yet, it seems not a trivial thing to do. If Tramp would support file storage over IMAP, it should be fairly easy to implement things like auto-saving of unfinished message buffers on an IMAP server. Even if continuing these messages only works with Gnus, it's still better than nothing. By the way, your way of storing files over IMAP seems to be more compatible than one might think - I just tested it with Thunderbird with the Enigmail plugin and it showed the files right away (this doesn't help for editing drafts, though). -David PS: I read ding over gmane, so no need to CC me. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2007-12-26 20:02 file storage in IMAP (eventually for Tramp) working and needs testing Ted Zlatanov 2007-12-29 22:34 ` David @ 2009-04-12 13:04 ` Magnus Henoch 2009-04-13 15:50 ` Ted Zlatanov 1 sibling, 1 reply; 58+ messages in thread From: Magnus Henoch @ 2009-04-12 13:04 UTC (permalink / raw) To: ding; +Cc: tramp-devel Ted Zlatanov <tzz@lifelogs.com> writes: > Using imap.el and encrypt.el, I was able to store files in IMAP. > [...] A year later, I had the same idea and found this article :) What happened to this? Is there a more complete trimp.el / tramp-imap.el that I can start hacking at? [Followup-To set to tramp list] Magnus ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-12 13:04 ` Magnus Henoch @ 2009-04-13 15:50 ` Ted Zlatanov 2009-04-14 8:44 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-04-13 15:50 UTC (permalink / raw) To: Magnus Henoch; +Cc: ding, tramp-devel On Sun, 12 Apr 2009 15:04:24 +0200 Magnus Henoch <mange@freemail.hu> wrote: MH> Ted Zlatanov <tzz@lifelogs.com> writes: >> Using imap.el and encrypt.el, I was able to store files in IMAP. >> [...] MH> A year later, I had the same idea and found this article :) What MH> happened to this? Is there a more complete trimp.el / tramp-imap.el MH> that I can start hacking at? No news, unfortunately, I haven't had the time to play with it. It was next on my TODO list, actually, but that has a 1+ month granulatiry :) I was about to start adding the TRAMP compatibility functions that Michael Albinus kindly told me were needed. The underlying store/fetch functionality should be all right, except it doesn't do attachments so the message is always one big text chunk. You could, instead, make the message contain metadata and put the payload in an attachment. It's all yours if you want it. I will gladly help with testing and whatever else you need. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-13 15:50 ` Ted Zlatanov @ 2009-04-14 8:44 ` Michael Albinus 2009-04-14 16:22 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Michael Albinus @ 2009-04-14 8:44 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, Magnus Henoch, ding Ted Zlatanov <tzz@lifelogs.com> writes: > It's all yours if you want it. I will gladly help with testing and > whatever else you need. As a first step, you could add the current version to the Tramp CVS repository. This might ease to work on it. > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-14 8:44 ` Michael Albinus @ 2009-04-14 16:22 ` Ted Zlatanov 2009-04-15 21:03 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-04-14 16:22 UTC (permalink / raw) To: ding; +Cc: tramp-devel [-- Attachment #1: Type: text/plain, Size: 733 bytes --] On Tue, 14 Apr 2009 10:44:56 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Ted Zlatanov <tzz@lifelogs.com> writes: >> It's all yours if you want it. I will gladly help with testing and >> whatever else you need. MA> As a first step, you could add the current version to the Tramp CVS MA> repository. This might ease to work on it. It's really, really embarassing to check that messy pile of code into Tramp. Most of it is just TODO information. I am attaching it for yours and Magnus' review. The encrypt-* calls should be replaced by auth-source-* calls, as well. If you really want me to check it in, I will (and if this is a sneaky way to get me to fix it up and write the missing code, it's working :) Ted [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 10760 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-14 16:22 ` Ted Zlatanov @ 2009-04-15 21:03 ` Michael Albinus 2009-04-23 0:15 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Michael Albinus @ 2009-04-15 21:03 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding Ted Zlatanov <tzz@lifelogs.com> writes: > It's really, really embarassing to check that messy pile of code into > Tramp. Most of it is just TODO information. I am attaching it for > yours and Magnus' review. The encrypt-* calls should be replaced by > auth-source-* calls, as well. No problem for me if you check it in. As long as we don't add it to the makefiles, and we don't load it in tramp.el, it doesn't hurt. Please adjust the copyright years, and replace GPLv2 by GPLv3. > If you really want me to check it in, I > will (and if this is a sneaky way to get me to fix it up and write the > missing code, it's working :) This sounds like a good plan :-) Anyway, I will comment the code in more detail next days, likely. > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-15 21:03 ` Michael Albinus @ 2009-04-23 0:15 ` Ted Zlatanov 2009-04-23 2:45 ` Daiki Ueno 2009-04-23 21:30 ` Ted Zlatanov 0 siblings, 2 replies; 58+ messages in thread From: Ted Zlatanov @ 2009-04-23 0:15 UTC (permalink / raw) To: ding; +Cc: tramp-devel [-- Attachment #1: Type: text/plain, Size: 2740 bytes --] On Wed, 15 Apr 2009 23:03:16 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: MA> No problem for me if you check it in. As long as we don't add it to the MA> makefiles, and we don't load it in tramp.el, it doesn't hurt. MA> Please adjust the copyright years, and replace GPLv2 by GPLv3. Let's wait until it at least does completion, reads, and writes, please. I made the year and GPL fixes. The current code will: - install many handlers, many unimplemented (copied from tramp-fish.el) - handle "/imap:blockstar.com:/INBOX.test/" for filename expansion (but the actual expansion is not working; tramp-imap-get-file-entries does not get the correct file name from Emacs) - save /imap:blockstar.com:/INBOX.test/new via tramp-imap-put-file (uses PGG) - read /imap:blockstar.com:/INBOX.test/new via tramp-imap-get-file (uses PGG) - use auth-source to get user name and password for a server - maintain an IMAP connection in the Tramp buffer specific for this connection - show you how much nasty code I can produce in a few hours My biggest problem right now is that I can't get filename completion to work (as mentioned above, tramp-imap-get-file-entries does not get the right name). I'm sure I'm missing a lot of details here, but ideally I should parse the filename myself without any interferences from Emacs or from Tramp (everything else--port, username, server, etc.--can be parsed by Tramp). Here's how I want the parsing to work: /imap:SERVER:/MAILBOX/ hitting TAB after MAILBOX or after MAILBOX/ should complete "/imap:SERVER:/MAILBOX" with or without the trailing slash. MAILBOX is the directory. /imap:SERVER:/MAILBOX/anything hitting TAB at the end should complete all the messages that match "anything". /imap:SERVER:/MAILBOX/anything/else The mailbox here will be "MAILBOX/anything" and completion will look for messages that match "else" So basically the mailbox name is everything up to the last / character, like a `dirname' call. IMAP mailboxes can contain many characters, including /, so we need to forbid those characters in the file name. The base file name is then just `basename' on the full name. I tried to implement this by hand in tramp-imap-file-name-mailbox-or-name but there must be a better way. My second problem is that messages are not yet sorted by date so you are not guaranteed to get the latest version (we agreed not to delete the older versions, though there probably should be a flexible way to mark them deleted past N backups). Third, ranges are not supported for most commands. The last major issue is that error handling and PGG integration are not good. Usability is not good. If anyone can help me with the above items, I'd appreciate it. Thanks Ted [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 25900 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 0:15 ` Ted Zlatanov @ 2009-04-23 2:45 ` Daiki Ueno 2009-04-23 12:51 ` Ted Zlatanov 2009-04-23 21:30 ` Ted Zlatanov 1 sibling, 1 reply; 58+ messages in thread From: Daiki Ueno @ 2009-04-23 2:45 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, tramp-devel >>>>> In <86k55cb4ye.fsf@jumptrading.com> >>>>> Ted Zlatanov <tzz@lifelogs.com> wrote: > On Wed, 15 Apr 2009 23:03:16 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: > - save /imap:blockstar.com:/INBOX.test/new via tramp-imap-put-file (uses PGG) > - read /imap:blockstar.com:/INBOX.test/new via tramp-imap-get-file (uses PGG) I think this (implicit encryption) is a misdesgined feature which should be removed. Why don't you leave the job to auto-encryption-mode so that a user can choose whether to encrypt a file by adding ".gpg"? Regards, -- Daiki Ueno ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 2:45 ` Daiki Ueno @ 2009-04-23 12:51 ` Ted Zlatanov 2009-04-23 13:09 ` Daiki Ueno 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-04-23 12:51 UTC (permalink / raw) To: ding; +Cc: tramp-devel On Thu, 23 Apr 2009 11:45:04 +0900 Daiki Ueno <ueno@unixuser.org> wrote: >>>>>> In <86k55cb4ye.fsf@jumptrading.com> >>>>>> Ted Zlatanov <tzz@lifelogs.com> wrote: >> On Wed, 15 Apr 2009 23:03:16 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: >> - save /imap:blockstar.com:/INBOX.test/new via tramp-imap-put-file (uses PGG) >> - read /imap:blockstar.com:/INBOX.test/new via tramp-imap-get-file (uses PGG) DU> I think this (implicit encryption) is a misdesgined feature which should DU> be removed. Why don't you leave the job to auto-encryption-mode so that DU> a user can choose whether to encrypt a file by adding ".gpg"? Unlike filesystems, IMAP servers are a semi-public space, in the sense that the vast majority of the users can't expect security or privacy beyond what the server administrators and owners are willing to grant. Very, very few users actually own and control their IMAP servers. Thus, encrypting by default is IMHO the best option and the only one I provided in this initial version. I disagree the user should choose whether to encrypt, especially by the burdensome mechanism of changing the file extension. The majority of users will use the default setting, whatever it is. I believe the default setting of tramp-imap.el should be to encrypt. If you or anyone else likes, feel free to provide a patch that will support a nil passphrase to mean no encryption should be performed. The usual Emacs-style settings for the passphrase (alist and function in addition to a string) which I plan to add eventually can return nil too for the same effect. I can't accept that as the tramp-imap default, though, and I don't have the time currently to write that support myself. Thanks Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 12:51 ` Ted Zlatanov @ 2009-04-23 13:09 ` Daiki Ueno 2009-04-23 13:44 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Daiki Ueno @ 2009-04-23 13:09 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, tramp-devel >>>>> In <86fxfzbkic.fsf@lifelogs.com> >>>>> Ted Zlatanov <tzz@lifelogs.com> wrote: > On Thu, 23 Apr 2009 11:45:04 +0900 Daiki Ueno <ueno@unixuser.org> wrote: > >> - save /imap:blockstar.com:/INBOX.test/new via tramp-imap-put-file (uses PGG) > >> - read /imap:blockstar.com:/INBOX.test/new via tramp-imap-get-file (uses PGG) DU> I think this (implicit encryption) is a misdesgined feature which should DU> be removed. Why don't you leave the job to auto-encryption-mode so that DU> a user can choose whether to encrypt a file by adding ".gpg"? > Unlike filesystems, IMAP servers are a semi-public space, Really? I use 3 IMAP servers for everyday use and they properly implement access control like file systems. In other words, no one can read my private emails unless s/he shares the passphrase with me. Otherwise, it's a horrible situation... > in the sense that the vast majority of the users can't expect security > or privacy beyond what the server administrators and owners are > willing to grant. Very, very few users actually own and control their > IMAP servers. Thus, encrypting by default is IMHO the best option and > the only one I provided in this initial version. I think you always lack "use-case analysis". Regards, -- Daiki Ueno ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 13:09 ` Daiki Ueno @ 2009-04-23 13:44 ` Ted Zlatanov 2009-04-23 13:48 ` Daiki Ueno 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-04-23 13:44 UTC (permalink / raw) To: ding; +Cc: tramp-devel On Thu, 23 Apr 2009 22:09:52 +0900 Daiki Ueno <ueno@unixuser.org> wrote: >>>>>> In <86fxfzbkic.fsf@lifelogs.com> >>>>>> Ted Zlatanov <tzz@lifelogs.com> wrote: >> On Thu, 23 Apr 2009 11:45:04 +0900 Daiki Ueno <ueno@unixuser.org> wrote: >> >> - save /imap:blockstar.com:/INBOX.test/new via tramp-imap-put-file (uses PGG) >> >> - read /imap:blockstar.com:/INBOX.test/new via tramp-imap-get-file (uses PGG) DU> I think this (implicit encryption) is a misdesgined feature which should DU> be removed. Why don't you leave the job to auto-encryption-mode so that DU> a user can choose whether to encrypt a file by adding ".gpg"? >> Unlike filesystems, IMAP servers are a semi-public space, DU> Really? I use 3 IMAP servers for everyday use and they properly DU> implement access control like file systems. In other words, no one can DU> read my private emails unless s/he shares the passphrase with me. Yes, really. I was not talking about other IMAP users. Whoever owns your IMAP servers can read your unencrypted e-mails with little effort. If you trust your 3 servers, that's great. The vast majority of IMAP users, as I said, have no expectation of privacy on their IMAP server (consider GMail, for instance). This is not a filesystem on a remote server where you can trust user permissions to be managed correctly. >> Very, very few users actually own and control their IMAP servers. >> Thus, encrypting by default is IMHO the best option and the only one >> I provided in this initial version. DU> I think you always lack "use-case analysis". I guess we'll have to disagree. As I said, feel free to provide patches to make turning encryption off easier, and of course if a vote goes against my design I'll reconsider it. Thanks Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 13:44 ` Ted Zlatanov @ 2009-04-23 13:48 ` Daiki Ueno 2009-04-23 14:19 ` Ted Zlatanov 2009-04-23 14:28 ` Daiki Ueno 0 siblings, 2 replies; 58+ messages in thread From: Daiki Ueno @ 2009-04-23 13:48 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, tramp-devel >>>>> In <86zle7a3in.fsf@lifelogs.com> >>>>> Ted Zlatanov <tzz@lifelogs.com> wrote: > If you trust your 3 servers, that's great. The vast majority of IMAP > users, as I said, have no expectation of privacy on their IMAP server > (consider GMail, for instance). This is not a filesystem on a remote > server where you can trust user permissions to be managed correctly. I agree with the facts. But in that case, you cannot even trust your ISP which delivers original e-mails, can you? Do you think that Gnus should encrypt all the outgoing e-mails by default? -- Daiki Ueno ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 13:48 ` Daiki Ueno @ 2009-04-23 14:19 ` Ted Zlatanov 2009-04-23 14:28 ` Daiki Ueno 1 sibling, 0 replies; 58+ messages in thread From: Ted Zlatanov @ 2009-04-23 14:19 UTC (permalink / raw) To: ding; +Cc: tramp-devel On Thu, 23 Apr 2009 22:48:39 +0900 Daiki Ueno <ueno@unixuser.org> wrote: >>>>>> In <86zle7a3in.fsf@lifelogs.com> >>>>>> Ted Zlatanov <tzz@lifelogs.com> wrote: >> If you trust your 3 servers, that's great. The vast majority of IMAP >> users, as I said, have no expectation of privacy on their IMAP server >> (consider GMail, for instance). This is not a filesystem on a remote >> server where you can trust user permissions to be managed correctly. DU> I agree with the facts. But in that case, you cannot even trust your DU> ISP which delivers original e-mails, can you? Users expect a higher level of security with their files than with e-mail. I think tramp-imap.el should provide it by default. Doing less is easier and lets us blame the user later ("Why didn't you read the manual? It clearly says to turn encryption on or specify the .gpg extension!") but I don't like that approach. DU> Do you think that Gnus should encrypt all the outgoing e-mails by DU> default? No, that's completely different from what we're discussing. We're discussing file storage over IMAP, not e-mail security. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 13:48 ` Daiki Ueno 2009-04-23 14:19 ` Ted Zlatanov @ 2009-04-23 14:28 ` Daiki Ueno 2009-04-23 15:16 ` Ted Zlatanov 1 sibling, 1 reply; 58+ messages in thread From: Daiki Ueno @ 2009-04-23 14:28 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, tramp-devel [-- Attachment #1: Type: text/plain, Size: 871 bytes --] >>>>> In <7d371a60-fbac-4b0e-a6b3-2001292f0205@broken.deisui.org> >>>>> Daiki Ueno <ueno@unixuser.org> wrote: > >>>>> In <86zle7a3in.fsf@lifelogs.com> > >>>>> Ted Zlatanov <tzz@lifelogs.com> wrote: > > If you trust your 3 servers, that's great. The vast majority of IMAP > > users, as I said, have no expectation of privacy on their IMAP server > > (consider GMail, for instance). This is not a filesystem on a remote > > server where you can trust user permissions to be managed correctly. > I agree with the facts. But in that case, you cannot even trust your > ISP which delivers original e-mails, can you? Ah, I see my confusion about e-mail systems and IMAP as a file system. That makes sense. I apologize to you for that. But still, I'm against the use of PGG (which is on the way to obsolescence) in the new package. Here is a patch to use EPG instead. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: tramp-imap.el.diff --] [-- Type: text/x-diff, Size: 2840 bytes --] --- tramp-imap.el~ 2009-04-23 23:13:51.000000000 +0900 +++ tramp-imap.el 2009-04-23 23:25:46.000000000 +0900 @@ -247,7 +247,7 @@ (require 'tramp-cache) (require 'tramp-compat) (require 'imap) -(require 'pgg) +(require 'epa) (autoload 'auth-source-user-or-password "auth-source") ;; Define Tramp IMAP method ... @@ -627,8 +627,7 @@ (goto-char (point-min)) (while (re-search-forward " " nil t) (replace-match "" nil nil)) - ;; TODO: catch errors from PGG here - (tramp-imap-pgg-decode-buffer)))) + (tramp-imap-decode-buffer)))) ;;; (tramp-imap-collapse-name "a b c / where ; strange ! characters $ abound") ;;; => "abcwherestrangecharactersabound" @@ -655,29 +654,40 @@ (with-current-buffer filename-or-buffer (buffer-string))) (insert-file-contents filename-or-buffer)) - (tramp-imap-pgg-encode-buffer)))) + (tramp-imap-encode-buffer)))) (current-buffer))) -(defun tramp-imap-pgg-encode-buffer () - (tramp-imap-pgg-process-buffer t)) - -(defun tramp-imap-pgg-decode-buffer () - (tramp-imap-pgg-process-buffer)) - -(defun tramp-imap-pgg-process-buffer (&optional encode) - "Use PGG to encode or decode the current buffer." - (let ((pfft (if encode 'pgg-encrypt-symmetric 'pgg-decrypt)) - (default-enable-multibyte-characters nil) - (input (buffer-substring-no-properties (point-min) (point-max))) - exit-data) - (with-temp-buffer - (insert input) - ;; note that we call pfft before pgg-display-output-buffer - (pgg-display-output-buffer (point-min) (point-max) (funcall pfft (point-min) (point-max) tramp-imap-passphrase)) - (setq exit-data - (buffer-substring-no-properties (point-min) (point-max)))) -; (debug exit-data) - exit-data)) +(defun tramp-imap-encode-buffer () + (let ((context (epg-make-context 'OpenPGP)) + cipher) + (epg-context-set-armor context t) + (epg-context-set-passphrase-callback context + #'epa-passphrase-callback-function) + (epg-context-set-progress-callback context + (cons #'epa-progress-callback-function + "Encrypting...")) + (message "Encrypting...") + (setq cipher (epg-encrypt-string + context + (encode-coding-string (buffer-string) 'utf-8) + nil)) + (message "Encrypting...done") + cipher)) + +(defun tramp-imap-decode-buffer () + (let ((context (epg-make-context 'OpenPGP)) + plain) + (epg-context-set-passphrase-callback context + #'epa-passphrase-callback-function) + (epg-context-set-progress-callback context + (cons #'epa-progress-callback-function + "Decrypting...")) + (message "Decrypting...") + (setq plain (decode-coding-string + (epg-decrypt-string context (buffer-string)) + 'utf-8)) + (message "Decrypting...done") + plain)) (provide 'tramp-imap) ;;; tramp-imap.el ends here [-- Attachment #3: Type: text/plain, Size: 25 bytes --] Regards, -- Daiki Ueno ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 14:28 ` Daiki Ueno @ 2009-04-23 15:16 ` Ted Zlatanov 2009-04-24 0:37 ` Daiki Ueno 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-04-23 15:16 UTC (permalink / raw) To: ding; +Cc: tramp-devel On Thu, 23 Apr 2009 23:28:08 +0900 Daiki Ueno <ueno@unixuser.org> wrote: DU> But still, I'm against the use of PGG (which is on the way to DU> obsolescence) in the new package. Here is a patch to use EPG instead. Wonderful. Two questions: - EPG asks for the passphrase. It should try tramp-imap-passphrase as a string first (I'll add the alist and function call support later). As an extra courtesy to the user, if the passphrase is nil, we should ask interactively if encryption is wanted, and if yes, read the passphrase. Would you be willing to make that addition? - I had to clean ^M characters from the buffer before; EPG seems to handle them automatically so I removed this code ;; TODO: do this better, removes all ^M in the buffer (goto-char (point-min)) (while (re-search-forward "^M" nil t) ;; this was the literal ^M (replace-match "" nil nil)) from tramp-imap-get-file. Is that OK? Thank you Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 15:16 ` Ted Zlatanov @ 2009-04-24 0:37 ` Daiki Ueno 0 siblings, 0 replies; 58+ messages in thread From: Daiki Ueno @ 2009-04-24 0:37 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, tramp-devel [-- Attachment #1: Type: text/plain, Size: 470 bytes --] >>>>> In <86r5zj9z96.fsf@lifelogs.com> >>>>> Ted Zlatanov <tzz@lifelogs.com> wrote: > - EPG asks for the passphrase. It should try tramp-imap-passphrase as a > string first (I'll add the alist and function call support later). As > an extra courtesy to the user, if the passphrase is nil, we should ask > interactively if encryption is wanted, and if yes, read the > passphrase. Would you be willing to make that addition? Yes, here is a patch to do that. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: tramp-imap.el.diff --] [-- Type: text/x-diff, Size: 1300 bytes --] --- tramp-imap.el~ 2009-04-24 08:59:11.000000000 +0900 +++ tramp-imap.el 2009-04-24 09:01:16.000000000 +0900 @@ -738,12 +738,17 @@ (error "Could not find the body separator in the encoded message!"))) (current-buffer))) +(defun tramp-imap-passphrase-callback-function (context key-id handback) + (or tramp-imap-passphrase + (epa-passphrase-callback-function context key-id handback))) + (defun tramp-imap-encode-buffer () (let ((context (epg-make-context 'OpenPGP)) cipher) (epg-context-set-armor context t) - (epg-context-set-passphrase-callback context - #'epa-passphrase-callback-function) + (epg-context-set-passphrase-callback + context + #'tramp-imap-passphrase-callback-function) (epg-context-set-progress-callback context (cons #'epa-progress-callback-function "Encrypting...")) @@ -758,8 +763,9 @@ (defun tramp-imap-decode-buffer () (let ((context (epg-make-context 'OpenPGP)) plain) - (epg-context-set-passphrase-callback context - #'epa-passphrase-callback-function) + (epg-context-set-passphrase-callback + context + #'tramp-imap-passphrase-callback-function) (epg-context-set-progress-callback context (cons #'epa-progress-callback-function "Decrypting...")) [-- Attachment #3: Type: text/plain, Size: 476 bytes --] > - I had to clean ^M characters from the buffer before; EPG seems to > handle them automatically so I removed this code > ;; TODO: do this better, removes all ^M in the buffer > (goto-char (point-min)) > (while (re-search-forward "^M" nil t) ;; this was the literal ^M > (replace-match "" nil nil)) > from tramp-imap-get-file. Is that OK? I think so - if data do not roundtrip before/after encryption, it seems a bug... Regards, -- Daiki Ueno ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 0:15 ` Ted Zlatanov 2009-04-23 2:45 ` Daiki Ueno @ 2009-04-23 21:30 ` Ted Zlatanov 2009-04-25 17:25 ` Michael Albinus 1 sibling, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-04-23 21:30 UTC (permalink / raw) To: ding; +Cc: tramp-devel [-- Attachment #1: Type: text/plain, Size: 1020 bytes --] On Wed, 22 Apr 2009 19:15:36 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: TZ> So basically the mailbox name is everything up to the last / character, TZ> like a `dirname' call. IMAP mailboxes can contain many characters, TZ> including /, so we need to forbid those characters in the file name. TZ> The base file name is then just `basename' on the full name. I tried to TZ> implement this by hand in tramp-imap-file-name-mailbox-or-name but there TZ> must be a better way. I fixed this in the code, and now it seems like filename completion works too. I always convert "/INBOX.test/a/b/filename" to mailbox "INBOX.test/a/b" and message name "filename". In addition, I collapse the file name to make searching and usage easier by subject, while keeping the real file name in the X-Tramp-IMAP header. I made many other fixes so please use the attached file as a baseline if you want to submit patches for any of the issues I listed before, or any new issues you notice. It's getting close to usable. Thanks Ted [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 29868 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-23 21:30 ` Ted Zlatanov @ 2009-04-25 17:25 ` Michael Albinus 2009-07-10 21:34 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Michael Albinus @ 2009-04-25 17:25 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 10404 bytes --] Ted Zlatanov <tzz@lifelogs.com> writes: > I made many other fixes so please use the attached file as a baseline if > you want to submit patches for any of the issues I listed before, or any > new issues you notice. It's getting close to usable. I've started to play with it. First error: Debugger entered--Lisp error: (wrong-number-of-arguments #[nil "dbÀÁÂÃ# \x13 " nil t delete-char 1] 4 ("/usr/local/share/emacs/23.0.92/lisp/gnus/nnheader.elc" . 25599)] 1) nnheader-ms-strip-cr("X-Tramp-Imap: *scratch* \nFrom: Tramp-Imap <tramp-imap@tramp-imap.tramp.emacs.org> \nTo: Tramp-Imap <tramp-imap@tramp-imap.tramp.emacs.org> \nSubject: tramp-imap-subject-marker fstab \nDate: Sat, 25 Apr 2009 19:08:13 +0200 \nMessage-ID: <87prf04q6a.fsf@gmx.de> \n \n") tramp-imap-get-message-headers(1) (nnheader-ms-strip-cr (tramp-imap-get-message-headers msgs)) (let ((headers ...)) (when (string-match "^X-Tramp-IMAP: \\(.+\\)" headers) (match-string 1 headers))) tramp-imap-get-message-x-tramp-imap(1) (setq sname (tramp-imap-get-message-x-tramp-imap msg)) (while --cl-dolist-temp-- (setq msg (car --cl-dolist-temp--)) (setq sname (tramp-imap-get-message-x-tramp-imap msg)) (when (and sname ...) (push ... res) (tramp-debug-message vec "found message %s: %s" msg ...)) (setq --cl-dolist-temp-- (cdr --cl-dolist-temp--))) (let ((--cl-dolist-temp-- ...) msg) (while --cl-dolist-temp-- (setq msg ...) (setq sname ...) (when ... ... ...) (setq --cl-dolist-temp-- ...)) nil) (catch (quote --cl-block-nil--) (let (... msg) (while --cl-dolist-temp-- ... ... ... ...) nil)) (cl-block-wrapper (catch (quote --cl-block-nil--) (let ... ... nil))) (block nil (let (... msg) (while --cl-dolist-temp-- ... ... ... ...) nil)) (dolist (msg (imap-search ...)) (setq sname (tramp-imap-get-message-x-tramp-imap msg)) (when (and sname ...) (push ... res) (tramp-debug-message vec "found message %s: %s" msg ...))) (let (sname) (tramp-debug-message vec "looking for '%s'" search-name) (dolist (msg ...) (setq sname ...) (when ... ... ...))) (if (and name (imap-mailbox-select mbox)) (let (sname) (tramp-debug-message vec "looking for '%s'" search-name) (dolist ... ... ...)) (tramp-error vec (quote none) "bad name %s or mailbox %s" name mbox)) (let* ((mbox ...) (name ...) (truename ...) (search-name ...) res) (tramp-debug-message vec "selecting mbox %s" mbox) (if (and name ...) (let ... ... ...) (tramp-error vec ... "bad name %s or mailbox %s" name mbox)) res) (save-current-buffer (set-buffer (tramp-imap-buffer vec)) (let* (... ... ... ... res) (tramp-debug-message vec "selecting mbox %s" mbox) (if ... ... ...) res)) (with-current-buffer (tramp-imap-buffer vec) (let* (... ... ... ... res) (tramp-debug-message vec "selecting mbox %s" mbox) (if ... ... ...) res)) tramp-imap-get-file-entries(["imap" "albinus" "localhost" "/INBOX.test"]) (car (tramp-imap-get-file-entries v)) (cdr (car (tramp-imap-get-file-entries v))) (progn (cdr (car ...))) (setq value (progn (cdr ...))) (progn (setq value (progn ...)) (tramp-set-file-property v localname (format "file-attributes-%s" id-format) value)) (if (eq value (quote undef)) (progn (setq value ...) (tramp-set-file-property v localname ... value))) (when (eq value (quote undef)) (setq value (progn ...)) (tramp-set-file-property v localname (format "file-attributes-%s" id-format) value)) (let ((value ...)) (when (eq value ...) (setq value ...) (tramp-set-file-property v localname ... value)) value) (if (file-name-absolute-p localname) (let (...) (when ... ... ...) value) (cdr (car ...))) (with-file-property v localname (format "file-attributes-%s" id-format) (cdr (car ...))) (let* ((v ...) (method ...) (user ...) (host ...) (localname ...)) (with-file-property v localname (format "file-attributes-%s" id-format) (cdr ...))) (with-parsed-tramp-file-name (expand-file-name filename) nil (with-file-property v localname (format "file-attributes-%s" id-format) (cdr ...))) tramp-imap-handle-file-attributes("/imap:albinus@localhost:/INBOX.test") apply(tramp-imap-handle-file-attributes "/imap:albinus@localhost:/INBOX.test") (progn (apply (cdr fn) args)) (unwind-protect (progn (apply ... args)) (set-match-data save-match-data-internal (quote evaporate))) (let ((save-match-data-internal ...)) (unwind-protect (progn ...) (set-match-data save-match-data-internal ...))) (save-match-data (apply (cdr fn) args)) (if fn (save-match-data (apply ... args)) (tramp-run-real-handler operation args)) (let ((fn ...)) (if fn (save-match-data ...) (tramp-run-real-handler operation args))) tramp-imap-file-name-handler(file-attributes "/imap:albinus@localhost:/INBOX.test") apply(tramp-imap-file-name-handler file-attributes "/imap:albinus@localhost:/INBOX.test") (cond ((and completion ... ...) t) ((and completion ... ...) filename) (foreign (apply foreign operation args)) (t (tramp-run-real-handler operation args))) (let* ((v ...) (method ...) (user ...) (host ...) (localname ...)) (cond (... t) (... filename) (foreign ...) (t ...))) (with-parsed-tramp-file-name filename nil (cond (... t) (... filename) (foreign ...) (t ...))) (let* ((filename ...) (completion ...) (foreign ...)) (with-parsed-tramp-file-name filename nil (cond ... ... ... ...))) (progn (let* (... ... ...) (with-parsed-tramp-file-name filename nil ...))) (unwind-protect (progn (let* ... ...)) (set-match-data save-match-data-internal (quote evaporate))) (let ((save-match-data-internal ...)) (unwind-protect (progn ...) (set-match-data save-match-data-internal ...))) (save-match-data (let* (... ... ...) (with-parsed-tramp-file-name filename nil ...))) (if tramp-mode (save-match-data (let* ... ...)) (tramp-run-real-handler operation args)) tramp-file-name-handler(file-attributes "/imap:albinus@localhost:/INBOX.test") file-attributes("/imap:albinus@localhost:/INBOX.test") (car (file-attributes filename)) (let ((x ...)) (when (stringp x) (if ... ... x))) (let* ((v ...) (method ...) (user ...) (host ...) (localname ...)) (let (...) (when ... ...))) (with-parsed-tramp-file-name filename nil (let (...) (when ... ...))) tramp-handle-file-symlink-p("/imap:albinus@localhost:/INBOX.test") apply(tramp-handle-file-symlink-p "/imap:albinus@localhost:/INBOX.test") (progn (apply (cdr fn) args)) (unwind-protect (progn (apply ... args)) (set-match-data save-match-data-internal (quote evaporate))) (let ((save-match-data-internal ...)) (unwind-protect (progn ...) (set-match-data save-match-data-internal ...))) (save-match-data (apply (cdr fn) args)) (if fn (save-match-data (apply ... args)) (tramp-run-real-handler operation args)) (let ((fn ...)) (if fn (save-match-data ...) (tramp-run-real-handler operation args))) tramp-imap-file-name-handler(file-symlink-p "/imap:albinus@localhost:/INBOX.test") apply(tramp-imap-file-name-handler file-symlink-p "/imap:albinus@localhost:/INBOX.test") (cond ((and completion ... ...) t) ((and completion ... ...) filename) (foreign (apply foreign operation args)) (t (tramp-run-real-handler operation args))) (let* ((v ...) (method ...) (user ...) (host ...) (localname ...)) (cond (... t) (... filename) (foreign ...) (t ...))) (with-parsed-tramp-file-name filename nil (cond (... t) (... filename) (foreign ...) (t ...))) (let* ((filename ...) (completion ...) (foreign ...)) (with-parsed-tramp-file-name filename nil (cond ... ... ... ...))) (progn (let* (... ... ...) (with-parsed-tramp-file-name filename nil ...))) (unwind-protect (progn (let* ... ...)) (set-match-data save-match-data-internal (quote evaporate))) (let ((save-match-data-internal ...)) (unwind-protect (progn ...) (set-match-data save-match-data-internal ...))) (save-match-data (let* (... ... ...) (with-parsed-tramp-file-name filename nil ...))) (if tramp-mode (save-match-data (let* ... ...)) (tramp-run-real-handler operation args)) tramp-file-name-handler(file-symlink-p "/imap:albinus@localhost:/INBOX.test") file-symlink-p("/imap:albinus@localhost:/INBOX.test") file-truename("/imap:albinus@localhost:/INBOX.test" (98) (nil)) file-truename("/imap:albinus@localhost:/INBOX.test/bbb") apply(file-truename "/imap:albinus@localhost:/INBOX.test/bbb") (let* ((inhibit-file-name-handlers ...) (inhibit-file-name-operation operation)) (apply operation args)) tramp-run-real-handler(file-truename ("/imap:albinus@localhost:/INBOX.test/bbb")) (if fn (save-match-data (apply ... args)) (tramp-run-real-handler operation args)) (let ((fn ...)) (if fn (save-match-data ...) (tramp-run-real-handler operation args))) tramp-imap-file-name-handler(file-truename "/imap:albinus@localhost:/INBOX.test/bbb") apply(tramp-imap-file-name-handler file-truename "/imap:albinus@localhost:/INBOX.test/bbb") (cond ((and completion ... ...) t) ((and completion ... ...) filename) (foreign (apply foreign operation args)) (t (tramp-run-real-handler operation args))) (let* ((v ...) (method ...) (user ...) (host ...) (localname ...)) (cond (... t) (... filename) (foreign ...) (t ...))) (with-parsed-tramp-file-name filename nil (cond (... t) (... filename) (foreign ...) (t ...))) (let* ((filename ...) (completion ...) (foreign ...)) (with-parsed-tramp-file-name filename nil (cond ... ... ... ...))) (progn (let* (... ... ...) (with-parsed-tramp-file-name filename nil ...))) (unwind-protect (progn (let* ... ...)) (set-match-data save-match-data-internal (quote evaporate))) (let ((save-match-data-internal ...)) (unwind-protect (progn ...) (set-match-data save-match-data-internal ...))) (save-match-data (let* (... ... ...) (with-parsed-tramp-file-name filename nil ...))) (if tramp-mode (save-match-data (let* ... ...)) (tramp-run-real-handler operation args)) tramp-file-name-handler(file-truename "/imap:albinus@localhost:/INBOX.test/bbb") file-truename("/imap:albinus@localhost:/INBOX.test/bbb") set-visited-file-name("/imap:localhost:INBOX.test/bbb" nil) write-file("/imap:localhost:INBOX.test/bbb" t) call-interactively(write-file nil nil) I'm using gnus as built in GNU Emacs 23.0.92. `nnheader-ms-strip-cr' has no argument there. You use by default 'ssl for connection. Can we expect this to be the standard? Or shall we let the user decide, for example by two methods "imap" and "imaps"? I've also applied some few patches, you might check. See appended. > Thanks > Ted Best regards, Michael. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Type: text/x-patch, Size: 5345 bytes --] *** /home/albinus/lisp/tramp-imap.el.~1~ 2009-04-25 18:27:52.000000000 +0200 --- /home/albinus/lisp/tramp-imap.el 2009-04-25 19:03:27.000000000 +0200 *************** *** 261,269 **** ;; ... and add it to the method list. (add-to-list 'tramp-methods (cons tramp-imap-method nil)) - ;; ... and add it to the method list. - (add-to-list 'tramp-methods (cons tramp-imap-method nil)) - ;; Add a default for `tramp-default-user-alist'. Default is the local user. (add-to-list 'tramp-default-user-alist `(,tramp-imap-method nil ,(user-login-name))) --- 261,266 ---- *************** *** 336,354 **** "Alist of handler functions for Tramp IMAP method. Operations not mentioned here will be handled by the default Emacs primitives.") - (defgroup tramp-imap nil - "Tramp over IMAP configuration." - :group 'applications) - (defcustom tramp-imap-subject-marker "tramp-imap-subject-marker" "The subject marker that Tramp-Imap will use." :type 'string ! :group 'tramp-imap) (defcustom tramp-imap-passphrase "my passphrase" "The passphrase that Tramp-Imap will use. TODO: allow function calls, etc. here" :type 'string ! :group 'tramp-imap) (defun tramp-imap-file-name-p (filename) "Check if it's a filename for IMAP protocol." --- 333,347 ---- "Alist of handler functions for Tramp IMAP method. Operations not mentioned here will be handled by the default Emacs primitives.") (defcustom tramp-imap-subject-marker "tramp-imap-subject-marker" "The subject marker that Tramp-Imap will use." :type 'string ! :group 'tramp) (defcustom tramp-imap-passphrase "my passphrase" "The passphrase that Tramp-Imap will use. TODO: allow function calls, etc. here" :type 'string ! :group 'tramp) (defun tramp-imap-file-name-p (filename) "Check if it's a filename for IMAP protocol." *************** *** 410,416 **** (save-match-data (let ((entries (with-file-property v localname "file-entries" ! (tramp-imap-get-file-entries v localname)))) (mapcar (lambda (x) (list --- 403,409 ---- (save-match-data (let ((entries (with-file-property v localname "file-entries" ! (tramp-imap-get-file-entries v)))) (mapcar (lambda (x) (list *************** *** 421,431 **** ;;; (tramp-imap-get-file-entries (tramp-dissect-file-name "/imap:blockstar.com:/INBOX.test/fstab") t) ! (defun tramp-imap-get-file-entries (vec localname &optional exact) "Read entries returned by IMAP server. EXACT limits to exact matches. Result is a list of (LOCALNAME LINK COUNT UID GID ATIME MTIME CTIME SIZE MODE WEIRD INODE DEVICE)." ! (tramp-debug-message vec "working on %s" localname) (with-current-buffer (tramp-imap-buffer vec) (let* ((mbox (tramp-imap-file-name-mailbox vec)) (name (tramp-imap-file-name-name vec)) --- 414,424 ---- ;;; (tramp-imap-get-file-entries (tramp-dissect-file-name "/imap:blockstar.com:/INBOX.test/fstab") t) ! (defun tramp-imap-get-file-entries (vec &optional exact) "Read entries returned by IMAP server. EXACT limits to exact matches. Result is a list of (LOCALNAME LINK COUNT UID GID ATIME MTIME CTIME SIZE MODE WEIRD INODE DEVICE)." ! (tramp-debug-message vec "working on %s" (tramp-file-name-localname vec)) (with-current-buffer (tramp-imap-buffer vec) (let* ((mbox (tramp-imap-file-name-mailbox vec)) (name (tramp-imap-file-name-name vec)) *************** *** 516,522 **** (let ((point (point)) size data) (tramp-message v 4 "Fetching file %s..." filename) ! (let* ((found (tramp-imap-get-file-entries v filename)) ;; inode is the message UID; we use the first message found (uid (nth 11 (nth 0 found)))) (when found --- 509,515 ---- (let ((point (point)) size data) (tramp-message v 4 "Fetching file %s..." filename) ! (let* ((found (tramp-imap-get-file-entries v)) ;; inode is the message UID; we use the first message found (uid (nth 11 (nth 0 found)))) (when found *************** *** 547,553 **** "Like `file-attributes' for Tramp files." (with-parsed-tramp-file-name (expand-file-name filename) nil (with-file-property v localname (format "file-attributes-%s" id-format) ! (cdr (car (tramp-imap-get-file-entries v localname)))))) (defun tramp-imap-handle-file-executable-p (filename) "Like `file-executable-p' for Tramp files. False for IMAP." --- 540,546 ---- "Like `file-attributes' for Tramp files." (with-parsed-tramp-file-name (expand-file-name filename) nil (with-file-property v localname (format "file-attributes-%s" id-format) ! (cdr (car (tramp-imap-get-file-entries v)))))) (defun tramp-imap-handle-file-executable-p (filename) "Like `file-executable-p' for Tramp files. False for IMAP." *************** *** 651,664 **** ;;; (with-current-buffer (tramp-imap-buffer (tramp-dissect-file-name "/imap:blockstar.com:/INBOX.test/new")) (tramp-imap-get-message-subject 24)) (defun tramp-imap-get-message-subject (msgs) "Get message subject over IMAP." ! (let ((allsubject ! (nnheader-ms-strip-cr ! (nth 2 (nth 0 (imap-fetch msgs ! "BODY[HEADER.FIELDS (SUBJECT)]" 'BODYDETAIL)))))) ! (when (string-match [-- Attachment #3: Type: text/plain, Size: 143 bytes --] _______________________________________________ Tramp-devel mailing list Tramp-devel@gnu.org http://lists.gnu.org/mailman/listinfo/tramp-devel ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: file storage in IMAP (eventually for Tramp) working and needs testing 2009-04-25 17:25 ` Michael Albinus @ 2009-07-10 21:34 ` Ted Zlatanov 2009-07-15 21:25 ` Tramp-IMAP works " Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-07-10 21:34 UTC (permalink / raw) To: ding; +Cc: tramp-devel [-- Attachment #1: Type: text/plain, Size: 1780 bytes --] On Sat, 25 Apr 2009 19:25:58 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Ted Zlatanov <tzz@lifelogs.com> writes: >> I made many other fixes so please use the attached file as a baseline if >> you want to submit patches for any of the issues I listed before, or any >> new issues you notice. It's getting close to usable. MA> I've started to play with it. First error: MA> Debugger entered--Lisp error: (wrong-number-of-arguments #[nil "ŠdbˆÀÁÂÃ#… MA> " nil t delete-char 1] 4 ("/usr/local/share/emacs/23.0.92/lisp/gnus/nnheader.elc" . 25599)] 1) MA> nnheader-ms-strip-cr("X-Tramp-Imap: *scratch* This is fixed, thanks. It was simplest to just copy those functions to tramp-imap.el, considering it shouldn't depend on Gnus anyhow. MA> You use by default 'ssl for connection. Can we expect this to be the MA> standard? Or shall we let the user decide, for example by two methods MA> "imap" and "imaps"? That makes sense, yes. Do you think you could add it? For now I left it nil, so imap.el will pick the best available stream. MA> I've also applied some few patches, you might check. See appended. I applied them all, thank you. Current status: passphrase management works, but I should hook auth-source into it. Saving a file works, both to a new name and to an existing name. Opening a file works. Deleting a file doesn't work yet. I hope users can start testing this so I'm not just testing it against my Courier IMAP server in GNU Emacs (XEmacs should work too, for example). I think it's OK to throw this into Tramp when file deletion is added and all the extraneous comments are removed. All the TODO items should also be done. By then we should have some test cases too. Thanks for your help and sorry for the long delay... Ted [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 32232 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Tramp-IMAP works and needs testing 2009-07-10 21:34 ` Ted Zlatanov @ 2009-07-15 21:25 ` Ted Zlatanov 2009-07-16 6:42 ` Michael Albinus 2009-07-18 11:11 ` Michael Albinus 0 siblings, 2 replies; 58+ messages in thread From: Ted Zlatanov @ 2009-07-15 21:25 UTC (permalink / raw) To: ding; +Cc: tramp-devel [-- Attachment #1: Type: text/plain, Size: 914 bytes --] On Fri, 10 Jul 2009 16:34:49 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: TZ> I think it's OK to throw this into Tramp when file deletion is added and TZ> all the extraneous comments are removed. All the TODO items should also TZ> be done. By then we should have some test cases too. All right, attached is one more release before the ready-for-Tramp one: - allow file deletion (all matching files are deleted) - allow imap and imaps method, connecting over SSL when appropriate - use auth-source for the symmetric passphrase - much more complex logic for passphrases, allowing caching them - give a small setup example - remove my mail server and username from the test cases littered throughout I need to do more cleanup, but at least this version is usable. The next release will be for wide release and will go into Tramp itself, so PLEASE test this version and let me know if you find problems. Ted [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 27423 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: Tramp-IMAP works and needs testing 2009-07-15 21:25 ` Tramp-IMAP works " Ted Zlatanov @ 2009-07-16 6:42 ` Michael Albinus 2009-07-18 11:11 ` Michael Albinus 1 sibling, 0 replies; 58+ messages in thread From: Michael Albinus @ 2009-07-16 6:42 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding Ted Zlatanov <tzz@lifelogs.com> writes: > I need to do more cleanup, but at least this version is usable. The > next release will be for wide release and will go into Tramp itself, so > PLEASE test this version and let me know if you find problems. Don't panic, it is on my todo list. As usual, this list is much too long. I hope to find time for test over the weekend. > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: Tramp-IMAP works and needs testing 2009-07-15 21:25 ` Tramp-IMAP works " Ted Zlatanov 2009-07-16 6:42 ` Michael Albinus @ 2009-07-18 11:11 ` Michael Albinus 2009-09-07 4:13 ` Ted Zlatanov 1 sibling, 1 reply; 58+ messages in thread From: Michael Albinus @ 2009-07-18 11:11 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding [-- Attachment #1: Type: text/plain, Size: 2802 bytes --] Ted Zlatanov <tzz@lifelogs.com> writes: > I need to do more cleanup, but at least this version is usable. The > next release will be for wide release and will go into Tramp itself, so > PLEASE test this version and let me know if you find problems. I've played a little bit around: - Several functions in `tramp-imap-file-name-handler-alist' are not implemented yet. - I doubt there will be implementation for `shell-command', `executable-find', and `process-file'. I've disabled them in `tramp-imap-file-name-handler-alist'. - You have used `tramp-debug-message'. I recommend to use `tramp-message' instead, the verbose levels allow a better analysis of the debug buffer. - `tramp-imap-handle-file-directory-p' must not return nil in all cases. When I apply (save-buffer), the imap folder is checked for being a directory. As workaround, I've changed `tramp-imap-handle-file-directory-p' to return `t', but it must be more precise. - When I have opened "/imap:albinus@localhost:/INBOX.test/1", and try to save it under the name "/imap:albinus@localhost:/INBOX.test/2", I get Debugger entered--Lisp error: (none "bad name 1 or mailbox INBOX.test/2") signal(none ("bad name 1 or mailbox INBOX.test/2")) tramp-error(["imap" "albinus" "localhost" "/INBOX.test/2/1"] none "bad name %s or mailbox %s" "1" "INBOX.test/2") (if (and name (imap-mailbox-select mbox)) (let (sname) (tramp-debug-message vec "looking for '%s'" search-name) (dolist ... ... ...)) (tramp-error vec (quote none) "bad name %s or mailbox %s" name mbox)) (let* ((mbox ...) (name ...) (truename ...) (search-name ...) res) (tramp-debug-message vec "selecting mbox %s" mbox) (if (and name ...) (let ... ... ...) (tramp-error vec ... "bad name %s or mailbox %s" name mbox)) res) (save-current-buffer (set-buffer (tramp-imap-buffer vec)) (let* (... ... ... ... res) (tramp-debug-message vec "selecting mbox %s" mbox) (if ... ... ...) res)) (with-current-buffer (tramp-imap-buffer vec) (let* (... ... ... ... res) (tramp-debug-message vec "selecting mbox %s" mbox) (if ... ... ...) res)) tramp-imap-get-file-entries(["imap" "albinus" "localhost" "/INBOX.test/2/1"] "/INBOX.test/2/1") [...] - I've fixed a small error in `tramp-imap-handle-delete-file' and `tramp-imap-delete-files'. - `tramp-retrieve-data', used in `tramp-imap-handle-file-local-copy', does not exist. - In `tramp-imap-passphrase-callback-function', you use the non-existing variable `v'. - The email address of stored messages looks like "a real one", one could try to send messages there. Better might be to use an invalid address as defined in RFC 2606, like "Tramp-IMAP <tramp-imap@tramp-imap.invalid>". My current patch is appended. Now I must go; my kids want to see HP6. > Ted Best regards, Michael. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Type: text/x-patch, Size: 16444 bytes --] *** /home/albinus/lisp/tramp-imap.el.~1~ 2009-07-18 10:45:43.000000000 +0200 --- /home/albinus/lisp/tramp-imap.el 2009-07-18 12:58:21.000000000 +0200 *************** *** 60,65 **** --- 60,66 ---- (defcustom tramp-imap-method "imap" "*Method to connect via IMAP protocol." :group 'tramp + :version "23.2" :type 'string) (add-to-list 'tramp-methods (cons tramp-imap-method nil)) *************** *** 72,77 **** --- 73,79 ---- (defcustom tramp-imaps-method "imaps" "*Method to connect via secure IMAP protocol." :group 'tramp + :version "23.2" :type 'string) ;; ... and add it to the method list. *************** *** 84,89 **** --- 86,92 ---- ;; Add completion function for IMAP method. ;; (tramp-set-completion-function ;; tramp-imap-method tramp-completion-function-alist-ssh) ; TODO: test this + ;; tramp-imaps-method tramp-completion-function-alist-ssh) ; TODO: test this ;; New handlers should be added here. (defconst tramp-imap-file-name-handler-alist *************** *** 136,161 **** (set-file-modes . tramp-imap-handle-set-file-modes) (set-file-times . tramp-imap-handle-set-file-times) (set-visited-file-modtime . ignore) ! (shell-command . tramp-handle-shell-command) (substitute-in-file-name . tramp-handle-substitute-in-file-name) (unhandled-file-name-directory . tramp-handle-unhandled-file-name-directory) (vc-registered . ignore) (verify-visited-file-modtime . ignore) (write-region . tramp-imap-handle-write-region) ! (executable-find . tramp-imap-handle-executable-find) (start-file-process . ignore) ! (process-file . tramp-imap-handle-process-file) ) "Alist of handler functions for Tramp IMAP method. Operations not mentioned here will be handled by the default Emacs primitives.") (defgroup tramp-imap nil "Tramp over IMAP configuration." :group 'applications) (defcustom tramp-imap-subject-marker "tramp-imap-subject-marker" "The subject marker that Tramp-IMAP will use." :type 'string :group 'tramp-imap) ;; TODO: these will be defcustoms later --- 139,166 ---- (set-file-modes . tramp-imap-handle-set-file-modes) (set-file-times . tramp-imap-handle-set-file-times) (set-visited-file-modtime . ignore) ! (shell-command . ignore) (substitute-in-file-name . tramp-handle-substitute-in-file-name) (unhandled-file-name-directory . tramp-handle-unhandled-file-name-directory) (vc-registered . ignore) (verify-visited-file-modtime . ignore) (write-region . tramp-imap-handle-write-region) ! (executable-find . ignore) (start-file-process . ignore) ! (process-file . ignore) ) "Alist of handler functions for Tramp IMAP method. Operations not mentioned here will be handled by the default Emacs primitives.") (defgroup tramp-imap nil "Tramp over IMAP configuration." + :version "23.2" :group 'applications) (defcustom tramp-imap-subject-marker "tramp-imap-subject-marker" "The subject marker that Tramp-IMAP will use." :type 'string + :version "23.2" :group 'tramp-imap) ;; TODO: these will be defcustoms later *************** *** 253,265 **** (tramp-debug-message vec "looking for '%s'" search-name) (dolist (msg (imap-search (format "SUBJECT \"%s %s\"" ! tramp-imap-subject-marker search-name))) ;; this is the name of the file (setq sname (tramp-imap-get-message-x-tramp-imap msg)) ! ! (when (and sname ; we need a valid X-Tramp-IMAP header (or (not exact) (equal sname truename))) ;; Return entry in file-attributes format --- 258,270 ---- (tramp-debug-message vec "looking for '%s'" search-name) (dolist (msg (imap-search (format "SUBJECT \"%s %s\"" ! tramp-imap-subject-marker search-name))) ;; this is the name of the file (setq sname (tramp-imap-get-message-x-tramp-imap msg)) ! ! (when (and sname ; we need a valid X-Tramp-IMAP header (or (not exact) (equal sname truename))) ;; Return entry in file-attributes format *************** *** 356,362 **** (defun tramp-imap-handle-file-directory-p (filename) "Like `file-directory-p' for Tramp files. False for IMAP." ! nil) (defun tramp-imap-handle-file-attributes (filename &optional id-format) "Like `file-attributes' for Tramp files." --- 361,367 ---- (defun tramp-imap-handle-file-directory-p (filename) "Like `file-directory-p' for Tramp files. False for IMAP." ! t);nil) (defun tramp-imap-handle-file-attributes (filename &optional id-format) "Like `file-attributes' for Tramp files." *************** *** 380,388 **** "Like `delete-file' for Tramp files." (cond ((not (file-exists-p file)) nil) ! (t (tramp-imap-delete-files ! ;; inode is the message UID; we use any messages found ! (tramp-imap-get-file-entries v filename))))) ;; TODO: fix this in tramp-imap-get-file-entries (defun tramp-imap-handle-file-newer-than-file-p (file1 file2) --- 385,395 ---- "Like `delete-file' for Tramp files." (cond ((not (file-exists-p file)) nil) ! (t (with-parsed-tramp-file-name (expand-file-name file) nil ! (with-current-buffer (tramp-imap-buffer v) ! (tramp-imap-delete-files ! ;; inode is the message UID; we use any messages found ! (tramp-imap-get-file-entries v file))))))) ;; TODO: fix this in tramp-imap-get-file-entries (defun tramp-imap-handle-file-newer-than-file-p (file1 file2) *************** *** 402,407 **** --- 409,415 ---- "Cannot make local copy of non-existing file `%s'" filename)) (let ((tmpfile (tramp-compat-make-temp-file filename))) (tramp-message v 4 "Fetching %s to tmp file %s..." filename tmpfile) + ;; TODO: tramp-retrieve-data does not exist!!! (when (tramp-imap-retrieve-data v) ;; Save file (with-current-buffer (tramp-get-buffer v) *************** *** 437,460 **** (defun tramp-imap-file-name-truename (vec) (tramp-imap-file-name-mailbox-or-name vec nil)) ! (defun tramp-imap-buffer (v) ! (if (imap-opened (tramp-get-buffer v)) ! (tramp-get-buffer v) ! (let* ((server (tramp-file-name-real-host v)) ! (port (tramp-file-name-port v)) (auth-info (auth-source-user-or-password '("login" "password") server port)) (auth-user (nth 0 auth-info)) (auth-passwd (nth 1 auth-info)) ! (buffer (imap-open ! server ! port ;; this is the only place where IMAP vs IMAPS matters ! (if (string= (tramp-file-name-method v) tramp-imap-method) nil 'ssl) nil ! (tramp-get-buffer v)))) (imap-authenticate auth-user auth-passwd buffer) buffer))) --- 445,468 ---- (defun tramp-imap-file-name-truename (vec) (tramp-imap-file-name-mailbox-or-name vec nil)) ! (defun tramp-imap-buffer (vec) ! (if (imap-opened (tramp-get-buffer vec)) ! (tramp-get-buffer vec) ! (let* ((server (tramp-file-name-real-host vec)) ! (port (tramp-file-name-port vec)) (auth-info (auth-source-user-or-password '("login" "password") server port)) (auth-user (nth 0 auth-info)) (auth-passwd (nth 1 auth-info)) ! (buffer (imap-open ! server ! port ;; this is the only place where IMAP vs IMAPS matters ! (if (string= (tramp-file-name-method vec) tramp-imap-method) nil 'ssl) nil ! (tramp-get-buffer vec)))) (imap-authenticate auth-user auth-passwd buffer) buffer))) *************** *** 462,468 **** ;;; (with-current-buffer (tramp-imap-buffer (tramp-dissect-file-name "/imap:yourhosthere.com:/INBOX.test/new")) (tramp-imap-put-file "INBOX.test" "/etc/fstab" "new5")) (defun tramp-imap-put-file (mailbox filename-or-buffer &optional subject) ! (imap-message-append mailbox ;;TODO: use better buffer name (tramp-imap-message-buffer "*tramp-imap-encode*" filename-or-buffer subject))) --- 470,476 ---- ;;; (with-current-buffer (tramp-imap-buffer (tramp-dissect-file-name "/imap:yourhosthere.com:/INBOX.test/new")) (tramp-imap-put-file "INBOX.test" "/etc/fstab" "new5")) (defun tramp-imap-put-file (mailbox filename-or-buffer &optional subject) ! (imap-message-append mailbox ;;TODO: use better buffer name (tramp-imap-message-buffer "*tramp-imap-encode*" filename-or-buffer subject))) *************** *** 479,504 **** (tramp-imap-decode-buffer)))) (defun tramp-imap-delete-files (entries) ! (with-current-buffer (tramp-imap-buffer v) ! (dolist (entry entries) ! ;; TODO: this should be aware of IMAP large integers ! (when (integerp (nth 11 entry)) ! (imap-message-flags-add (format "%d" (nth 11 entry)) ! "\\Seen \\Deleted"))) ! (imap-mailbox-expunge))) ;; TODO: make it use tramp-imap-get-message-details, so it's compatible with non-IMAP4rev1 IMAP servers ;;; (with-current-buffer (tramp-imap-buffer (tramp-dissect-file-name "/imap:yourhosthere.com:/INBOX.test/new")) (tramp-imap-get-message-subject 24)) (defun tramp-imap-get-message-subject (msgs) "Get message subject over IMAP." ! (let ((allsubject ! (tramp-imap-ms-strip-cr ! (nth 2 (nth 0 (imap-fetch msgs ! "BODY[HEADER.FIELDS (SUBJECT)]" 'BODYDETAIL)))))) ! (when (string-match ! (format "Subject: %s \\(.+\\)" tramp-imap-subject-marker) allsubject) (match-string 1 allsubject)))) --- 487,511 ---- (tramp-imap-decode-buffer)))) (defun tramp-imap-delete-files (entries) ! (dolist (entry entries) ! ;; TODO: this should be aware of IMAP large integers ! (when (integerp (nth 11 entry)) ! (imap-message-flags-add (format "%d" (nth 11 entry)) ! "\\Seen \\Deleted"))) ! (imap-mailbox-expunge)) ;; TODO: make it use tramp-imap-get-message-details, so it's compatible with non-IMAP4rev1 IMAP servers ;;; (with-current-buffer (tramp-imap-buffer (tramp-dissect-file-name "/imap:yourhosthere.com:/INBOX.test/new")) (tramp-imap-get-message-subject 24)) (defun tramp-imap-get-message-subject (msgs) "Get message subject over IMAP." ! (let ((allsubject ! (tramp-imap-ms-strip-cr ! (nth 2 (nth 0 (imap-fetch msgs ! "BODY[HEADER.FIELDS (SUBJECT)]" 'BODYDETAIL)))))) ! (when (string-match ! (format "Subject: %s \\(.+\\)" tramp-imap-subject-marker) allsubject) (match-string 1 allsubject)))) *************** *** 514,520 **** (defun tramp-imap-get-message-headers (msgs) "Get message headers over IMAP." ;; nil means "all headers" ! (tramp-imap-ms-strip-cr (nth 2 (nth 0 (tramp-imap-get-message-details msgs nil))))) ;; from nnheader.el --- 521,527 ---- (defun tramp-imap-get-message-headers (msgs) "Get message headers over IMAP." ;; nil means "all headers" ! (tramp-imap-ms-strip-cr (nth 2 (nth 0 (tramp-imap-get-message-details msgs nil))))) ;; from nnheader.el *************** *** 536,542 **** (defun tramp-imap-get-message-body (msgs) "Get message body over IMAP." ! (tramp-imap-ms-strip-cr (nth 2 (nth 0 (tramp-imap-get-message-details msgs 'body))))) (defun tramp-imap-get-message-details (msgs body-or-header) --- 543,549 ---- (defun tramp-imap-get-message-body (msgs) "Get message body over IMAP." ! (tramp-imap-ms-strip-cr (nth 2 (nth 0 (tramp-imap-get-message-details msgs 'body))))) (defun tramp-imap-get-message-details (msgs body-or-header) *************** *** 549,563 **** (if body "BODY.PEEK[TEXT]" "BODY.PEEK[HEADER]") (if body "RFC822.TEXT.PEEK" "RFC822.HEADER")) ;; the RECEIVE parameter ! (if i4r1 'BODYDETAIL (if body 'RFC822.TEXT 'RFC822.HEADER))))) ! ;;; (tramp-imap-collapse-name "a b c / where ; strange ! characters $ abound") ;;; => "abcwherestrangecharactersabound" (defun tramp-imap-collapse-name (name) "Return NAME with only [A-Za-z0-9] characters" ! (when name (replace-regexp-in-string "[^A-Za-z0-9]" "" name))) ;;; (tramp-imap-message-buffer "testimap" "/etc/fstab" "fstab") --- 556,570 ---- (if body "BODY.PEEK[TEXT]" "BODY.PEEK[HEADER]") (if body "RFC822.TEXT.PEEK" "RFC822.HEADER")) ;; the RECEIVE parameter ! (if i4r1 'BODYDETAIL (if body 'RFC822.TEXT 'RFC822.HEADER))))) ! ;;; (tramp-imap-collapse-name "a b c / where ; strange ! characters $ abound") ;;; => "abcwherestrangecharactersabound" (defun tramp-imap-collapse-name (name) "Return NAME with only [A-Za-z0-9] characters" ! (when name (replace-regexp-in-string "[^A-Za-z0-9]" "" name))) ;;; (tramp-imap-message-buffer "testimap" "/etc/fstab" "fstab") *************** *** 579,586 **** (message-setup `((To . "Tramp-IMAP <tramp-imap@tramp-imap.tramp.emacs.org>") (From . "Tramp-IMAP <tramp-imap@tramp-imap.tramp.emacs.org>") ! (Subject . ,(format ! "%s %s" tramp-imap-subject-marker (or subject (tramp-imap-collapse-name sname)))) ;; TODO: make sure this can handle non-ASCII data --- 586,593 ---- (message-setup `((To . "Tramp-IMAP <tramp-imap@tramp-imap.tramp.emacs.org>") (From . "Tramp-IMAP <tramp-imap@tramp-imap.tramp.emacs.org>") ! (Subject . ,(format ! "%s %s" tramp-imap-subject-marker (or subject (tramp-imap-collapse-name sname)))) ;; TODO: make sure this can handle non-ASCII data *************** *** 601,606 **** --- 608,614 ---- CONTEXT is the encryption/decryption EPG context. HANDBACK is just carried through. KEY-ID can be 'SYM or 'PIN among others." + ;; TODO: Variable `v' is not declared!!! Existing due to side-effects only. (let* ((server (tramp-file-name-real-host v)) (port "tramp-imap") ; this is NOT the server password! (auth-passwd *************** *** 613,625 **** ;; do we reuse it? (if (y-or-n-p "Reuse the passphrase? ") (copy-sequence tramp-imap-passphrase) ! ;; don't reuse: revert caching behavior to nil, erase passphrase, ;; call ourselves again (setq tramp-imap-passphrase-cache nil) (setq tramp-imap-passphrase nil) (tramp-imap-passphrase-callback-function context key-id handback)) (let ((p (if (eq key-id 'SYM) ! (read-passwd "Tramp-IMAP passphrase for symmetric encryption: " (eq (epg-context-operation context) 'encrypt) tramp-imap-passphrase) --- 621,633 ---- ;; do we reuse it? (if (y-or-n-p "Reuse the passphrase? ") (copy-sequence tramp-imap-passphrase) ! ;; don't reuse: revert caching behavior to nil, erase passphrase, ;; call ourselves again (setq tramp-imap-passphrase-cache nil) (setq tramp-imap-passphrase nil) (tramp-imap-passphrase-callback-function context key-id handback)) (let ((p (if (eq key-id 'SYM) ! (read-passwd "Tramp-IMAP passphrase for symmetric encryption: " (eq (epg-context-operation context) 'encrypt) tramp-imap-passphrase) *************** *** 628,643 **** "Tramp-IMAP passphrase for PIN: " (let ((entry (assoc key-id epg-user-id-alist))) (if entry ! (format "Tramp-IMAP passphrase for %s %s: " key-id (cdr entry)) (format "Tramp-IMAP passphrase for %s: " key-id)))) nil tramp-imap-passphrase)))) ! ;; if we have an answer, the passphrase has changed, ! ;; the user hasn't declined keeping the passphrase, ;; and they answer yes to keep it now... ! (when (and p (not (equal tramp-imap-passphrase p)) (not (eq tramp-imap-passphrase-cache 'never)) --- 636,651 ---- "Tramp-IMAP passphrase for PIN: " (let ((entry (assoc key-id epg-user-id-alist))) (if entry ! (format "Tramp-IMAP passphrase for %s %s: " key-id (cdr entry)) (format "Tramp-IMAP passphrase for %s: " key-id)))) nil tramp-imap-passphrase)))) ! ;; if we have an answer, the passphrase has changed, ! ;; the user hasn't declined keeping the passphrase, ;; and they answer yes to keep it now... ! (when (and p (not (equal tramp-imap-passphrase p)) (not (eq tramp-imap-passphrase-cache 'never)) [-- Attachment #3: Type: text/plain, Size: 143 bytes --] _______________________________________________ Tramp-devel mailing list Tramp-devel@gnu.org http://lists.gnu.org/mailman/listinfo/tramp-devel ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: Tramp-IMAP works and needs testing 2009-07-18 11:11 ` Michael Albinus @ 2009-09-07 4:13 ` Ted Zlatanov 2009-09-08 13:33 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-09-07 4:13 UTC (permalink / raw) To: ding; +Cc: tramp-devel [-- Attachment #1: Type: text/plain, Size: 4426 bytes --] On Sat, 18 Jul 2009 13:11:51 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: MA> - Several functions in `tramp-imap-file-name-handler-alist' are not MA> implemented yet. I'm trying to get basic file read/write working before those other functions. Should I just set them to nil for now? Sorry for the delay, also. I've been busy and the weekends are too short... MA> - I doubt there will be implementation for `shell-command', MA> `executable-find', and `process-file'. I've disabled them in MA> `tramp-imap-file-name-handler-alist'. OK. MA> - You have used `tramp-debug-message'. I recommend to use MA> `tramp-message' instead, the verbose levels allow a better analysis of MA> the debug buffer. OK. MA> - `tramp-imap-handle-file-directory-p' must not return nil in all MA> cases. When I apply (save-buffer), the imap folder is checked for MA> being a directory. As workaround, I've changed MA> `tramp-imap-handle-file-directory-p' to return `t', but it must be MA> more precise. This is actually tough, because an entry could be both. We could have the files INBOX.test/b and INBOX.test/b/c at the same time. I will have to make the directory name the mailbox name and everything above that the filename. Thus no subdirectories are allowed, which simplifies the logic. Otherwise I don't think Tramp can handle it. So: /imap:albinus@localhost:/INBOX.test = directory /imap:albinus@localhost:/INBOX.test/ = directory /imap:albinus@localhost:/INBOX.test/1 = file "1" /imap:albinus@localhost:/INBOX.test/1/2 = illegal file name I think that's sufficient for most users. I hate to kill functionality this way, but otherwise things will get too confusing. Does that sound OK to you? MA> - When I have opened "/imap:albinus@localhost:/INBOX.test/1", and try to MA> save it under the name "/imap:albinus@localhost:/INBOX.test/2", I get MA> Debugger entered--Lisp error: (none "bad name 1 or mailbox INBOX.test/2") MA> signal(none ("bad name 1 or mailbox INBOX.test/2")) MA> tramp-error(["imap" "albinus" "localhost" "/INBOX.test/2/1"] none "bad name %s or mailbox %s" "1" "INBOX.test/2") MA> (if (and name (imap-mailbox-select mbox)) (let (sname) (tramp-debug-message vec "looking for '%s'" search-name) (dolist ... ... ...)) (tramp-error vec (quote none) "bad name %s or mailbox %s" name mbox)) MA> (let* ((mbox ...) (name ...) (truename ...) (search-name ...) res) (tramp-debug-message vec "selecting mbox %s" mbox) (if (and name ...) (let ... ... ...) (tramp-error vec ... "bad name %s or mailbox %s" name mbox)) res) MA> (save-current-buffer (set-buffer (tramp-imap-buffer vec)) (let* (... ... ... ... res) (tramp-debug-message vec "selecting mbox %s" mbox) (if ... ... ...) res)) MA> (with-current-buffer (tramp-imap-buffer vec) (let* (... ... ... ... res) (tramp-debug-message vec "selecting mbox %s" mbox) (if ... ... ...) res)) MA> tramp-imap-get-file-entries(["imap" "albinus" "localhost" "/INBOX.test/2/1"] "/INBOX.test/2/1") MA> [...] OK. I think the code is a mess, I'll improve it as stated above and below. MA> - I've fixed a small error in `tramp-imap-handle-delete-file' and MA> `tramp-imap-delete-files'. MA> - `tramp-retrieve-data', used in `tramp-imap-handle-file-local-copy', MA> does not exist. MA> - In `tramp-imap-passphrase-callback-function', you use the non-existing MA> variable `v'. Thanks for these. For tramp-imap-retrieve-data I was looking at the tramp-fish.el code, which has that function. I think in general my code needs to be redone, it's too messy. I was just playing around when I put it together; the core functionality is OK but it's a mess of spaghetti code that I would hate to support. I'll try to rewrite it before the next Tramp release. I'm attaching the last spaghetti version, I hope. In the passphrase callback, I don't have access to the Tramp context. I only have the EPG context. Daiki Ueno: is it OK if I insert the Tramp context (basically a cons) at the end of the EPG context? It will make the EPG context longer by 1 element so I worry it may break things in EPG. If not, how can I pass the Tramp context to the passphrase callback? MA> - The email address of stored messages looks like "a real one", one MA> could try to send messages there. Better might be to use an invalid MA> address as defined in RFC 2606, like "Tramp-IMAP MA> <tramp-imap@tramp-imap.invalid>". OK. Ted [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 27603 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: Tramp-IMAP works and needs testing 2009-09-07 4:13 ` Ted Zlatanov @ 2009-09-08 13:33 ` Michael Albinus 2009-09-09 16:13 ` Ted Zlatanov 2009-09-21 19:28 ` imap-hash.el and tramp-imap.el Ted Zlatanov 0 siblings, 2 replies; 58+ messages in thread From: Michael Albinus @ 2009-09-08 13:33 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding [-- Attachment #1: Type: text/plain, Size: 3152 bytes --] Ted Zlatanov <tzz@lifelogs.com> writes: > On Sat, 18 Jul 2009 13:11:51 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: > > MA> - Several functions in `tramp-imap-file-name-handler-alist' are not > MA> implemented yet. > > I'm trying to get basic file read/write working before those other > functions. Should I just set them to nil for now? We could set them to `ignore', and add a comment. I've done it for all functions with a missing implementation. > MA> - `tramp-imap-handle-file-directory-p' must not return nil in all > MA> cases. When I apply (save-buffer), the imap folder is checked for > MA> being a directory. As workaround, I've changed > MA> `tramp-imap-handle-file-directory-p' to return `t', but it must be > MA> more precise. > > This is actually tough, because an entry could be both. We could have > the files INBOX.test/b and INBOX.test/b/c at the same time. I will have > to make the directory name the mailbox name and everything above that > the filename. Thus no subdirectories are allowed, which simplifies the > logic. Otherwise I don't think Tramp can handle it. So: > > /imap:albinus@localhost:/INBOX.test = directory > /imap:albinus@localhost:/INBOX.test/ = directory > /imap:albinus@localhost:/INBOX.test/1 = file "1" > /imap:albinus@localhost:/INBOX.test/1/2 = illegal file name > > I think that's sufficient for most users. I hate to kill functionality > this way, but otherwise things will get too confusing. Does that sound > OK to you? OK. I've added a corresponding implementation. Problems: `tramp-imap-buffer' can return nil, if the connection to IMAP fails. You always use (with-current-buffer (tramp-imap-buffer vec) ...), which gives an error then. It might be better to check the result of `tramp-imap-buffer' first. `tramp-imap-handle-name-all-completions' does not complete mailbox names. If you apply `write-file', and the file exists already on the IMAP server, a second file with the same name (subject) will be written instead of overwriting the existing one. In `tramp-imap-buffer', you use auth-source in order to determine user and password. I believe, it shall be rather done via `tramp-read-passwd' (which includes auth-source). By this, the user name given in the Tramp filename is respected. Furthermore, I plan to add more password handling mechanisms there, for example the Secret Service API for gnome-keyring and kwallet. Furthermore, I've changed the following: - Change the copyright owner to FSF. I hope you are OK with this; otherwise we have no chance to add your package to Tramp and Emacs. - Make `tramp-imap-method' and `tramp-imaps-method' a defconst. I know that in some other tramp-*.el files the methods are also defcustom's, but I don't believe it is necessary (and I shall change it everywhere). - Change implementation of `tramp-imap-handle-file-readable-p ' and `tramp-imap-handle-file-writable-p'. After all, it is working already pretty fine. I believe we shall add it to Tramp's CVS repository, and likely we shall add it also to the upcoming Tramp 2.1.17 release (maybe marked as experimental). > Ted Best regards, Michael. [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 27836 bytes --] [-- Attachment #3: Type: text/plain, Size: 143 bytes --] _______________________________________________ Tramp-devel mailing list Tramp-devel@gnu.org http://lists.gnu.org/mailman/listinfo/tramp-devel ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: Tramp-IMAP works and needs testing 2009-09-08 13:33 ` Michael Albinus @ 2009-09-09 16:13 ` Ted Zlatanov 2009-09-10 7:18 ` Michael Albinus 2009-09-21 19:28 ` imap-hash.el and tramp-imap.el Ted Zlatanov 1 sibling, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-09-09 16:13 UTC (permalink / raw) To: Michael Albinus; +Cc: tramp-devel, ding On Tue, 08 Sep 2009 15:33:11 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: ... MA> Furthermore, I've changed the following: MA> - Change the copyright owner to FSF. I hope you are OK with this; MA> otherwise we have no chance to add your package to Tramp and Emacs. MA> - Make `tramp-imap-method' and `tramp-imaps-method' a defconst. I know MA> that in some other tramp-*.el files the methods are also defcustom's, MA> but I don't believe it is necessary (and I shall change it everywhere). MA> - Change implementation of `tramp-imap-handle-file-readable-p ' and MA> `tramp-imap-handle-file-writable-p'. MA> After all, it is working already pretty fine. I believe we shall add it MA> to Tramp's CVS repository, and likely we shall add it also to the MA> upcoming Tramp 2.1.17 release (maybe marked as experimental). Thanks for all your help. I am working on a storage layer (imap-hash.el) that will let me treat a mailbox as a hash table, making retrieval and storage much easier. That way I can make tramp-imap.el correspondingly simpler. I will incorporate all your changes as well; my changes are at the storage layer and don't interfere with the Tramp functionality. I am working hard on this so I expect imap-hash.el to be ready soon. Afterwards I can merge your changes and overhaul the tramp-imap.el code. Can you hold on before adding the old tramp-imap.el to Tramp? I don't want my bad code harming other programmers' minds ;) I think I can make the 2.1.17 release; I'll follow up with a progress report in a week if I'm not done by then. imap-hash.el is relevant to the Gnus project as well. I plan to use it to store things like the Gnus registry and maybe the Gnus newsrc file, essentially providing a distributed hashtable in an IMAP mailbox. This was another motive for developing a separate storage layer. Thanks Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: Tramp-IMAP works and needs testing 2009-09-09 16:13 ` Ted Zlatanov @ 2009-09-10 7:18 ` Michael Albinus 0 siblings, 0 replies; 58+ messages in thread From: Michael Albinus @ 2009-09-10 7:18 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding Ted Zlatanov <tzz@lifelogs.com> writes: > MA> After all, it is working already pretty fine. I believe we shall add it > MA> to Tramp's CVS repository, and likely we shall add it also to the > MA> upcoming Tramp 2.1.17 release (maybe marked as experimental). > > Can you hold on before adding the old tramp-imap.el to Tramp? I don't > want my bad code harming other programmers' minds ;) I think I can > make the 2.1.17 release; I'll follow up with a progress report in a week > if I'm not done by then. That's easy. Starting on Saturday, I'll be almost offline for a week. > Thanks > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* imap-hash.el and tramp-imap.el 2009-09-08 13:33 ` Michael Albinus 2009-09-09 16:13 ` Ted Zlatanov @ 2009-09-21 19:28 ` Ted Zlatanov 2009-09-22 14:52 ` Michael Albinus 2009-09-24 6:44 ` Reiner Steib 1 sibling, 2 replies; 58+ messages in thread From: Ted Zlatanov @ 2009-09-21 19:28 UTC (permalink / raw) To: ding; +Cc: tramp-devel [-- Attachment #1: Type: text/plain, Size: 1007 bytes --] Attached are imap-hash.el and tramp-imap.el, I worked on them this weekend. imap-hash.el lets you treat an IMAP mailbox as a hash. It lets you get individual keys' headers and body data (keys are message UIDs) or map a function across *all the messages in the mailbox*. The function is only called for those that match a given subject, but really I should be using SEARCH. So the library is not optimized for speed. Also, it does not handle invalid mailbox names gracefully yet. Consider it an alpha version. I plan to eventually use imap-hash.el to store distributed Gnus data, specifically the Gnus registry, the newsrc file, scoring files, etc. tramp-imap.el uses imap-hash.el to provide a Tramp interface to read and write files as messages in an IMAP mailbox. It's also an alpha version, but it will at least let you (given a valid IMAP mailbox) read and write directory contents. It uses EPG (thanks to Daiki Ueno's help) to encrypt the file contents when they are stored as messages. Ted [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 25494 bytes --] [-- Attachment #3: imap-hash.el --] [-- Type: application/emacs-lisp, Size: 14078 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-21 19:28 ` imap-hash.el and tramp-imap.el Ted Zlatanov @ 2009-09-22 14:52 ` Michael Albinus 2009-09-23 21:07 ` Ted Zlatanov 2009-09-24 6:44 ` Reiner Steib 1 sibling, 1 reply; 58+ messages in thread From: Michael Albinus @ 2009-09-22 14:52 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding [-- Attachment #1: Type: text/plain, Size: 2149 bytes --] Ted Zlatanov <tzz@lifelogs.com> writes: Hi Ted, > Attached are imap-hash.el and tramp-imap.el, I worked on them this > weekend. Thanks for this. I've played a little bit with them (no real test yet). > imap-hash.el lets you treat an IMAP mailbox as a hash. It lets you get > individual keys' headers and body data (keys are message UIDs) or map a > function across *all the messages in the mailbox*. The function is only > called for those that match a given subject, but really I should be > using SEARCH. So the library is not optimized for speed. Also, it does > not handle invalid mailbox names gracefully yet. Consider it an alpha > version. Could you, please, add "(require 'assoc)"? It's not loaded by default. > tramp-imap.el uses imap-hash.el to provide a Tramp interface to read and > write files as messages in an IMAP mailbox. It's also an alpha version, > but it will at least let you (given a valid IMAP mailbox) read and write > directory contents. It uses EPG (thanks to Daiki Ueno's help) to > encrypt the file contents when they are stored as messages. As usual, I have some few patches :-) * imaps didn't work for me any longer. I've patched `tramp-imap-make-iht' until it worked ... maybe my sledge-hammer patch could be done more elegant. There is still an error message in the mini-buffer when opening the IMAP folder, but it doesn't seem to hurt. * I have renamed `tramp-imap-handle-file-inode' to `tramp-imap-get-file-inode'. tramp-*-handle-* function names shall be reserved to basic file name operations implementation. * I have used `tramp-current-host' to solve the problem of unresolved vector in `tramp-imap-passphrase-callback-function'. That variable was introduced in tramp.el for exactly this kind of problems. * When you byte-compile tramp-imap.el, there is a warning about unknown function `tramp-imap-get-message-headers'. No idea, what to do here. My patch is appended. I really would like to add tramp-imap.el to the CVS repository. I test on different machines, and the current situation is inconvenient for me. Do you agree for a checkin? > Ted Best regards, Michael. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Type: text/x-patch, Size: 11446 bytes --] *** /tmp/ediff21621-xQ 2009-09-22 16:49:14.000000000 +0200 --- /tmp/ediff21621L8W 2009-09-22 16:49:14.000000000 +0200 *************** *** 260,266 **** "-rw-rw-rw-" nil uid ! 1))) iht t))) (defun tramp-imap-handle-write-region (start end filename &optional append visit lockname confirm) --- 260,266 ---- "-rw-rw-rw-" nil uid ! 1))) iht t))) (defun tramp-imap-handle-write-region (start end filename &optional append visit lockname confirm) *************** *** 275,281 **** (tramp-error v 'file-error "File not overwritten"))) (tramp-flush-file-property v localname) (let* ((old-buffer (current-buffer)) ! (inode (tramp-imap-handle-file-inode filename)) (min 1) (max (point-max)) ;; make sure we have good start and end values --- 275,281 ---- (tramp-error v 'file-error "File not overwritten"))) (tramp-flush-file-property v localname) (let* ((old-buffer (current-buffer)) ! (inode (tramp-imap-get-file-inode filename)) (min 1) (max (point-max)) ;; make sure we have good start and end values *************** *** 290,298 **** (with-current-buffer old-buffer (buffer-substring-no-properties start end))) (current-buffer))) ! (tramp-imap-put-file v ! temp-buffer ! (tramp-imap-file-name-name v) inode))) (when (eq visit t) (set-visited-file-modtime)))) --- 290,298 ---- (with-current-buffer old-buffer (buffer-substring-no-properties start end))) (current-buffer))) ! (tramp-imap-put-file v ! temp-buffer ! (tramp-imap-file-name-name v) inode))) (when (eq visit t) (set-visited-file-modtime)))) *************** *** 307,313 **** (with-parsed-tramp-file-name filename nil (if (not (file-exists-p filename)) (tramp-error ! v 'file-error "File '%s' not found on remote host" filename) (let ((point (point)) size data) (tramp-message v 4 "Fetching file %s..." filename) --- 307,313 ---- (with-parsed-tramp-file-name filename nil (if (not (file-exists-p filename)) (tramp-error ! v 'file-error "File `%s' not found on remote host" filename) (let ((point (point)) size data) (tramp-message v 4 "Fetching file %s..." filename) *************** *** 339,346 **** ;; (cdr (nth 0 (tramp-imap-get-file-entries v localname)))) (cdr-safe (nth 0 (tramp-imap-get-file-entries v localname))))) ! (defun tramp-imap-handle-file-inode (filename &optional id-format) ! "Get inode equivalent (actually the UID) for Tramp-IMAP FILENAME." (nth 10 (file-attributes filename))) (defun tramp-imap-handle-file-executable-p (filename) --- 339,346 ---- ;; (cdr (nth 0 (tramp-imap-get-file-entries v localname)))) (cdr-safe (nth 0 (tramp-imap-get-file-entries v localname))))) ! (defun tramp-imap-get-file-inode (filename &optional id-format) ! "Get inode equivalent \(actually the UID) for Tramp-IMAP FILENAME." (nth 10 (file-attributes filename))) (defun tramp-imap-handle-file-executable-p (filename) *************** *** 361,367 **** ((not (file-exists-p file)) nil) (t (with-parsed-tramp-file-name (expand-file-name file) nil (let ((iht (tramp-imap-make-iht v))) ! (imap-hash-rem (tramp-imap-handle-file-inode filename) iht)))))) ;; TODO: fix this in tramp-imap-get-file-entries (defun tramp-imap-handle-file-newer-than-file-p (file1 file2) --- 361,367 ---- ((not (file-exists-p file)) nil) (t (with-parsed-tramp-file-name (expand-file-name file) nil (let ((iht (tramp-imap-make-iht v))) ! (imap-hash-rem (tramp-imap-get-file-inode filename) iht)))))) ;; TODO: fix this in tramp-imap-get-file-entries (defun tramp-imap-handle-file-newer-than-file-p (file1 file2) *************** *** 387,402 **** tmpfile)))) (defun tramp-imap-put-file (vec filename-or-buffer &optional subject inode) ! "Write contents of FILENAME-OR-BUFFER to Tramp-IMAP file VEC with name SUBJECT. When INODE is given, delete that old remote file after writing the new one ! (normally this is the old file with the same name)." ! (let ((iht (tramp-imap-make-iht vec))) (imap-hash-put (list (list (cons 'Subject ! (format ! "%s%s" ! tramp-imap-subject-marker (or subject "no subject")))) (cond ((bufferp filename-or-buffer) (with-current-buffer filename-or-buffer --- 387,404 ---- tmpfile)))) (defun tramp-imap-put-file (vec filename-or-buffer &optional subject inode) ! "Write contents of FILENAME-OR-BUFFER to Tramp-IMAP file VEC with name SUBJECT. When INODE is given, delete that old remote file after writing the new one ! \(normally this is the old file with the same name)." ! ;; `tramp-current-host' is used in `tramp-imap-passphrase-callback-function'. ! (let ((tramp-current-host (tramp-file-name-real-host vec)) ! (iht (tramp-imap-make-iht vec))) (imap-hash-put (list (list (cons 'Subject ! (format ! "%s%s" ! tramp-imap-subject-marker (or subject "no subject")))) (cond ((bufferp filename-or-buffer) (with-current-buffer filename-or-buffer *************** *** 408,435 **** (defun tramp-imap-get-file (filename) ! ;; (debug (tramp-imap-handle-file-inode filename)) (with-parsed-tramp-file-name (expand-file-name filename) nil (condition-case () ! (let ((iht (tramp-imap-make-iht v)) ! (inode (tramp-imap-handle-file-inode filename)) ! (data (imap-hash-get (tramp-imap-handle-file-inode ! filename) ! iht t))) (with-temp-buffer (insert (nth 1 data)) ;;(debug inode (buffer-string)) (tramp-imap-decode-buffer))) (error (tramp-error ! v 'file-error "File '%s' could not be read" filename))))) (defun tramp-imap-passphrase-callback-function (context key-id handback) "Called by EPG to get a passphrase for Tramp-IMAP. CONTEXT is the encryption/decryption EPG context. HANDBACK is just carried through. KEY-ID can be 'SYM or 'PIN among others." ! ;; TODO: Variable `v' is not declared!!! Existing due to side-effects only. ! (let* ((server (tramp-file-name-real-host v)) (port "tramp-imap") ; this is NOT the server password! (auth-passwd (auth-source-user-or-password "password" server port))) --- 410,437 ---- (defun tramp-imap-get-file (filename) ! ;; (debug (tramp-imap-get-file-inode filename)) (with-parsed-tramp-file-name (expand-file-name filename) nil (condition-case () ! ;; `tramp-current-host' is used in ! ;; `tramp-imap-passphrase-callback-function'. ! (let* ((tramp-current-host (tramp-file-name-real-host v)) ! (iht (tramp-imap-make-iht v)) ! (inode (tramp-imap-get-file-inode filename)) ! (data (imap-hash-get inode iht t))) (with-temp-buffer (insert (nth 1 data)) ;;(debug inode (buffer-string)) (tramp-imap-decode-buffer))) (error (tramp-error ! v 'file-error "File `%s' could not be read" filename))))) (defun tramp-imap-passphrase-callback-function (context key-id handback) "Called by EPG to get a passphrase for Tramp-IMAP. CONTEXT is the encryption/decryption EPG context. HANDBACK is just carried through. KEY-ID can be 'SYM or 'PIN among others." ! (let* ((server tramp-current-host) (port "tramp-imap") ; this is NOT the server password! (auth-passwd (auth-source-user-or-password "password" server port))) *************** *** 532,545 **** (defun tramp-imap-make-iht (vec &optional needed-subject) "Translate the Tramp vector VEC to the imap-hash structure. With NEEDED-SUBJECT, alters the imap-hash test accordingly." ! (let ((mbox (tramp-imap-file-name-mailbox vec)) ! (server (tramp-file-name-real-host vec)) ! (port (or (tramp-file-name-port vec) "imap"))) ;; return the IHT with a test override to look for the subject marker ! (plist-put (imap-hash-make server port mbox) ! :test (format "^%s%s" ! tramp-imap-subject-marker ! (if needed-subject needed-subject ""))))) (provide 'tramp-imap) ;;; tramp-imap.el ends here --- 534,550 ---- (defun tramp-imap-make-iht (vec &optional needed-subject) "Translate the Tramp vector VEC to the imap-hash structure. With NEEDED-SUBJECT, alters the imap-hash test accordingly." ! (let* ((mbox (tramp-imap-file-name-mailbox vec)) ! (server (tramp-file-name-real-host vec)) ! (ssl (string-equal (tramp-file-name-method vec) tramp-imaps-method)) ! (port (or (tramp-file-name-port vec) (if ssl 993 143)))) ;; return the IHT with a test override to look for the subject marker ! (plist-put ! (plist-put (imap-hash-make server port mbox) ! :test (format "^%s%s" ! tramp-imap-subject-marker ! (if needed-subject needed-subject ""))) ! :ssl ssl))) (provide 'tramp-imap) ;;; tramp-imap.el ends here *************** *** 553,559 **** ;;; (tramp-imap-get-file-entries (tramp-dissect-file-name "/imap:yourhosthere.com:/test/") t) ;;; (tramp-imap-get-file-entries (tramp-dissect-file-name "/imap:yourhosthere.com:/test/welcommen") t) ;;; (tramp-imap-get-file-entries (tramp-dissect-file-name "/imap:yourhosthere.com:/test/welcommen") t t) ! ;;;(tramp-imap-handle-file-inode "/imap:yourhosthere.com:/test/welcome") ;;; (dired-copy-file "/etc/fstab" "/imap:yourhosthere.com:/test/welcome" t) ;;; (write-region 1 100 "/imap:yourhosthere.com:/test/welcome") ;;; (tramp-imap-get-file "/imap:yourhosthere.com:/test/welcome") --- 558,564 ---- ;;; (tramp-imap-get-file-entries (tramp-dissect-file-name "/imap:yourhosthere.com:/test/") t) ;;; (tramp-imap-get-file-entries (tramp-dissect-file-name "/imap:yourhosthere.com:/test/welcommen") t) ;;; (tramp-imap-get-file-entries (tramp-dissect-file-name "/imap:yourhosthere.com:/test/welcommen") t t) ! ;;;(tramp-imap-get-file-inode "/imap:yourhosthere.com:/test/welcome") ;;; (dired-copy-file "/etc/fstab" "/imap:yourhosthere.com:/test/welcome" t) ;;; (write-region 1 100 "/imap:yourhosthere.com:/test/welcome") ;;; (tramp-imap-get-file "/imap:yourhosthere.com:/test/welcome") *************** *** 574,581 **** ;;;(file-exists-p "/imap:yourhosthere.com:/test/welcome2") ;;;(setq tramp-cache-data (make-hash-table :test 'equal)) ;;;(tramp-imap-handle-file-attributes "/imap:yourhosthere.com:/test/welcome") ! ;;;(tramp-imap-handle-file-inode "/imap:yourhosthere.com:/test/welcommen") ! ;;;(tramp-imap-handle-file-inode "/imap:yourhosthere.com:/test/welcome") ;;;(file-writable-p "/imap:yourhosthere.com:/test/welcome2") ;;; (delete-file "/imap:yourhosthere.com:/test/welcome") ;;; (tramp-imap-get-file "/imap:yourhosthere.com:/test/welcommen") --- 579,586 ---- ;;;(file-exists-p "/imap:yourhosthere.com:/test/welcome2") ;;;(setq tramp-cache-data (make-hash-table :test 'equal)) ;;;(tramp-imap-handle-file-attributes "/imap:yourhosthere.com:/test/welcome") ! ;;;(tramp-imap-get-file-inode "/imap:yourhosthere.com:/test/welcommen") ! ;;;(tramp-imap-get-file-inode "/imap:yourhosthere.com:/test/welcome") ;;;(file-writable-p "/imap:yourhosthere.com:/test/welcome2") ;;; (delete-file "/imap:yourhosthere.com:/test/welcome") ;;; (tramp-imap-get-file "/imap:yourhosthere.com:/test/welcommen") [-- Attachment #3: Type: text/plain, Size: 143 bytes --] _______________________________________________ Tramp-devel mailing list Tramp-devel@gnu.org http://lists.gnu.org/mailman/listinfo/tramp-devel ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-22 14:52 ` Michael Albinus @ 2009-09-23 21:07 ` Ted Zlatanov 2009-09-24 10:38 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-09-23 21:07 UTC (permalink / raw) To: ding; +Cc: tramp-devel [-- Attachment #1: Type: text/plain, Size: 2126 bytes --] On Tue, 22 Sep 2009 16:52:59 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Could you, please, add "(require 'assoc)"? It's not loaded by default. OK. MA> * imaps didn't work for me any longer. I've patched MA> `tramp-imap-make-iht' until it worked ... maybe my sledge-hammer patch MA> could be done more elegant. There is still an error message in the MA> mini-buffer when opening the IMAP folder, but it doesn't seem to hurt. Looks OK. I haven't tested imaps thoroughly and it's not crucial to releasing IMO. The approach is OK; I want the clients of imap-hash.el to modify the IHT as needed after it's created, so it can carry extra information as appropriate to the particular client. MA> * I have renamed `tramp-imap-handle-file-inode' to MA> `tramp-imap-get-file-inode'. tramp-*-handle-* function names shall be MA> reserved to basic file name operations implementation. MA> * I have used `tramp-current-host' to solve the problem of unresolved MA> vector in `tramp-imap-passphrase-callback-function'. That variable MA> was introduced in tramp.el for exactly this kind of problems. Great. Thanks so much for looking into this. MA> * When you byte-compile tramp-imap.el, there is a warning about unknown MA> function `tramp-imap-get-message-headers'. No idea, what to do here. I don't see that warning. I do see: tramp-imap.el:364:53:Warning: reference to free variable `filename' which is strange, because with-parsed-tramp-file-name should set `filename'. MA> My patch is appended. MA> I really would like to add tramp-imap.el to the CVS repository. I test MA> on different machines, and the current situation is inconvenient for MA> me. Do you agree for a checkin? That's fine now, it's sufficiently presentable and maintaineable. Latest attached for your review; no changes other than the require 'assoc. I'll ask on emacs-devel about imap-hash.el; it will be used in Tramp and Gnus so I would prefer to host it inside Emacs itself, probably under lisp/net/ together with imap.el. Tramp, in order to be self-sufficient on XEmacs, may need to package it as well. Ted [-- Attachment #2: tramp-imap.el --] [-- Type: application/emacs-lisp, Size: 25739 bytes --] ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-23 21:07 ` Ted Zlatanov @ 2009-09-24 10:38 ` Michael Albinus 2009-09-25 20:48 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Michael Albinus @ 2009-09-24 10:38 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding Ted Zlatanov <tzz@lifelogs.com> writes: > MA> * imaps didn't work for me any longer. I've patched > MA> `tramp-imap-make-iht' until it worked ... maybe my sledge-hammer patch > MA> could be done more elegant. There is still an error message in the > MA> mini-buffer when opening the IMAP folder, but it doesn't seem to hurt. > > Looks OK. I haven't tested imaps thoroughly and it's not crucial to > releasing IMO. The approach is OK; I want the clients of imap-hash.el > to modify the IHT as needed after it's created, so it can carry extra > information as appropriate to the particular client. I have polished my change a little bit. The default port numbers are kept now in `tramp-methods', like it is done for other methods as well. And I also update the user tag in the structure. > MA> * When you byte-compile tramp-imap.el, there is a warning about unknown > MA> function `tramp-imap-get-message-headers'. No idea, what to do here. > > I don't see that warning. I do see: > > tramp-imap.el:364:53:Warning: reference to free variable `filename' > > which is strange, because with-parsed-tramp-file-name should set > `filename'. This I have fixed. However, there is still the warning for `tramp-imap-get-message-headers. You could check the source code; this function does not seem to be defined, indeed. > MA> I really would like to add tramp-imap.el to the CVS repository. I test > MA> on different machines, and the current situation is inconvenient for > MA> me. Do you agree for a checkin? > > That's fine now, it's sufficiently presentable and maintaineable. > Latest attached for your review; no changes other than the require 'assoc. Thanks. There seems to be a misunderstanding; 'assoc needs to be required in imap-hash.el, not in tramp-imap.el. I have added tramp-imap.el to Tramp's CVS now. It is also loaded from tramp.el, when imap-hash.el is available. There are some other cleanups I have applied before checkin; you might check whether there is something I have misunderstood ... In tramp.texi, I have added some short sentences about imap and imaps methods. I'm not so good in documentation; maybe you find a better wording. > I'll ask on emacs-devel about imap-hash.el; it will be used in Tramp and > Gnus so I would prefer to host it inside Emacs itself, probably under > lisp/net/ together with imap.el. Tramp, in order to be self-sufficient > on XEmacs, may need to package it as well. OK. Once imap-hash.el has arrived Emacs CVS, I will synchronize the corresponding changes from Tramp. With XEmacs, we could wait until it is requested. Tramp hasn't even been upgraded to 2.1 in the XEmacs package repository (due to license reasons), so I'm a little bit lazy to work on it. Of course, I won't protest if somebody else works on the XEmacs part :-) I will continue to test tramp-imap.el. And likely, I will add an implementation for `tramp-imap-insert-directory'; I'm using dired a lot. > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-24 10:38 ` Michael Albinus @ 2009-09-25 20:48 ` Ted Zlatanov 2009-09-27 15:57 ` Ted Zlatanov 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-09-25 20:48 UTC (permalink / raw) To: ding; +Cc: tramp-devel On Thu, 24 Sep 2009 12:38:43 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: MA> Ted Zlatanov <tzz@lifelogs.com> writes: MA> * imaps didn't work for me any longer. I've patched MA> `tramp-imap-make-iht' until it worked ... maybe my sledge-hammer patch MA> could be done more elegant. There is still an error message in the MA> mini-buffer when opening the IMAP folder, but it doesn't seem to hurt. >> >> Looks OK. I haven't tested imaps thoroughly and it's not crucial to >> releasing IMO. The approach is OK; I want the clients of imap-hash.el >> to modify the IHT as needed after it's created, so it can carry extra >> information as appropriate to the particular client. MA> I have polished my change a little bit. The default port numbers are MA> kept now in `tramp-methods', like it is done for other methods as MA> well. And I also update the user tag in the structure. OK. We'll assume Tramp's version of tramp-imap.el is authoritative from now on, and I'll commit my updates there. MA> This I have fixed. However, there is still the warning for MA> `tramp-imap-get-message-headers. You could check the source code; this MA> function does not seem to be defined, indeed. OK, I'll make a patch for this and also to reflect Reiner's comment about the message-mode body separator. MA> I really would like to add tramp-imap.el to the CVS repository. I test MA> on different machines, and the current situation is inconvenient for MA> me. Do you agree for a checkin? >> >> That's fine now, it's sufficiently presentable and maintaineable. >> Latest attached for your review; no changes other than the require 'assoc. MA> Thanks. There seems to be a misunderstanding; 'assoc needs to be MA> required in imap-hash.el, not in tramp-imap.el. Oops. I'll make the change in the imap-hash.el that's going into Emacs. MA> I have added tramp-imap.el to Tramp's CVS now. It is also loaded from MA> tramp.el, when imap-hash.el is available. There are some other cleanups MA> I have applied before checkin; you might check whether there is MA> something I have misunderstood ... OK, something to do over the weekend. MA> In tramp.texi, I have added some short sentences about imap and imaps MA> methods. I'm not so good in documentation; maybe you find a better MA> wording. OK. MA> With XEmacs, we could wait until it is requested. Tramp hasn't even been MA> upgraded to 2.1 in the XEmacs package repository (due to license MA> reasons), so I'm a little bit lazy to work on it. Of course, I won't MA> protest if somebody else works on the XEmacs part :-) MA> I will continue to test tramp-imap.el. And likely, I will add an MA> implementation for `tramp-imap-insert-directory'; I'm using dired a lot. Wonderful. I'll soon improve imap-hash.el to add SEARCH when the :test key is set in the IHT, instead of FETCH, so only the matching messages are retrieved. Also I need to fix the error when a nonexistent mailbox is specified. If you see any other issues, please let me know. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-25 20:48 ` Ted Zlatanov @ 2009-09-27 15:57 ` Ted Zlatanov 2009-09-27 19:24 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Ted Zlatanov @ 2009-09-27 15:57 UTC (permalink / raw) To: tramp-devel; +Cc: ding On Fri, 25 Sep 2009 15:48:07 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: MA> This I have fixed. However, there is still the warning for MA> `tramp-imap-get-message-headers. You could check the source code; this MA> function does not seem to be defined, indeed. I removed the file-local-copy handler. From looking at the source it seems that it's automatically filled-in by Tramp, so the implementation was not necessary. The function I removed (commented out until you decide what's next) was left over from the old tramp-imap and I didn't hit it in any of my testing; sorry for the annoyance. TZ> OK, I'll make a patch for this and also to reflect Reiner's comment TZ> about the message-mode body separator. This was done in imap-hash.el, which is now in the Emacs CVS. I didn't do the docs or much else. I did see your comment fixes--thanks for cleaning them up. Your other changes look fine too. Thanks Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-27 15:57 ` Ted Zlatanov @ 2009-09-27 19:24 ` Michael Albinus 2009-09-28 12:03 ` Michael Albinus 0 siblings, 1 reply; 58+ messages in thread From: Michael Albinus @ 2009-09-27 19:24 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding Ted Zlatanov <tzz@lifelogs.com> writes: > MA> This I have fixed. However, there is still the warning for > MA> `tramp-imap-get-message-headers. You could check the source code; this > MA> function does not seem to be defined, indeed. > > I removed the file-local-copy handler. From looking at the source it > seems that it's automatically filled-in by Tramp, so the implementation > was not necessary. The function I removed (commented out until you > decide what's next) was left over from the old tramp-imap and I didn't > hit it in any of my testing; sorry for the annoyance. OK, I'll check it. I've checked in some changes as well. * 'assoc must also be required in tramp-imap.el, you were right. * I've implemented handlers for `copy-file', `insert-directory' and `rename-file'. `copy-file' and `rename-file' need optimization, when both files are located on the IMAP server, maybe you can have a look on them. I did also extend `tramp-imap-put-file' and `tramp-imap-get-file' because of this. * Some few other, minor fixes. Next days, after some further tests, I'll bring tramp-imap.el to Emacs CVS. > Thanks > Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-27 19:24 ` Michael Albinus @ 2009-09-28 12:03 ` Michael Albinus 0 siblings, 0 replies; 58+ messages in thread From: Michael Albinus @ 2009-09-28 12:03 UTC (permalink / raw) To: Ted Zlatanov; +Cc: tramp-devel, ding Michael Albinus <michael.albinus@gmx.de> writes: > Next days, after some further tests, I'll bring tramp-imap.el to Emacs CVS. I've added handlers for `directory-files-and-attributes' and `file-local-copy'. Furthermore, I've added a TODO section at the end of the file, containing some problems I've observed during testing. Maybe you can have a look on them; all those items are out of my capabilities. tramp-imap.el is added now to Emacs CVS as well. >> Thanks >> Ted Best regards, Michael. ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-21 19:28 ` imap-hash.el and tramp-imap.el Ted Zlatanov 2009-09-22 14:52 ` Michael Albinus @ 2009-09-24 6:44 ` Reiner Steib 2009-09-25 20:49 ` Ted Zlatanov 1 sibling, 1 reply; 58+ messages in thread From: Reiner Steib @ 2009-09-24 6:44 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, tramp-devel On Mon, Sep 21 2009, Ted Zlatanov wrote: > ;; TODO: make this search better > (if (search-forward "--text follows this line--" nil t) > (delete-region (line-beginning-position) (line-end-position)) > (error "Could not find the body separator in the encoded message!")))) Why not use `mail-header-separator'? Bye, Reiner. -- ,,, (o o) ---ooO-(_)-Ooo--- | PGP key available | http://rsteib.home.pages.de/ ^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: imap-hash.el and tramp-imap.el 2009-09-24 6:44 ` Reiner Steib @ 2009-09-25 20:49 ` Ted Zlatanov 0 siblings, 0 replies; 58+ messages in thread From: Ted Zlatanov @ 2009-09-25 20:49 UTC (permalink / raw) To: ding; +Cc: tramp-devel On Thu, 24 Sep 2009 08:44:50 +0200 Reiner Steib <reinersteib+gmane@imap.cc> wrote: RS> On Mon, Sep 21 2009, Ted Zlatanov wrote: >> ;; TODO: make this search better >> (if (search-forward "--text follows this line--" nil t) >> (delete-region (line-beginning-position) (line-end-position)) >> (error "Could not find the body separator in the encoded message!")))) RS> Why not use `mail-header-separator'? I looked for "message-*" variable names, didn't find anything relevant, put a TODO in there and then promptly forgot about it :) Thanks for the suggestion, I'll update this in the Tramp CVS. Ted ^ permalink raw reply [flat|nested] 58+ messages in thread
end of thread, other threads:[~2009-09-28 12:03 UTC | newest] Thread overview: 58+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-12-26 20:02 file storage in IMAP (eventually for Tramp) working and needs testing Ted Zlatanov 2007-12-29 22:34 ` David 2007-12-30 20:35 ` Ted Zlatanov 2007-12-31 10:14 ` Michael Albinus 2007-12-31 12:30 ` Ted Zlatanov 2007-12-31 13:35 ` Michael Albinus 2007-12-31 14:52 ` Ted Zlatanov 2007-12-31 17:14 ` Michael Albinus 2007-12-31 17:45 ` Ted Zlatanov 2007-12-31 21:32 ` Michael Albinus 2008-01-02 20:43 ` Ted Zlatanov 2008-01-02 21:40 ` Reiner Steib 2008-01-02 22:15 ` Ted Zlatanov 2008-01-02 23:50 ` Michael Albinus 2008-01-03 11:32 ` Reiner Steib 2008-01-03 12:43 ` Ted Zlatanov 2008-01-03 12:56 ` Michael Albinus 2008-01-03 13:21 ` Ted Zlatanov 2008-01-03 13:27 ` Ted Zlatanov 2008-01-03 17:31 ` James Cloos 2008-01-03 17:42 ` Ted Zlatanov 2008-01-04 11:00 ` Michael Albinus 2007-12-31 17:09 ` David 2009-04-12 13:04 ` Magnus Henoch 2009-04-13 15:50 ` Ted Zlatanov 2009-04-14 8:44 ` Michael Albinus 2009-04-14 16:22 ` Ted Zlatanov 2009-04-15 21:03 ` Michael Albinus 2009-04-23 0:15 ` Ted Zlatanov 2009-04-23 2:45 ` Daiki Ueno 2009-04-23 12:51 ` Ted Zlatanov 2009-04-23 13:09 ` Daiki Ueno 2009-04-23 13:44 ` Ted Zlatanov 2009-04-23 13:48 ` Daiki Ueno 2009-04-23 14:19 ` Ted Zlatanov 2009-04-23 14:28 ` Daiki Ueno 2009-04-23 15:16 ` Ted Zlatanov 2009-04-24 0:37 ` Daiki Ueno 2009-04-23 21:30 ` Ted Zlatanov 2009-04-25 17:25 ` Michael Albinus 2009-07-10 21:34 ` Ted Zlatanov 2009-07-15 21:25 ` Tramp-IMAP works " Ted Zlatanov 2009-07-16 6:42 ` Michael Albinus 2009-07-18 11:11 ` Michael Albinus 2009-09-07 4:13 ` Ted Zlatanov 2009-09-08 13:33 ` Michael Albinus 2009-09-09 16:13 ` Ted Zlatanov 2009-09-10 7:18 ` Michael Albinus 2009-09-21 19:28 ` imap-hash.el and tramp-imap.el Ted Zlatanov 2009-09-22 14:52 ` Michael Albinus 2009-09-23 21:07 ` Ted Zlatanov 2009-09-24 10:38 ` Michael Albinus 2009-09-25 20:48 ` Ted Zlatanov 2009-09-27 15:57 ` Ted Zlatanov 2009-09-27 19:24 ` Michael Albinus 2009-09-28 12:03 ` Michael Albinus 2009-09-24 6:44 ` Reiner Steib 2009-09-25 20:49 ` Ted Zlatanov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).