POP to IMAP

POP to IMAP

[Xavier Maillard]

[-- Attachment #1: Type: text/plain, Size: 688 bytes --]

Hi,

 As the subject line say, I am about to migrate all my data from POP3 to
IMAP mail protocol. 

 Does anyone know if there are difficulties to achieve such a task ?

 I have succesfully converted one account to IMAP but starting from
scratch at Gnus level so there were no problems.

  Here I am talking of something a little bigger : tons of groups, tons
of topics, several high-traffic ML, fancy split rules, et caetera... In
fact I am talking of my main account so would be nice if my migration
was not a disaster ;)

Regards,

zeDek
-- 
http://www.gnusfr.org -- French Gnus user site

Anti-war disclaimer:
	"Bombing for peace is like fucking for virginity"

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: POP to IMAP

[Lute Kamstra]

Xavier Maillard <zedek@gnu-rox.org> writes:

> As the subject line say, I am about to migrate all my data from POP3
> to IMAP mail protocol.
>
> Does anyone know if there are difficulties to achieve such a task ?

I just migrated from POP to IMAP and I encountered three problems.

Firstly, if you use nnmail-split-fancy-with-parent and still have your
old POP (nnfolder) mail groups lying around, Gnus recognizes if new
mails refer to a mail you have in you old archive, but tries to split
them into a mailbox on the IMAP server with the same name as the group
the parent is in on the old backend.  This is temporary, but can be a
bit confusing at first.

Secondly, I had a few problems with moving my nnfolder mail archive to
the IMAP server; approximately one in 500 messages could not be
copied.  I got several error messages.  A few were caused by incorrect
headers (which you can just delete by editing the message), but most
errors were due to non ASCII characters in the message bodies.
Especially, the \201 character seemed to cause trouble.  What usually
helped was first moving the message to the same nnfolder group is was
already in and then trying to copy it to the IMAP server.  If that
didn't work, editing a message (e) and saving it (C-c C-c) worked for
me as it removed the troublesome characters.

Thirdly, it took quite some time to move all my archived messages to
the IMAP server.  I had to go into every group (C-u RET), select all
messages (M P b), and move (B m) them an IMAP group.  With the number
of mail groups I accumulated over the years, this took quit some time.
I would be nice if there were an easier way to do this.  Maybe there
is, but I just don't know about it.

Lute.

-- 
Lute Kamstra  <Lute.Kamstra@cwi.nl>
CWI  department PNA4
Room M233  phone (+31) 20 592 4214
[Echelon material: Compsec White Water Ceridian]

Re: POP to IMAP

[Reiner Steib]

On Mon, Jun 30 2003, Lute Kamstra wrote:

> Firstly, if you use nnmail-split-fancy-with-parent and still have your
> old POP (nnfolder) mail groups lying around, Gnus recognizes if new
> mails refer to a mail you have in you old archive, but tries to split
> them into a mailbox on the IMAP server with the same name as the group
> the parent is in on the old backend.

Adding the old groups to
`nnmail-split-fancy-with-parent-ignore-groups' should avoid this
problem.

> Thirdly, it took quite some time to move all my archived messages to
> the IMAP server.  I had to go into every group (C-u RET), select all
> messages (M P b), and move (B m) them an IMAP group.[...]  I would
> be nice if there were an easier way to do this.  Maybe there is, but
> I just don't know about it.

You may loop over all relevant groups and let a function do the task.
Something like this:

--8<---------------cut here---------------start------------->8---
(defvar rs-gnus-migrate-old-group-regexp "^nnfolder.*Test"
  "Regexp matching old groups.")
(defvar rs-gnus-migrate-target-group-prefix "nnimap+foo:"
  "Prefix for new groups.")
(defun rs-gnus-migrate-loop ()
  (interactive)
  (let (gnus-large-newsgroup
	gnus-parameters
	case-fold-search
	g-list full method g-name target)
    (dolist (group gnus-newsrc-alist) ;; loop over all groups
      (setq full (car group)
	    g-name (gnus-replace-in-string full "^.*:" "")
	    method (gnus-replace-in-string full (regexp-quote g-name) "")
	    target (concat rs-gnus-migrate-target-group-prefix g-name))
      (when (string-match rs-gnus-migrate-old-group-regexp full)
	(when (string= method "")
	  (setq method nil))
	(gnus-message 8
		      "full=`%s', method=`%s', group=`%s', target=`%s'"
		      full method g-name target)
	;; Create target group next to the old one:
	(gnus-group-jump-to-group full) ;; j
	;; create target group unless it already exists:
	(save-excursion
	  (if (gnus-gethash target gnus-newsrc-hashtb)
	      (gnus-group-jump-to-group target)
	    (gnus-group-make-group
	     g-name
	     ;; FIXME: Use Gnus functions instead of regexps.  Which one?
	     (gnus-replace-in-string rs-gnus-migrate-target-group-prefix
				     "^\\([^+:]*\\).*$" "\\1")
	     (gnus-replace-in-string rs-gnus-migrate-target-group-prefix
				     "^[^+]+\\+\\([^:]+\\).*$" "\\1"))))
	(gnus-group-quick-select-group t) ;; 0 M-RET
	(gnus-summary-toggle-threads t)
	(gnus-summary-sort-by-date) ;; optional
	(gnus-uu-mark-buffer) ;; M P b
	;; Maybe use `-move-' here:
	(gnus-summary-copy-article nil target)
	(gnus-summary-exit)
	;; Maybe unsubscribe/delete old group here:
	;; ...
	(gnus-message 7 "Processed `%s'" g-name)))))
--8<---------------cut here---------------end--------------->8---

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: POP to IMAP

[Xavier Maillard]

[-- Attachment #1: Type: text/plain, Size: 2436 bytes --]

On 30 jun 2003, Lute Kamstra spake thusly:

>  Xavier Maillard <zedek@gnu-rox.org> writes:
>  
> >  As the subject line say, I am about to migrate all my data from
> >  POP3 to IMAP mail protocol.
> >  
> >  Does anyone know if there are difficulties to achieve such a task ?
>  
>  I just migrated from POP to IMAP and I encountered three problems.

Oh it seems I should have read people ansxers before :(
In fact I have encountered problems here too.
  
>  Firstly, if you use nnmail-split-fancy-with-parent and still have
>  your old POP (nnfolder) mail groups lying around, Gnus recognizes if
>  new mails refer to a mail you have in you old archive, but tries to
>  split them into a mailbox on the IMAP server with the same name as
>  the group the parent is in on the old backend.  This is temporary,
>  but can be a bit confusing at first.

It is not my case ;)
  
>  Secondly, I had a few problems with moving my nnfolder mail archive
>  to the IMAP server; approximately one in 500 messages could not be
>  copied.  I got several error messages.  A few were caused by
>  incorrect headers (which you can just delete by editing the message),
>  but most errors were due to non ASCII characters in the message
>  bodies.  Especially, the \201 character seemed to cause trouble.
>  What usually helped was first moving the message to the same nnfolder
>  group is was already in and then trying to copy it to the IMAP
>  server.  If that didn't work, editing a message (e) and saving it
>  (C-c C-c) worked for me as it removed the troublesome characters.

I have encountered the same problem here with a few messages too. The
trick used there was to edit each of 'em manually and remove/correct
the errorneous part.
  
>  Thirdly, it took quite some time to move all my archived messages to
>  the IMAP server.  I had to go into every group (C-u RET), select all
>  messages (M P b), and move (B m) them an IMAP group.  With the number
>  of mail groups I accumulated over the years, this took quit some
>  time.  I would be nice if there were an easier way to do this.  Maybe
>  there is, but I just don't know about it.

Thanx for the hint ;) I have a big group (3500 msg) here to move into the IMAP
server so may be of interest for me :)  

zeDek
-- 
http://www.gnusfr.org -- French Gnus user site

Anti-war disclaimer:
	"Bombing for peace is like fucking for virginity"

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: POP to IMAP

[Lute Kamstra]

Reiner Steib <4.uce.03.r.s@nurfuerspam.de> writes:

[...]

>> Thirdly, it took quite some time to move all my archived messages to
>> the IMAP server.  I had to go into every group (C-u RET), select all
>> messages (M P b), and move (B m) them an IMAP group.[...]  I would
>> be nice if there were an easier way to do this.  Maybe there is, but
>> I just don't know about it.
>
> You may loop over all relevant groups and let a function do the task.
> Something like this:
>

[... group copying code ...]

Thanks.

What would be really nice, is to have a command (able to deal with
marked groups in the *Group* buffer) that can copy/move a group from
one virtual server to another (provided the target backend allows
this).

This would make migrating from POP to IMAP as easy as first process
marking all groups on the POP virtual server and moving them to the
IMAP virtual server, and then doing some cleanup on some groups on the
POP virtual server that were not deleted due to messages that failed
to copy.

The group copy/move command would be useful for other things as well.
For example, you could easily change from nnml to nnfolder if your
mail archive starts to consume an alarming number of inodes.

Lute.


PS  Hmm, I just see that there is a command
    ,----
    | gnus-group-move-group-to-server is an interactive compiled Lisp function in `gnus-move'.
    | (gnus-group-move-group-to-server INFO FROM-SERVER TO-SERVER)
    | 
    | Move the group on the current line from FROM-SERVER to TO-SERVER.
    | 
    | [back]
    `----
    but this seems to relate to synchronising basically the same newsgroup
    on two different newsservers.