[9front] upas/fs: copy messages between IMAP folders

[9front] upas/fs: copy messages between IMAP folders

[risto.salminen]

Dear 9front community,

it is currently not possible to copy messages between
IMAP folders in any way using upas, if I understand
correctly.  upas/fs does not support putting messages
to the mailbox directories in /mail/fs/.  The need for
this arises when someone wants to archive messages
from inbox to another IMAP folder.

The following patch attempts to implement the support
for copying messages between IMAP folders in a similar
way in which flagging and deleting messages are handled.
The same functionality could be extended to other mailbox
types too, although it seems that currently this is only
useful with IMAP.  I have been using this for half a year
now without any problems.

The alternative way to achieve the same result would
be to make upas/fs support writing files to the
/mail/fs/ folders.  Then perhaps upas/mbappend could
be used to write messages to these folders in the same
way as it can currently be used to write messages to
the /mail/box/ folders.  However, the message numbering
schemes are different between these two folder
hierarchies, so there would have to be some special
casing for that, which would complicate things.
Implementing the feature that way would seem to be
more challenging in other ways too, which is why I am
presenting a simpler approach.

Would this be useful for the broader audience?

Thanks,
rsal

diff -r 7c895ae504fa sys/man/4/upasfs
--- a/sys/man/4/upasfs	Fri May 28 13:02:58 2021 +0200
+++ b/sys/man/4/upasfs	Fri May 28 16:40:56 2021 +0300
@@ -206,7 +206,7 @@
 .I fs
 to open, close, rename, create or remove new mailboxes,
 and also to
-delete or flag groups of messages atomically.
+delete, flag, or copy groups of messages atomically.
 The messages that can be written to this file are:
 .TP 2i
 .PD 0
@@ -260,6 +260,13 @@
 .TP
 .B "flag \fImboxname flags number ...\fP
 flag the given messages.
+.TP
+.B "copy \fImboxname number ... target\fP
+Copy the given messages from
+.IR mboxname
+to mailbox named
+.IR target.
+At the moment only supported with IMAP mailboxes.
 .PD
 .PP
 The
diff -r 7c895ae504fa sys/src/cmd/upas/fs/dat.h
--- a/sys/src/cmd/upas/fs/dat.h	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/dat.h	Fri May 28 16:40:56 2021 +0300
@@ -174,6 +174,7 @@
 	void	(*decache)(Mailbox*, Message*);
 	int	(*fetch)(Mailbox*, Message*, uvlong, ulong);
 	void	(*delete)(Mailbox*, Message*);
+	char	*(*copy)(Mailbox*, Message*, char*);
 	char	*(*ctl)(Mailbox*, int, char**);
 	char	*(*remove)(Mailbox *, int);
 	char	*(*rename)(Mailbox*, char*, int);
@@ -215,6 +216,7 @@
 void		unnewmessage(Mailbox*, Message*, Message*);
 char*		delmessages(int, char**);
 char		*flagmessages(int, char**);
+char*		copymessages(int, char**);
 void		digestmessage(Mailbox*, Message*);

 int		wraptls(int, char*);
diff -r 7c895ae504fa sys/src/cmd/upas/fs/fs.c
--- a/sys/src/cmd/upas/fs/fs.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/fs.c	Fri May 28 16:40:56 2021 +0300
@@ -1242,6 +1242,11 @@
 				return nil;
 			return flagmessages(argc - 1, argv + 1);
 		}
+		if(strcmp(argv[0], "copy") == 0){
+			if(argc < 4)
+				return nil;
+			return copymessages(argc - 1, argv + 1);
+		}
 		if(strcmp(argv[0], "remove") == 0){
 			v0 = argv0;
 			flags = 0;
diff -r 7c895ae504fa sys/src/cmd/upas/fs/imap.c
--- a/sys/src/cmd/upas/fs/imap.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/imap.c	Fri May 28 16:40:56 2021 +0300
@@ -1025,6 +1025,22 @@
 }

 static char*
+imap4copy(Mailbox *mb, Message *m, char *dest)
+{
+	char *r;
+	Imap *imap;
+
+	imap = mb->aux;
+	if((ulong)(m->imapuid>>32) == imap->validity){
+		imap4cmd(imap, "uid copy %lud %s", (ulong)m->imapuid, dest);
+		r = imap4resp(imap);
+		if(!isokay(r))
+			return r;
+	}
+	return 0;
+}
+
+static char*
 imap4sync(Mailbox *mb)
 {
 	char *err;
@@ -1183,6 +1199,7 @@
 	mb->delete = imap4delete;
 	mb->rename = imap4rename;
 	mb->modflags = imap4modflags;
+	mb->copy = imap4copy;
 	mb->addfrom = 1;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/mbox.c
--- a/sys/src/cmd/upas/fs/mbox.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/mbox.c	Fri May 28 16:40:56 2021 +0300
@@ -1137,6 +1137,31 @@
 	return rerr;
 }

+char*
+copymessages(int argc, char **argv)
+{
+	char *err, *dest, *rerr;
+	int i;
+	Mailbox *mb;
+	Message *m;
+
+	rerr = 0;
+	for(mb = mbl; mb != nil; mb = mb->next)
+		if(strcmp(*argv, mb->name) == 0)
+			break;
+	if(mb == nil)
+		return "no such mailbox";
+	if(mb->copy == nil)
+		return "copy not supported";
+	dest = argv[argc - 1];
+	for(i = 1; i < argc - 1; i++)
+		for(m = mb->root->part; m; m = m->next)
+			if(strcmp(m->name, argv[i]) == 0)
+				if(err = mb->copy(mb, m, dest))
+					rerr = err;
+	return rerr;
+}
+
 void
 msgincref(Mailbox *mb, Message *m)
 {
diff -r 7c895ae504fa sys/src/cmd/upas/fs/mdir.c
--- a/sys/src/cmd/upas/fs/mdir.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/mdir.c	Fri May 28 16:40:56 2021 +0300
@@ -294,5 +294,6 @@
 	mb->idxread = idxr;
 	mb->idxwrite = idxw;
 	mb->ctl = mdirctl;
+	mb->copy = nil;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/plan9.c
--- a/sys/src/cmd/upas/fs/plan9.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/plan9.c	Fri May 28 16:40:56 2021 +0300
@@ -411,5 +411,6 @@
 	mb->remove = localremove;
 	mb->rename = localrename;
 	mb->decache = plan9decache;
+	mb->copy = nil;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/pop3.c
--- a/sys/src/cmd/upas/fs/pop3.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/pop3.c	Fri May 28 16:40:56 2021 +0300
@@ -624,6 +624,7 @@
 	mb->sync = pop3sync;
 	mb->close = pop3close;
 	mb->ctl = pop3ctl;
+	mb->copy = nil;
 	mb->addfrom = 1;
 	return nil;
 }

Re: [9front] upas/fs: copy messages between IMAP folders

[Stanley Lieber]

On May 28, 2021 11:10:37 AM EDT, risto.salminen@gmx.com wrote:
>Dear 9front community,
>
>it is currently not possible to copy messages between
>IMAP folders in any way using upas, if I understand
>correctly.  upas/fs does not support putting messages
>to the mailbox directories in /mail/fs/.  The need for
>this arises when someone wants to archive messages
>from inbox to another IMAP folder.
>
>The following patch attempts to implement the support
>for copying messages between IMAP folders in a similar
>way in which flagging and deleting messages are handled.
>The same functionality could be extended to other mailbox
>types too, although it seems that currently this is only
>useful with IMAP.  I have been using this for half a year
>now without any problems.
>
>The alternative way to achieve the same result would
>be to make upas/fs support writing files to the
>/mail/fs/ folders.  Then perhaps upas/mbappend could
>be used to write messages to these folders in the same
>way as it can currently be used to write messages to
>the /mail/box/ folders.  However, the message numbering
>schemes are different between these two folder
>hierarchies, so there would have to be some special
>casing for that, which would complicate things.
>Implementing the feature that way would seem to be
>more challenging in other ways too, which is why I am
>presenting a simpler approach.
>
>Would this be useful for the broader audience?
>
>Thanks,
>rsal
>
>diff -r 7c895ae504fa sys/man/4/upasfs
>--- a/sys/man/4/upasfs	Fri May 28 13:02:58 2021 +0200
>+++ b/sys/man/4/upasfs	Fri May 28 16:40:56 2021 +0300
>@@ -206,7 +206,7 @@
> .I fs
> to open, close, rename, create or remove new mailboxes,
> and also to
>-delete or flag groups of messages atomically.
>+delete, flag, or copy groups of messages atomically.
> The messages that can be written to this file are:
> .TP 2i
> .PD 0
>@@ -260,6 +260,13 @@
> .TP
> .B "flag \fImboxname flags number ...\fP
> flag the given messages.
>+.TP
>+.B "copy \fImboxname number ... target\fP
>+Copy the given messages from
>+.IR mboxname
>+to mailbox named
>+.IR target.
>+At the moment only supported with IMAP mailboxes.
> .PD
> .PP
> The
>diff -r 7c895ae504fa sys/src/cmd/upas/fs/dat.h
>--- a/sys/src/cmd/upas/fs/dat.h	Fri May 28 13:02:58 2021 +0200
>+++ b/sys/src/cmd/upas/fs/dat.h	Fri May 28 16:40:56 2021 +0300
>@@ -174,6 +174,7 @@
> 	void	(*decache)(Mailbox*, Message*);
> 	int	(*fetch)(Mailbox*, Message*, uvlong, ulong);
> 	void	(*delete)(Mailbox*, Message*);
>+	char	*(*copy)(Mailbox*, Message*, char*);
> 	char	*(*ctl)(Mailbox*, int, char**);
> 	char	*(*remove)(Mailbox *, int);
> 	char	*(*rename)(Mailbox*, char*, int);
>@@ -215,6 +216,7 @@
> void		unnewmessage(Mailbox*, Message*, Message*);
> char*		delmessages(int, char**);
> char		*flagmessages(int, char**);
>+char*		copymessages(int, char**);
> void		digestmessage(Mailbox*, Message*);
>
> int		wraptls(int, char*);
>diff -r 7c895ae504fa sys/src/cmd/upas/fs/fs.c
>--- a/sys/src/cmd/upas/fs/fs.c	Fri May 28 13:02:58 2021 +0200
>+++ b/sys/src/cmd/upas/fs/fs.c	Fri May 28 16:40:56 2021 +0300
>@@ -1242,6 +1242,11 @@
> 				return nil;
> 			return flagmessages(argc - 1, argv + 1);
> 		}
>+		if(strcmp(argv[0], "copy") == 0){
>+			if(argc < 4)
>+				return nil;
>+			return copymessages(argc - 1, argv + 1);
>+		}
> 		if(strcmp(argv[0], "remove") == 0){
> 			v0 = argv0;
> 			flags = 0;
>diff -r 7c895ae504fa sys/src/cmd/upas/fs/imap.c
>--- a/sys/src/cmd/upas/fs/imap.c	Fri May 28 13:02:58 2021 +0200
>+++ b/sys/src/cmd/upas/fs/imap.c	Fri May 28 16:40:56 2021 +0300
>@@ -1025,6 +1025,22 @@
> }
>
> static char*
>+imap4copy(Mailbox *mb, Message *m, char *dest)
>+{
>+	char *r;
>+	Imap *imap;
>+
>+	imap = mb->aux;
>+	if((ulong)(m->imapuid>>32) == imap->validity){
>+		imap4cmd(imap, "uid copy %lud %s", (ulong)m->imapuid, dest);
>+		r = imap4resp(imap);
>+		if(!isokay(r))
>+			return r;
>+	}
>+	return 0;
>+}
>+
>+static char*
> imap4sync(Mailbox *mb)
> {
> 	char *err;
>@@ -1183,6 +1199,7 @@
> 	mb->delete = imap4delete;
> 	mb->rename = imap4rename;
> 	mb->modflags = imap4modflags;
>+	mb->copy = imap4copy;
> 	mb->addfrom = 1;
> 	return nil;
> }
>diff -r 7c895ae504fa sys/src/cmd/upas/fs/mbox.c
>--- a/sys/src/cmd/upas/fs/mbox.c	Fri May 28 13:02:58 2021 +0200
>+++ b/sys/src/cmd/upas/fs/mbox.c	Fri May 28 16:40:56 2021 +0300
>@@ -1137,6 +1137,31 @@
> 	return rerr;
> }
>
>+char*
>+copymessages(int argc, char **argv)
>+{
>+	char *err, *dest, *rerr;
>+	int i;
>+	Mailbox *mb;
>+	Message *m;
>+
>+	rerr = 0;
>+	for(mb = mbl; mb != nil; mb = mb->next)
>+		if(strcmp(*argv, mb->name) == 0)
>+			break;
>+	if(mb == nil)
>+		return "no such mailbox";
>+	if(mb->copy == nil)
>+		return "copy not supported";
>+	dest = argv[argc - 1];
>+	for(i = 1; i < argc - 1; i++)
>+		for(m = mb->root->part; m; m = m->next)
>+			if(strcmp(m->name, argv[i]) == 0)
>+				if(err = mb->copy(mb, m, dest))
>+					rerr = err;
>+	return rerr;
>+}
>+
> void
> msgincref(Mailbox *mb, Message *m)
> {
>diff -r 7c895ae504fa sys/src/cmd/upas/fs/mdir.c
>--- a/sys/src/cmd/upas/fs/mdir.c	Fri May 28 13:02:58 2021 +0200
>+++ b/sys/src/cmd/upas/fs/mdir.c	Fri May 28 16:40:56 2021 +0300
>@@ -294,5 +294,6 @@
> 	mb->idxread = idxr;
> 	mb->idxwrite = idxw;
> 	mb->ctl = mdirctl;
>+	mb->copy = nil;
> 	return nil;
> }
>diff -r 7c895ae504fa sys/src/cmd/upas/fs/plan9.c
>--- a/sys/src/cmd/upas/fs/plan9.c	Fri May 28 13:02:58 2021 +0200
>+++ b/sys/src/cmd/upas/fs/plan9.c	Fri May 28 16:40:56 2021 +0300
>@@ -411,5 +411,6 @@
> 	mb->remove = localremove;
> 	mb->rename = localrename;
> 	mb->decache = plan9decache;
>+	mb->copy = nil;
> 	return nil;
> }
>diff -r 7c895ae504fa sys/src/cmd/upas/fs/pop3.c
>--- a/sys/src/cmd/upas/fs/pop3.c	Fri May 28 13:02:58 2021 +0200
>+++ b/sys/src/cmd/upas/fs/pop3.c	Fri May 28 16:40:56 2021 +0300
>@@ -624,6 +624,7 @@
> 	mb->sync = pop3sync;
> 	mb->close = pop3close;
> 	mb->ctl = pop3ctl;
>+	mb->copy = nil;
> 	mb->addfrom = 1;
> 	return nil;
> }
>

not sure I understand the problem clearly, but I copy messages from mbox to other folders via imap every day.

sl

Re: [9front] upas/fs: copy messages between IMAP folders

[risto.salminen]

Hello,

sl wrote:
> not sure I understand the problem clearly, but I copy messages from mbox to other folders via imap every day.

If it is possible to copy messages in some way already,
then there is indeed something that I do not understand.

Normally I open my mailbox as

  ; echo open /imaps/$mailserver/$mailuser mbox >/mail/fs/ctl

where $mailserver and $mailuser contain the actual server
and user values, so that then /mail/fs/mbox contains the
IMAP folder that I can open with just running 'mail'.
Then I open my archive folder as

  ; echo open /imaps/$mailserver/$mailuser/Archive Archive >/mail/fs/ctl

so that /mail/fs/Archive contains the archive, which can
be opened by running 'mail' and then 'mb Archive'.
If I now do

  ; upas/mbappend Archive /mail/fs/mbox/$id/raw

I get the message in /mail/box/$user/Archive, which is
just a local folder.  If I do

  ; upas/mbappend /imaps/$mailserver/$mailuser/Archive /mail/fs/mbox/$id/raw

then I get back 'mbappend somepid: fail', because obviously
this is not the correct way to do it.  If I do

  ; upas/mbappend /mail/fs/Archive /mail/fs/mbox/$id/raw

which would perhaps seem sensible, I also get back
'mbappend somepid: fail', this time because upas/fs
indeed disallows opening files inside /mail/fs/ folders
for writing.

Therefore, I have used the configuration in my patches
for some time, because I could not figure out any other
way to do this.

Perhaps someone would like to enlighten me how this is
supposed to work.

Thanks,
rsal

Re: [9front] upas/fs: copy messages between IMAP folders

[Stanley Lieber]

On May 28, 2021 12:26:51 PM EDT, risto.salminen@gmx.com wrote:
>Hello,
>
>sl wrote:
>> not sure I understand the problem clearly, but I copy messages from mbox to other folders via imap every day.
>
>If it is possible to copy messages in some way already,
>then there is indeed something that I do not understand.
>
>Normally I open my mailbox as
>
>  ; echo open /imaps/$mailserver/$mailuser mbox >/mail/fs/ctl
>
>where $mailserver and $mailuser contain the actual server
>and user values, so that then /mail/fs/mbox contains the
>IMAP folder that I can open with just running 'mail'.
>Then I open my archive folder as
>
>  ; echo open /imaps/$mailserver/$mailuser/Archive Archive >/mail/fs/ctl
>
>so that /mail/fs/Archive contains the archive, which can
>be opened by running 'mail' and then 'mb Archive'.
>If I now do
>
>  ; upas/mbappend Archive /mail/fs/mbox/$id/raw
>
>I get the message in /mail/box/$user/Archive, which is
>just a local folder.  If I do
>
>  ; upas/mbappend /imaps/$mailserver/$mailuser/Archive /mail/fs/mbox/$id/raw
>
>then I get back 'mbappend somepid: fail', because obviously
>this is not the correct way to do it.  If I do
>
>  ; upas/mbappend /mail/fs/Archive /mail/fs/mbox/$id/raw
>
>which would perhaps seem sensible, I also get back
>'mbappend somepid: fail', this time because upas/fs
>indeed disallows opening files inside /mail/fs/ folders
>for writing.
>
>Therefore, I have used the configuration in my patches
>for some time, because I could not figure out any other
>way to do this.
>
>Perhaps someone would like to enlighten me how this is
>supposed to work.
>
>Thanks,
>rsal
>

now I get it. I observe the same behavior when running commands on a local 9front against a remote 9front imap server.

however, using k9 mail client on android against a remote 9front imap server, I can copy or move messages between folders on the remote server with no problem.

sl

Re: [9front] upas/fs: copy messages between IMAP folders

[risto.salminen]

Hello,

sl wrote:
> now I get it. I observe the same behavior when
> running commands on a local 9front against a
> remote 9front imap server.

That is indeed how it currently works, with any
remote IMAP server, not just 9front one.  The
patch attempts to improve the situation in the
IMAP client end.

sl wrote:
> however, using k9 mail client on android
> against a remote 9front imap server, I can
> copy or move messages between folders on the
> remote server with no problem.

This is actually irrelevant to my patch, but
good to know that this works.  It is just the
IMAP client, namely upas/fs, which, as far as
I understand, without my patch does not support
telling the remote IMAP server to copy messages
between IMAP folders, or have any other means
to put messages to the server.  As a detail,
IMAP, as a protocol, does not support moving
messages between folders, so moving has to be
implemented by first copying the message to the
target folder and then deleting the original
message from the source folder.  I chose to
offload this logic to the mail client, as
presented in the upas/nedmail patch, which
could also be adapted to acme's Mail.

Thanks,
rsal

Re: [9front] upas/fs: copy messages between IMAP folders

[Lyndon Nerenberg (VE7TFX/VE6BBM)]

> As a detail,
> IMAP, as a protocol, does not support moving
> messages between folders, so moving has to be
> implemented by first copying the message to the
> target folder and then deleting the original
> message from the source folder.  I chose to
> offload this logic to the mail client, as
> presented in the upas/nedmail patch, which
> could also be adapted to acme's Mail.

The IMAP "move" model has always been 'copy + store +flags \deleted'.
This performs the copy operation internally to the IMAP server, so
no data need go over the wire.  I haven't looked at your patch in
detail, but if you have devolved this to the case where a "copy"
between folders on the same server always involves a round trip
through the MUA, this is a horrible pessimization and the patch
should NOT be incorporated as is.

Doing IMAP "right" isn't always easy, and this is a case where you
have to do the extra work in order to do the correct optimization.
You need to reliably figure out if the source and destination paths
in the "move" correspond to the same server+user.  If they do, use
the IMAP COPY command, otherwise fall back to the MUA loop.

--lyndon

Re: [9front] upas/fs: copy messages between IMAP folders

[Risto Salminen]

Hello,

lyndon wrote:
> The IMAP "move" model has always been 'copy + store +flags \deleted'.
> This performs the copy operation internally to the IMAP server, so
> no data need go over the wire.

This is exactly what the patch does.  That
however happens in two parts, explained
below.

lyndon wrote:
> I haven't looked at your patch in
> detail, but if you have devolved this to the case where a "copy"
> between folders on the same server always involves a round trip
> through the MUA, this is a horrible pessimization and the patch
> should NOT be incorporated as is.

And therefore, this is not what the patch does.
With the patch, upas/fs issues a "copy" command
to the IMAP server, to copy the message from
a folder to another folder inside the IMAP
server, for the exact reasons that you
presented.  In my presented design, issuing
the "store +flags \deleted" portion is left to
the responsibility of the MUA, for example
nedmail.

However, I will send a version of the patch,
that is in my opinion more elegant, soon
separately.  That version combines the copy and
delete commands into one move command in the
upas/fs level, which seems better than having
the client deal with deletion separately, as it
simplifies the MUA's duties.

lyndon wrote:
> Doing IMAP "right" isn't always easy, and this is a case where you
> have to do the extra work in order to do the correct optimization.
> You need to reliably figure out if the source and destination paths
> in the "move" correspond to the same server+user.  If they do, use
> the IMAP COPY command, otherwise fall back to the MUA loop.

This reveals a shortcoming of my patch.  It indeed
only enables one to move mails inside an IMAP server,
and not between IMAP servers.  It also does not allow
one to put new messages to an IMAP server.  That is
for simplicity and performance reasons.  However, at
least I only need to move mails inside an IMAP server,
which is why the patch only does that.

Thanks,
rsal

Re: [9front] upas/fs: copy messages between IMAP folders

[Lyndon Nerenberg (VE7TFX/VE6BBM)]

Risto, thanks for the additional explanation.  I guess I didn't
quite grasp what you were doing when I read the original message.

--lyndon

Re: [9front] upas/fs: copy messages between IMAP folders

[kjn]

Quoth risto.salminen@gmx.com:
> Dear 9front community,
> 
> it is currently not possible to copy messages between
> IMAP folders in any way using upas, if I understand
> correctly.  upas/fs does not support putting messages
> to the mailbox directories in /mail/fs/.  The need for
> this arises when someone wants to archive messages
> from inbox to another IMAP folder.
> 
> The following patch attempts to implement the support
> for copying messages between IMAP folders in a similar
> way in which flagging and deleting messages are handled.
> The same functionality could be extended to other mailbox
> types too, although it seems that currently this is only
> useful with IMAP.  I have been using this for half a year
> now without any problems.
> 
> The alternative way to achieve the same result would
> be to make upas/fs support writing files to the
> /mail/fs/ folders.  Then perhaps upas/mbappend could
> be used to write messages to these folders in the same
> way as it can currently be used to write messages to
> the /mail/box/ folders.  However, the message numbering
> schemes are different between these two folder
> hierarchies, so there would have to be some special
> casing for that, which would complicate things.
> Implementing the feature that way would seem to be
> more challenging in other ways too, which is why I am
> presenting a simpler approach.
> 
> Would this be useful for the broader audience?
> 
> Thanks,
> rsal
> 
> diff -r 7c895ae504fa sys/man/4/upasfs
> --- a/sys/man/4/upasfs	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/man/4/upasfs	Fri May 28 16:40:56 2021 +0300
> @@ -206,7 +206,7 @@
>  .I fs
>  to open, close, rename, create or remove new mailboxes,
>  and also to
> -delete or flag groups of messages atomically.
> +delete, flag, or copy groups of messages atomically.
>  The messages that can be written to this file are:
>  .TP 2i
>  .PD 0
> @@ -260,6 +260,13 @@
>  .TP
>  .B "flag \fImboxname flags number ...\fP
>  flag the given messages.
> +.TP
> +.B "copy \fImboxname number ... target\fP
> +Copy the given messages from
> +.IR mboxname
> +to mailbox named
> +.IR target.
> +At the moment only supported with IMAP mailboxes.
>  .PD
>  .PP
>  The
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/dat.h
> --- a/sys/src/cmd/upas/fs/dat.h	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/dat.h	Fri May 28 16:40:56 2021 +0300
> @@ -174,6 +174,7 @@
>  	void	(*decache)(Mailbox*, Message*);
>  	int	(*fetch)(Mailbox*, Message*, uvlong, ulong);
>  	void	(*delete)(Mailbox*, Message*);
> +	char	*(*copy)(Mailbox*, Message*, char*);
>  	char	*(*ctl)(Mailbox*, int, char**);
>  	char	*(*remove)(Mailbox *, int);
>  	char	*(*rename)(Mailbox*, char*, int);
> @@ -215,6 +216,7 @@
>  void		unnewmessage(Mailbox*, Message*, Message*);
>  char*		delmessages(int, char**);
>  char		*flagmessages(int, char**);
> +char*		copymessages(int, char**);
>  void		digestmessage(Mailbox*, Message*);
> 
>  int		wraptls(int, char*);
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/fs.c
> --- a/sys/src/cmd/upas/fs/fs.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/fs.c	Fri May 28 16:40:56 2021 +0300
> @@ -1242,6 +1242,11 @@
>  				return nil;
>  			return flagmessages(argc - 1, argv + 1);
>  		}
> +		if(strcmp(argv[0], "copy") == 0){
> +			if(argc < 4)
> +				return nil;
> +			return copymessages(argc - 1, argv + 1);
> +		}
>  		if(strcmp(argv[0], "remove") == 0){
>  			v0 = argv0;
>  			flags = 0;
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/imap.c
> --- a/sys/src/cmd/upas/fs/imap.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/imap.c	Fri May 28 16:40:56 2021 +0300
> @@ -1025,6 +1025,22 @@
>  }
> 
>  static char*
> +imap4copy(Mailbox *mb, Message *m, char *dest)
> +{
> +	char *r;
> +	Imap *imap;
> +
> +	imap = mb->aux;
> +	if((ulong)(m->imapuid>>32) == imap->validity){
> +		imap4cmd(imap, "uid copy %lud %s", (ulong)m->imapuid, dest);
> +		r = imap4resp(imap);
> +		if(!isokay(r))
> +			return r;
> +	}
> +	return 0;
> +}
> +
> +static char*
>  imap4sync(Mailbox *mb)
>  {
>  	char *err;
> @@ -1183,6 +1199,7 @@
>  	mb->delete = imap4delete;
>  	mb->rename = imap4rename;
>  	mb->modflags = imap4modflags;
> +	mb->copy = imap4copy;
>  	mb->addfrom = 1;
>  	return nil;
>  }
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/mbox.c
> --- a/sys/src/cmd/upas/fs/mbox.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/mbox.c	Fri May 28 16:40:56 2021 +0300
> @@ -1137,6 +1137,31 @@
>  	return rerr;
>  }
> 
> +char*
> +copymessages(int argc, char **argv)
> +{
> +	char *err, *dest, *rerr;
> +	int i;
> +	Mailbox *mb;
> +	Message *m;
> +
> +	rerr = 0;
> +	for(mb = mbl; mb != nil; mb = mb->next)
> +		if(strcmp(*argv, mb->name) == 0)
> +			break;
> +	if(mb == nil)
> +		return "no such mailbox";
> +	if(mb->copy == nil)
> +		return "copy not supported";
> +	dest = argv[argc - 1];
> +	for(i = 1; i < argc - 1; i++)
> +		for(m = mb->root->part; m; m = m->next)
> +			if(strcmp(m->name, argv[i]) == 0)
> +				if(err = mb->copy(mb, m, dest))
> +					rerr = err;
> +	return rerr;
> +}
> +
>  void
>  msgincref(Mailbox *mb, Message *m)
>  {
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/mdir.c
> --- a/sys/src/cmd/upas/fs/mdir.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/mdir.c	Fri May 28 16:40:56 2021 +0300
> @@ -294,5 +294,6 @@
>  	mb->idxread = idxr;
>  	mb->idxwrite = idxw;
>  	mb->ctl = mdirctl;
> +	mb->copy = nil;
>  	return nil;
>  }
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/plan9.c
> --- a/sys/src/cmd/upas/fs/plan9.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/plan9.c	Fri May 28 16:40:56 2021 +0300
> @@ -411,5 +411,6 @@
>  	mb->remove = localremove;
>  	mb->rename = localrename;
>  	mb->decache = plan9decache;
> +	mb->copy = nil;
>  	return nil;
>  }
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/pop3.c
> --- a/sys/src/cmd/upas/fs/pop3.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/pop3.c	Fri May 28 16:40:56 2021 +0300
> @@ -624,6 +624,7 @@
>  	mb->sync = pop3sync;
>  	mb->close = pop3close;
>  	mb->ctl = pop3ctl;
> +	mb->copy = nil;
>  	mb->addfrom = 1;
>  	return nil;
>  }

I have not tried out the patch, but I definitely have a use for
copying messages from one imap folder to another.  That's one
remaining critical feature that I miss with mail on 9front.

-- Kyle

Re: [9front] upas/fs: copy messages between IMAP folders

[Risto Salminen]

Hello,

another iteration of the patch only but simply
allows one to move, and not copy, messages
inside a IMAP server from a folder to another
folder.  This simplifies the MUA's, for example
nedmail's, duties, as it does not have to
separately delete the moved mail anymore.

Would this be useful for the broader audience?

Thanks,
rsal

diff -r 7c895ae504fa sys/man/4/upasfs
--- a/sys/man/4/upasfs	Fri May 28 13:02:58 2021 +0200
+++ b/sys/man/4/upasfs	Mon May 31 13:55:01 2021 +0300
@@ -206,7 +206,7 @@
 .I fs
 to open, close, rename, create or remove new mailboxes,
 and also to
-delete or flag groups of messages atomically.
+delete, flag, or move groups of messages atomically.
 The messages that can be written to this file are:
 .TP 2i
 .PD 0
@@ -260,6 +260,13 @@
 .TP
 .B "flag \fImboxname flags number ...\fP
 flag the given messages.
+.TP
+.B "move \fImboxname number ... target\fP
+Move the given messages from
+.IR mboxname
+to mailbox named
+.IR target.
+At the moment only supported with IMAP mailboxes.
 .PD
 .PP
 The
diff -r 7c895ae504fa sys/src/cmd/upas/fs/dat.h
--- a/sys/src/cmd/upas/fs/dat.h	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/dat.h	Mon May 31 13:55:01 2021 +0300
@@ -174,6 +174,7 @@
 	void	(*decache)(Mailbox*, Message*);
 	int	(*fetch)(Mailbox*, Message*, uvlong, ulong);
 	void	(*delete)(Mailbox*, Message*);
+	char	*(*move)(Mailbox*, Message*, char*);
 	char	*(*ctl)(Mailbox*, int, char**);
 	char	*(*remove)(Mailbox *, int);
 	char	*(*rename)(Mailbox*, char*, int);
@@ -215,6 +216,7 @@
 void		unnewmessage(Mailbox*, Message*, Message*);
 char*		delmessages(int, char**);
 char		*flagmessages(int, char**);
+char*		movemessages(int, char**);
 void		digestmessage(Mailbox*, Message*);

 int		wraptls(int, char*);
diff -r 7c895ae504fa sys/src/cmd/upas/fs/fs.c
--- a/sys/src/cmd/upas/fs/fs.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/fs.c	Mon May 31 13:55:01 2021 +0300
@@ -1242,6 +1242,11 @@
 				return nil;
 			return flagmessages(argc - 1, argv + 1);
 		}
+		if(strcmp(argv[0], "move") == 0){
+			if(argc < 4)
+				return nil;
+			return movemessages(argc - 1, argv + 1);
+		}
 		if(strcmp(argv[0], "remove") == 0){
 			v0 = argv0;
 			flags = 0;
diff -r 7c895ae504fa sys/src/cmd/upas/fs/imap.c
--- a/sys/src/cmd/upas/fs/imap.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/imap.c	Mon May 31 13:55:01 2021 +0300
@@ -1025,6 +1025,29 @@
 }

 static char*
+imap4move(Mailbox *mb, Message *m, char *dest)
+{
+	char *r;
+	Imap *imap;
+
+	imap = mb->aux;
+	imap4cmd(imap, "uid copy %lud %s", (ulong)m->imapuid, dest);
+	r = imap4resp(imap);
+	if(!isokay(r))
+		return r;
+	imap4cmd(imap, "uid store %lud +flags (\\Deleted)", (ulong)m->imapuid);
+	r = imap4resp(imap);
+	if(!isokay(r))
+		return r;
+	imap4cmd(imap, "expunge");
+	r = imap4resp(imap);
+	if(!isokay(r))
+		return r;
+	m->inmbox = 0;
+	return 0;
+}
+
+static char*
 imap4sync(Mailbox *mb)
 {
 	char *err;
@@ -1183,6 +1206,7 @@
 	mb->delete = imap4delete;
 	mb->rename = imap4rename;
 	mb->modflags = imap4modflags;
+	mb->move = imap4move;
 	mb->addfrom = 1;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/mbox.c
--- a/sys/src/cmd/upas/fs/mbox.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/mbox.c	Mon May 31 13:55:01 2021 +0300
@@ -1137,6 +1137,37 @@
 	return rerr;
 }

+char*
+movemessages(int argc, char **argv)
+{
+	char *err, *dest, *rerr;
+	int i, needwrite;
+	Mailbox *mb;
+	Message *m;
+
+	rerr = 0;
+	for(mb = mbl; mb != nil; mb = mb->next)
+		if(strcmp(*argv, mb->name) == 0)
+			break;
+	if(mb == nil)
+		return "no such mailbox";
+	if(mb->move == nil)
+		return "move not supported";
+	dest = argv[argc - 1];
+	needwrite = 0;
+	for(i = 1; i < argc - 1; i++)
+		for(m = mb->root->part; m; m = m->next)
+			if(strcmp(m->name, argv[i]) == 0){
+				if(err = mb->move(mb, m, dest))
+					rerr = err;
+				else
+					needwrite = 1;
+			}
+	if(needwrite)
+		syncmbox(mb, 1);
+	return rerr;
+}
+
 void
 msgincref(Mailbox *mb, Message *m)
 {
diff -r 7c895ae504fa sys/src/cmd/upas/fs/mdir.c
--- a/sys/src/cmd/upas/fs/mdir.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/mdir.c	Mon May 31 13:55:01 2021 +0300
@@ -294,5 +294,6 @@
 	mb->idxread = idxr;
 	mb->idxwrite = idxw;
 	mb->ctl = mdirctl;
+	mb->move = nil;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/plan9.c
--- a/sys/src/cmd/upas/fs/plan9.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/plan9.c	Mon May 31 13:55:01 2021 +0300
@@ -411,5 +411,6 @@
 	mb->remove = localremove;
 	mb->rename = localrename;
 	mb->decache = plan9decache;
+	mb->move = nil;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/pop3.c
--- a/sys/src/cmd/upas/fs/pop3.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/pop3.c	Mon May 31 13:55:01 2021 +0300
@@ -624,6 +624,7 @@
 	mb->sync = pop3sync;
 	mb->close = pop3close;
 	mb->ctl = pop3ctl;
+	mb->move = nil;
 	mb->addfrom = 1;
 	return nil;
 }

Re: [9front] upas/fs: copy messages between IMAP folders

[igor]

Thanks for this rsal. Tested the patch on a few accounts with many
imap folders and it works well for me.

The only nitpick after using this for a while is re: moves to and from
the 'Inbox'.

Assuming the following mailboxes:

  term% upas/fs -f /imaps/mail.9lab.org/igor
  term% echo open /imaps/mail.9lab.org/igor/Hacking Hacking  >/mail/fs/ctl

Moving two message from 'Hacking' to 'Inbox' looks like this:

  term% echo move Hacking 677 674 Inbox >/mail/fs/ctl

Moving messages from 'Inbox' to 'Hacking' looks like this:

  term% echo move mbox 4 3 Hacking >/mail/fs/ctl

Note that the former move requires the IMAP folder name 'Inbox', the
latter move requires the use of 'mbox' (i.e. 'Inbox' does not work).

Sorry no patch for this so far, just found some time to test
at this stage…

Cheers,
Igor

Quoth Risto Salminen <risto.salminen@gmx.com>:
> Hello,
> 
> another iteration of the patch only but simply
> allows one to move, and not copy, messages
> inside a IMAP server from a folder to another
> folder.  This simplifies the MUA's, for example
> nedmail's, duties, as it does not have to
> separately delete the moved mail anymore.
> 
> Would this be useful for the broader audience?
> 
> Thanks,
> rsal
> 
> diff -r 7c895ae504fa sys/man/4/upasfs
> --- a/sys/man/4/upasfs	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/man/4/upasfs	Mon May 31 13:55:01 2021 +0300
> @@ -206,7 +206,7 @@
>  .I fs
>  to open, close, rename, create or remove new mailboxes,
>  and also to
> -delete or flag groups of messages atomically.
> +delete, flag, or move groups of messages atomically.
>  The messages that can be written to this file are:
>  .TP 2i
>  .PD 0
> @@ -260,6 +260,13 @@
>  .TP
>  .B "flag \fImboxname flags number ...\fP
>  flag the given messages.
> +.TP
> +.B "move \fImboxname number ... target\fP
> +Move the given messages from
> +.IR mboxname
> +to mailbox named
> +.IR target.
> +At the moment only supported with IMAP mailboxes.
>  .PD
>  .PP
>  The
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/dat.h
> --- a/sys/src/cmd/upas/fs/dat.h	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/dat.h	Mon May 31 13:55:01 2021 +0300
> @@ -174,6 +174,7 @@
>  	void	(*decache)(Mailbox*, Message*);
>  	int	(*fetch)(Mailbox*, Message*, uvlong, ulong);
>  	void	(*delete)(Mailbox*, Message*);
> +	char	*(*move)(Mailbox*, Message*, char*);
>  	char	*(*ctl)(Mailbox*, int, char**);
>  	char	*(*remove)(Mailbox *, int);
>  	char	*(*rename)(Mailbox*, char*, int);
> @@ -215,6 +216,7 @@
>  void		unnewmessage(Mailbox*, Message*, Message*);
>  char*		delmessages(int, char**);
>  char		*flagmessages(int, char**);
> +char*		movemessages(int, char**);
>  void		digestmessage(Mailbox*, Message*);
> 
>  int		wraptls(int, char*);
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/fs.c
> --- a/sys/src/cmd/upas/fs/fs.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/fs.c	Mon May 31 13:55:01 2021 +0300
> @@ -1242,6 +1242,11 @@
>  				return nil;
>  			return flagmessages(argc - 1, argv + 1);
>  		}
> +		if(strcmp(argv[0], "move") == 0){
> +			if(argc < 4)
> +				return nil;
> +			return movemessages(argc - 1, argv + 1);
> +		}
>  		if(strcmp(argv[0], "remove") == 0){
>  			v0 = argv0;
>  			flags = 0;
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/imap.c
> --- a/sys/src/cmd/upas/fs/imap.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/imap.c	Mon May 31 13:55:01 2021 +0300
> @@ -1025,6 +1025,29 @@
>  }
> 
>  static char*
> +imap4move(Mailbox *mb, Message *m, char *dest)
> +{
> +	char *r;
> +	Imap *imap;
> +
> +	imap = mb->aux;
> +	imap4cmd(imap, "uid copy %lud %s", (ulong)m->imapuid, dest);
> +	r = imap4resp(imap);
> +	if(!isokay(r))
> +		return r;
> +	imap4cmd(imap, "uid store %lud +flags (\\Deleted)", (ulong)m->imapuid);
> +	r = imap4resp(imap);
> +	if(!isokay(r))
> +		return r;
> +	imap4cmd(imap, "expunge");
> +	r = imap4resp(imap);
> +	if(!isokay(r))
> +		return r;
> +	m->inmbox = 0;
> +	return 0;
> +}
> +
> +static char*
>  imap4sync(Mailbox *mb)
>  {
>  	char *err;
> @@ -1183,6 +1206,7 @@
>  	mb->delete = imap4delete;
>  	mb->rename = imap4rename;
>  	mb->modflags = imap4modflags;
> +	mb->move = imap4move;
>  	mb->addfrom = 1;
>  	return nil;
>  }
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/mbox.c
> --- a/sys/src/cmd/upas/fs/mbox.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/mbox.c	Mon May 31 13:55:01 2021 +0300
> @@ -1137,6 +1137,37 @@
>  	return rerr;
>  }
> 
> +char*
> +movemessages(int argc, char **argv)
> +{
> +	char *err, *dest, *rerr;
> +	int i, needwrite;
> +	Mailbox *mb;
> +	Message *m;
> +
> +	rerr = 0;
> +	for(mb = mbl; mb != nil; mb = mb->next)
> +		if(strcmp(*argv, mb->name) == 0)
> +			break;
> +	if(mb == nil)
> +		return "no such mailbox";
> +	if(mb->move == nil)
> +		return "move not supported";
> +	dest = argv[argc - 1];
> +	needwrite = 0;
> +	for(i = 1; i < argc - 1; i++)
> +		for(m = mb->root->part; m; m = m->next)
> +			if(strcmp(m->name, argv[i]) == 0){
> +				if(err = mb->move(mb, m, dest))
> +					rerr = err;
> +				else
> +					needwrite = 1;
> +			}
> +	if(needwrite)
> +		syncmbox(mb, 1);
> +	return rerr;
> +}
> +
>  void
>  msgincref(Mailbox *mb, Message *m)
>  {
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/mdir.c
> --- a/sys/src/cmd/upas/fs/mdir.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/mdir.c	Mon May 31 13:55:01 2021 +0300
> @@ -294,5 +294,6 @@
>  	mb->idxread = idxr;
>  	mb->idxwrite = idxw;
>  	mb->ctl = mdirctl;
> +	mb->move = nil;
>  	return nil;
>  }
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/plan9.c
> --- a/sys/src/cmd/upas/fs/plan9.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/plan9.c	Mon May 31 13:55:01 2021 +0300
> @@ -411,5 +411,6 @@
>  	mb->remove = localremove;
>  	mb->rename = localrename;
>  	mb->decache = plan9decache;
> +	mb->move = nil;
>  	return nil;
>  }
> diff -r 7c895ae504fa sys/src/cmd/upas/fs/pop3.c
> --- a/sys/src/cmd/upas/fs/pop3.c	Fri May 28 13:02:58 2021 +0200
> +++ b/sys/src/cmd/upas/fs/pop3.c	Mon May 31 13:55:01 2021 +0300
> @@ -624,6 +624,7 @@
>  	mb->sync = pop3sync;
>  	mb->close = pop3close;
>  	mb->ctl = pop3ctl;
> +	mb->move = nil;
>  	mb->addfrom = 1;
>  	return nil;
>  }

Re: [9front] upas/fs: copy messages between IMAP folders

[Risto Salminen]

Hello,

> The only nitpick after using this for a while is re: moves to and from
> the 'Inbox'.
>
> Assuming the following mailboxes:
>
>   term% upas/fs -f /imaps/mail.9lab.org/igor
>   term% echo open /imaps/mail.9lab.org/igor/Hacking Hacking  >/mail/fs/ctl
>
> Moving two message from 'Hacking' to 'Inbox' looks like this:
>
>   term% echo move Hacking 677 674 Inbox >/mail/fs/ctl
>
> Moving messages from 'Inbox' to 'Hacking' looks like this:
>
>   term% echo move mbox 4 3 Hacking >/mail/fs/ctl
>
> Note that the former move requires the IMAP folder name 'Inbox', the
> latter move requires the use of 'mbox' (i.e. 'Inbox' does not work).

This is how it is supposed to work.

The source of the move is the local label by which upas
presents the folder in /mail/fs/, which is implicitly
'mbox' if not explicitly specified.  Therefore ones
inbox, perhaps usually 'Inbox' in the IMAP server, is
commonly presented as 'mbox' by upas, and not by its
actual folder name.

The target, on the other hand, is the name of the folder
in the IMAP server, because the move is done inside the
IMAP server.  The local label defines the IMAP connection,
and therefore it indeed can be different from the actual
name of the folder in the IMAP server.

The confusion therefore arises from the fact that people
usually present their IMAP 'Inbox' as 'mbox' in upas,
but then present every other folder by their actual
names in the IMAP server, for example IMAP 'Hacking'
as 'Hacking' in upas.  In this case, if IMAP 'Hacking'
folder was known as 'nonhacking' by upas, then messages
could be moved as 'move nonhacking 677 674 Inbox' and
'move mbox 4 3 Hacking'.

As a summary, the source is the upas label and the
target is the folder in the IMAP server.

Thanks,
rsal