ftpfs no-cache patch

ftpfs no-cache patch

[mycroftiv]

[-- Attachment #1: Type: text/plain, Size: 2438 bytes --]

Summary: ftpfs caching behavior seems mostly undesirable.  This patch
eliminates caching by default and allows the previous behavior to be
restored with a -h flag.

Details: the ftpfs manpage currently states "Ftpfs caches files and
directories.  A directory will fall from the cache after 5 quiescent
minutes or if the local user changes the directory by writing or
removing a file." - but I believe this is not accurate.  According to
my testing, there is no time-based directory cache expiry.  The
ftpfs.h file does define an ISOLD macro that would fit the manpage,
but a grep reveals that the ISOLD macro is never used anywhere in the
current code.  The cache can be cleared by using the special
.flush.ftpfs directory entry.

Right now, the 9gridchan public grid is using ftpd/ftpfs to store
files with more rapid upload/download than are available from the 9p
ramfs service.  Clients also transmit messages via a shared plumber to
cause other connected users to view images and documents.  The current
ftpfs behavior clashes badly with this mode of usage - someone uploads
a file, and no one else can see it and it cannot be plumbed
until/unless clients and the plumber hit the .flush.ftpfs.

Given modern networks and machines, the caching seems unnecessary even
in usage situations other than the public grid.  From looking at the
code, the only complication I see might be the the way the caching is
used to create synthetic top directories for the VM and VMS operating
systems in proto.c.

The patch is implemented crudely by finding all the uses of the
ISCACHED macro in the code and making them conditional on the -h
docache flag.  This leaves the calls to uncache() and uncachedir()
unchanged but I don't believe this has any negative consequences.  The
patch has been only lightly tested and seems to do the trick as
desired, but I haven't studied the code or tested extensively enough
to guarantee that there aren't any other side effects.

If people think the current behavior should remain the default, with
the flag being used to turn off caching, that seems just as good to me
and the patch could remain the same except flipping the polarity of
the tests and changing the var name from 'docache' to 'nocache' and
changing my edit to the manpage to reflect this.

In general ftpfs is a crufty relic of an earlier era, but we are
getting some use from it on the public grid, so I thought I'd put this
forward.
-mycroftiv

[-- Attachment #2: Type: text/plain, Size: 1613 bytes --]

--- /sys/src/cmd/ip/ftpfs/ftpfs.c	Sat Dec 31 17:31:22 2016
+++ ftpfs.c	Mon Feb 12 17:21:42 2018
@@ -85,6 +85,7 @@
 
 char *nosuchfile = "file does not exist";
 char *keyspec = "";
+int docache = 0;
 
 void
 usage(void)
@@ -149,6 +150,9 @@
 	case 'q':
 		quiet = 1;
 		break;
+	case 'h':
+		docache = 1;
+		break;
 	} ARGEND
 	if(argc != 1)
 		usage();
@@ -401,7 +405,7 @@
 
 			/* everything else */
 			node = extendpath(node, s_copy(elems[i]));
-			if(ISCACHED(node->parent)){
+			if(docache && ISCACHED(node->parent)){
 				/* the cache of the parent is good, believe it */
 				if(!ISVALID(node)){
 					err = nosuchfile;
@@ -454,7 +458,7 @@
 		filedirty(f->node);
 	} else {
 		/* read the remote file or directory */
-		if(!ISCACHED(f->node)){
+		if(!docache || !ISCACHED(f->node)){
 			filefree(f->node);
 			if(f->node->d->qid.type & QTDIR){
 				invalidate(f->node);
@@ -464,7 +468,8 @@
 				if(readfile(f->node) < 0)
 					return errstring;
 			}
-			CACHED(f->node);
+			if(docache)
+				CACHED(f->node);
 		}
 	}
 
@@ -534,10 +539,11 @@
 		}
 	} else {
 		/* reread file if it's fallen out of the cache */
-		if(!ISCACHED(f->node))
+		if(!docache || !ISCACHED(f->node))
 			if(readfile(f->node) < 0)
 				return errstring;
-		CACHED(f->node);
+		if(docache)
+			CACHED(f->node);
 		rv = fileread(f->node, (char*)mbuf, off, cnt);
 		if(rv < 0)
 			return errstring;
@@ -614,10 +620,11 @@
 	Node *p;
 
 	p = f->node->parent;
-	if(!ISCACHED(p)){
+	if(!docache || !ISCACHED(p)){
 		invalidate(p);
 		readdir(p);
-		CACHED(p);
+		if(docache)
+			CACHED(p);
 	}
 	if(!ISVALID(f->node))
 		return nosuchfile;

[-- Attachment #3: Type: text/plain, Size: 1039 bytes --]

--- /sys/man/4/ftpfs	Tue Jan 17 18:13:59 2012
+++ ftpfs.man	Tue Feb 13 02:42:57 2018
@@ -4,7 +4,7 @@
 .SH SYNOPSIS
 .B ftpfs
 [
-.B -/dqnt
+.B -/dqnth
 ]
 [
 .B -m
@@ -90,6 +90,10 @@
 To see all messages from the server use option
 .BR -d .
 .PP
+By default ftpfs does no caching of files from the remote server. The
+.BR -h
+flag enables caching. This flag may be necessary when connecting to VM and VMS systems.
+.PP
 Some systems will hangup an ftp connection that has no activity
 for a given period.  The
 .BR -K
@@ -183,11 +187,7 @@
 However, walking to any valid directory on that machine will succeed
 and cause that directory entry to appear under the mount point.
 .PP
-.I Ftpfs
-caches files and directories.  A directory will fall from the cache
-after 5 quiescent minutes or if the local user changes the
-directory by writing or removing a file.
-Otherwise, remote
+If caching is active, remote
 changes to the directory that occur after the directory has
 been cached might not be immediately visible.
 Attempting to walk to

Re: [9front] ftpfs no-cache patch

[Ethan Grammatikidis]

On Tue, Feb 13, 2018, at 8:09 PM, mycroftiv@sphericalharmony.com wrote:
> Summary: ftpfs caching behavior seems mostly undesirable.  This patch
> eliminates caching by default and allows the previous behavior to be
> restored with a -h flag.
> 
> Details: the ftpfs manpage currently states "Ftpfs caches files and
> directories.  A directory will fall from the cache after 5 quiescent
> minutes or if the local user changes the directory by writing or
> removing a file." - but I believe this is not accurate.  According to
> my testing, there is no time-based directory cache expiry.  The
> ftpfs.h file does define an ISOLD macro that would fit the manpage,
> but a grep reveals that the ISOLD macro is never used anywhere in the
> current code.  The cache can be cleared by using the special
> .flush.ftpfs directory entry.

I support this. I've occasionally used ftpfs to transfer files from or to (or browse) my Android devices. Even in that light usage, having to manually flush was a big pain. I wrote a script, called it "fff", so I could flush it with just that one word, but but it was still a big pain. In the end, I configured rc-httpd to serve a big chunk of my homedir to my local network only, and used the web browser on my phone.