Re: [9fans] what a mess the Unix world has become

Re: [9fans] what a mess the Unix world has become

[C H Forsyth]

[-- Attachment #1: Type: text/plain, Size: 129 bytes --]

i don't quite see how they could provide a `commit' effect unless they finally
added a Close operation to the clunky protocol.

[-- Attachment #2: Type: message/rfc822, Size: 2625 bytes --]

From: "Ronald G. Minnich" <rminnich@lanl.gov>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] what a mess the Unix world has become
Date: Mon, 10 Feb 2003 08:20:07 -0700 (MST)
Message-ID: <Pine.LNX.4.44.0302100818320.23236-100000@carotid.ccs.lanl.gov>

On Mon, 10 Feb 2003, Douglas A. Gwyn wrote:

>There are
> transaction-oriented filesystems (MUMPS comes to mind)
> that cluster requests until a "commit" and implement a
> well-defined rollback scheme when the commit is
> unsuccessful.

Actually at some point in the 90s NFS became like that on SunOS (well,
minus the rollback system). That was about the time that close could
return an error, meaning "remember those writes you did yesterday? Well,
some of them may not have worked out, so you probably lost data. Have a
nice day."

Needless to say, many programs did not quite grasp the concept.

ron

Re: [9fans] what a mess the Unix world has become

[Ronald G. Minnich]

On Mon, 10 Feb 2003, C H Forsyth wrote:

> i don't quite see how they could provide a `commit' effect unless they finally
> added a Close operation to the clunky protocol.

that was the joy of it. NFS was a very informal business. Close turned to
a set of operations that were in essence a commit of cached writes. These
could fail ...

ron

Re: [9fans] what a mess the Unix world has become

[Geoff Collyer]

Checking close() return values after writing on a descriptor is good
hygiene, but as I recall, NFS made error detection harder by not
necessarily reporting write errors (even synchronous ones like ENOSPC)
via either write() or close(); one had to use fsync() just before the
close() to force any write errors to be reported.

 From code I wrote in the late 1980s:

/*
 * nfclose(stream) - flush the stream, fsync its file descriptor and
 * fclose the stream, checking for errors at all stages.  This dance
 * is needed to work around the lack of Unix file system semantics
 * in Sun's NFS.  Returns EOF on error.
 */

#include <stdio.h>

int
nfclose(stream)
register FILE *stream;
{
	register int ret = 0;

	if (fflush(stream) == EOF)
		ret = EOF;
	if (fsync(fileno(stream)) < 0)		/* may get delayed error here */
		ret = EOF;
	if (fclose(stream) == EOF)
		ret = EOF;
	return ret;
}

Re: [9fans] what a mess the Unix world has become

[Douglas A. Gwyn]

Ronald G. Minnich wrote:
> Actually at some point in the 90s NFS became like that on SunOS (well,
> minus the rollback system). That was about the time that close could
> return an error, meaning "remember those writes you did yesterday? Well,
> some of them may not have worked out, so you probably lost data. Have a
> nice day."

Unix has always been like that.  Write() would succeed if
a kernel buffer received the data, without waiting for the
actual transfer to the device.  This has been known to
cause great fun with terminal lines.

Re: [9fans] what a mess the Unix world has become

[Ronald G. Minnich]

On Mon, 10 Feb 2003, Douglas A. Gwyn wrote:

>There are
> transaction-oriented filesystems (MUMPS comes to mind)
> that cluster requests until a "commit" and implement a
> well-defined rollback scheme when the commit is
> unsuccessful.

Actually at some point in the 90s NFS became like that on SunOS (well,
minus the rollback system). That was about the time that close could
return an error, meaning "remember those writes you did yesterday? Well,
some of them may not have worked out, so you probably lost data. Have a
nice day."

Needless to say, many programs did not quite grasp the concept.

ron

Re: [9fans] what a mess the Unix world has become

[Russ Cox]

> What OS facility is there for determining whether a file
> would be writable without actually modifying the file?

umm, open(2)?  this doesn't handle the AEXEC|AREAD
or AEXEC|AWRITE cases particularly well, and it's not
as fast as traditional access, but it gives good answers.

#include <u.h>
#include <libc.h>

int
access(char *name, int mode)
{
	int fd;
	Dir *db;
	static char omode[] = {
		0,
		OEXEC,
		OWRITE,
		ORDWR,
		OREAD,
		OEXEC,	/* only approximate */
		ORDWR,
		ORDWR	/* only approximate */
	};

	if(mode == AEXIST){
		db = dirstat(name);
		free(db);
		if(db != nil)
			return 0;
		return -1;
	}
	fd = open(name, omode[mode&7]);
	if(fd >= 0){
		close(fd);
		return 0;
	}
	return -1;
}

Re: [9fans] what a mess the Unix world has become

[Douglas A. Gwyn]

Aharon Robbins wrote:
>>I think that one really should be able to do "test -r foo && cat foo"
>>without cat failing because of permissions.

Yes, that was the intention.  But the question comes into
even sharper focus if you try to test for writability.
What OS facility is there for determining whether a file
would be writable without actually modifying the file?
The answer: access(2).  Not a mere examination of the
mode bits; not a trial write to the file.  When something
like NFS is bolted onto Unix, access(2) is one of the
things that needs to be made to work right; if NFS has
complicated access rules then that is what access() needs
to tap into.  (Better in the filesystem support than in
user-mode utilities.)
VAX/VMS had a "probe" call that, as I recall, checked
whether an operation would succeed but didn't do the
operation.
Of course, all such facilities are somewhat limited in
that the condition of the file might change between a
probe and actually doing the operation.  There are
transaction-oriented filesystems (MUMPS comes to mind)
that cluster requests until a "commit" and implement a
well-defined rollback scheme when the commit is
unsuccessful.

Re: [9fans] what a mess the Unix world has become

[Mike Haertel]

> I think that one really should be able to do "test -r foo && cat foo"
> without cat failing because of permissions.

This could never be guaranteed to succeed, even with a supposedly
perfect version of "test", because "foo" might get altered between
the "test" and the "cat".

[9fans] what a mess the Unix world has become

[Aharon Robbins]

I forward this for the amusement of the list members.  My initial
reply to this question was that the "Unix guru gives up in disgust
and moves to Plan 9."  Would that I could.

Arnold

> Date: Sat, 8 Feb 2003 09:37:50 -0700 (MST)
> Subject: Readable files: a Unix philosophy question for gurus
>
> In "man test", we find this documentation:
>
>        -r FILE
>               FILE exists and is readable
>
> Similar wording appears in vendor man pages, although Compaq OSF/1
> is a bit more precise:
>
>        -r file
>            TRUE if file exists and has read permission.
>
> Consider a file with these permissions:
>
> 	% ls -l foo.xml
> 	-rwSr-Srwx    2 beebe    staff      174440 Feb  8 08:20 foo.xml
>
> created by
>
> 	chmod u+s,g+s,o+rwxt foo.xml
>
> Now look at this:
>
> (1) On the file server:
>
> 	/bin/test -r foo.xml && echo file is readable
> 	file is readable
>
> 	/usr/local/bin/test -r foo.xml && echo file is readable
> 	file is readable
>
> 	/usr/local/bin/test --version
> 	test (GNU coreutils) 4.5.6
> 	...
>
> 	/bin/cat foo.xml > /dev/null
>
> 	echo $?
> 	0
>
> (2) On a [Sun Solaris 2.8] client, where the filesystem is mounted via
>     NFS with the attributes remote/read/write/nosuid/grpid/intr/noquota:
>
> 	/bin/test -r foo.xml && echo file is readable
> 	[no output]
>
> 	/usr/local/bin/test -r foo.xml && echo file is readable
> 	file is readable
>
> 	/bin/cat foo.xml > /dev/null
> 	cat: cannot open foo.xml
>
> 	/usr/local/bin/cat foo.xml > /dev/null
> 	/usr/local/bin/cat: foo.xml: Permission denied
>
> 	echo $?
> 	1
>
> The $65,536 question is: what should test report?  In this case, the
> setuid bit combined with the mount options makes the file unreadable,
> and the Solaris test command reflects that, but the GNU coreutils
> one does not.
>
> Experiments with native test on other systems produced these results:
>
> 	SGI IRIX 6.5:		file is readable, cat fails
> 	Compaq OSF/1 4.0:	no output from test, cat fails
>
> On AIX, my home filesystem is not mounted with special attributes, so
> I could not make the same experiment.
>
> On FreeBSD, chmod would not do g+s or o+t for me, and the file
> remained readable for both test and cat.
>
> My feeling is that the GNU version of test (going back to at least
> sh-utils 2.0.11 on Red Hat 7.2, and likely much earlier), and the IRIX
> version too, are wrong.
>
> I think that one really should be able to do "test -r foo && cat foo"
> without cat failing because of permissions.
>
> Comments?