Re: [9fans] AFS-client for Plan9 - ?

Re: [9fans] AFS-client for Plan9 - ?

[nigel]

[-- Attachment #1: Type: text/plain, Size: 842 bytes --]

Let me explain (and, yes, this is a quote from Fawlty Towers)

Programs which do not explicitly close flies before invoking exit() are
not

1) badly written

or

2) forcing Unix implementations to have to do close on exit to compensate

These are the defined semantics. So the reason is "because that's what
the program quite reasonably expects".

Just in the same way that you don't have to free all memory before
calling exit, unless, as in td's words "they've changed that since I
last looked".  I know "they" did for 16 bit Windows applications, and
it's certainly the case for semaphores in Unix.

> Grr... OK, let me rephrase it:
> 
> 	I really doubt that there is (or ever was) a UNIX variant that
> would not do close-on-exit. Reason: lots and lots of programs that would
> kill such system with file leaks.


[-- Attachment #2: Type: message/rfc822, Size: 2202 bytes --]

From: Alexander Viro <viro@math.psu.edu>
To: Tom Duff <td@pixar.com>
Cc: 9fans@cse.psu.edu
Subject: Re: [9fans] AFS-client for Plan9 - ?
Date: Tue, 14 Nov 2000 19:31:13 -0500 (EST)
Message-ID: <Pine.GSO.4.21.0011141923210.5482-100000@weyl.math.psu.edu>



On Tue, 14 Nov 2000, Tom Duff wrote:

> > Precisely. And any kernel that will try _not_ to do that will die under the
> > leaks produced by the $BIGNUM of applications expecting it do act as any
> > sane UNIX should.
> Which UNIX version lacks this behavior?
> (Or are you trying to run UNIX programs
> on some non-UNIX?)

Grr... OK, let me rephrase it:

	I really doubt that there is (or ever was) a UNIX variant that
would not do close-on-exit. Reason: lots and lots of programs that would
kill such system with file leaks.

IOW, "not likely" was about "some numskull changed it" part - such attempt
would backfire immediately. We are in violent agreement...

Re: [9fans] AFS-client for Plan9 - ?

[Alexander Viro]

On Wed, 15 Nov 2000 nigel@9fs.org wrote:

> Let me explain (and, yes, this is a quote from Fawlty Towers)
> 
> Programs which do not explicitly close flies before invoking exit() are
> not
> 
> 1) badly written

Sure.

> or
> 
> 2) forcing Unix implementations to have to do close on exit to compensate

Sure.

> These are the defined semantics. So the reason is "because that's what
> the program quite reasonably expects".
... and so many programs _do_ expect it (quite reasonably) that changing
that behaviour would require the audit and rewrite of the whole userland.

Folks, how about finally agreeing that we agree and stopping that subthread?
It's getting really silly.

Re: [9fans] AFS-client for Plan9 - ?

[rog]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

however, it's worth remembering, that in the Unix i remember at any
rate, _exit() is not always invoked on program exit.  it isn't invoked
if the program dies from a signal, for instance.

i've always presumed that the final close of fds is done when their
refcount drops to zero, i.e.  when the exited process's file table is
cleared up by the kernel.

so the actual _?exit() system calls are not that crucial to the issue
of file closing under AFS. stdio is a different matter, of course.

but i'm not entirely sure whether we're on topic here...  :-)

  cheers,
    rog.


[-- Attachment #2: Type: message/rfc822, Size: 2268 bytes --]

To: <cse.psu.edu!9fans>
Subject: Re: [9fans] AFS-client for Plan9 - ?
Date: Mon, 20 Nov 2000 14:24:48 +0100
Message-ID: <009201c052f5$420a2480$0ab9c6d4@cybercable.fr>

From: Douglas A. Gwyn <gwyn@arl.army.mil>

> Boyd Roberts wrote:
> > here we go.  the kernel closes the fd's as a part of exit,
> > like td said.
> 
> Further, upon return from main(), all the exit() actions
> are performed.
> 
> _exit(), on the other hand, does not perform any stdio.
> (Its main use is in a child branch of a fork.)

absolutely correct.

Re: [9fans] AFS-client for Plan9 - ?

[Douglas A. Gwyn]

rog@vitanuova.com wrote:
> however, it's worth remembering, that in the Unix i remember at any
> rate, _exit() is not always invoked on program exit.  it isn't invoked
> if the program dies from a signal, for instance.

Right; upon abnormal termination (what POSIX terms "termination
with actions" as I recall), the application process has lost
control, so only the kernel clean-up actions can be performed
(which include closing open FDs, as others have remarked).

> i've always presumed that the final close of fds is done when their
> refcount drops to zero, i.e.  when the exited process's file table is
> cleared up by the kernel.

It depends on the kernel (there might be process-owned locks to
release, for example), but that's the roght general idea.
Closing a "file" decrements its in-use count and only when that
count goes to zero is the "real" close function performed, e.g.
device driver drops READY etc.

> so the actual _?exit() system calls are not that crucial to the issue
> of file closing under AFS. stdio is a different matter, of course.

Yes, sorry if I digressed.

Re: [9fans] AFS-client for Plan9 - ?

[David Gordon Hogan]

> > Precisely. And any kernel that will try _not_ to do that will die under the
> > leaks produced by the $BIGNUM of applications expecting it do act as any
> > sane UNIX should.
> Which UNIX version lacks this behavior?
> (Or are you trying to run UNIX programs
> on some non-UNIX?)

The hypothetical one positted earlier in this thread.

Sorry to butt in, but this is just getting riddiculous.

Re: [9fans] AFS-client for Plan9 - ?

[presotto]

[-- Attachment #1: Type: text/plain, Size: 161 bytes --]

Andrew can cache whole files locally and send back journaled updates.
It's a lot more than cfs.  This is not a trivial project.  Might be
interesting though.

[-- Attachment #2: Type: message/rfc822, Size: 1501 bytes --]

From: anothy@cosym.net
To: 9fans@cse.psu.edu
Subject: Re: [9fans] AFS-client for Plan9 - ?
Date: Mon, 13 Nov 2000 15:19:02 -0500
Message-ID: <20001113201911.6003E199EB@mail.cse.psu.edu>

//Local caching would be great, I think.

okay, let me say off the bat i have no idea how
AFS handles caching, so this comparison may be
totally inapropriate. but for fs caching in
Plan 9 (without 9p↔AFS), take a look at cfs(4)
and the -C option to mount in bind(1).

if your goal is talking to AIX or other Unix
boxes, take a look at u9fs(4).
-α.

Re: [9fans] AFS-client for Plan9 - ?

[anothy]

//Local caching would be great, I think.

okay, let me say off the bat i have no idea how
AFS handles caching, so this comparison may be
totally inapropriate. but for fs caching in
Plan 9 (without 9p↔AFS), take a look at cfs(4)
and the -C option to mount in bind(1).

if your goal is talking to AIX or other Unix
boxes, take a look at u9fs(4).
-α.

Re: [9fans] AFS-client for Plan9 - ?

[Wladimir Mutel]

anothy@cosym.net wrote:
> //Local caching would be great, I think.

> okay, let me say off the bat i have no idea how
> AFS handles caching, so this comparison may be
> totally inapropriate. but for fs caching in
> Plan 9 (without 9pAFS), take a look at cfs(4)
> and the -C option to mount in bind(1).

> if your goal is talking to AIX or other Unix
> boxes, take a look at u9fs(4).

	Here is a quote from their doc
http://oss.software.ibm.com/developerworks/opensource/afs/docs/html/AdminGd/auagd007.htm :

AFS Implements Save on Close

When an application issues the UNIX close system call on a file, the Cache
Manager performs a synchronous write of the data to
the File Server that maintains the central copy of the file. It does not
return control to the application until the File
Server has acknowledged receipt of the data. For the fsync system call,
control does not return to the application until the
File Server indicates that it has written the data to non-volatile storage
on the file server machine. 

When an application issues the UNIX write system call, the Cache Manager
writes modifications to the local AFS client cache
only. If the local machine crashes or an application program exits without
issuing the close system call, it is possible that
the modifications are not recorded in the central copy of the file
maintained by the File Server. The Cache Manager does
sometimes write this type of modified data from the cache to the File
Server without receiving the close or fsync system call,
for example if it needs to free cache chunks for new data. However, it is
not generally possible to predict when the Cache
Manager transfers modified data to the File Server in this way. 

The implication is that if an application's Save option invokes the write
system call rather than close or fsync, the changes
are not necessarily stored permanently on the File Server machine. Most
application programs issue the close system call for
save operations, as well as when they finish handling a file and when they
exit. 


--
mwg@alkar.net, 399916, 340044, 7442333, 7786458 -  

Re: [9fans] AFS-client for Plan9 - ?

[Tom Duff]

Wladimir Mutel <mwg@alkar.net> wrote:
> If the local machine crashes or an application program exits without
> issuing the close system call, it is possible that
> the modifications are not recorded in the central copy of the file
> maintained by the File Server.

I'm confused.  UNIX applications can't exit without issuing the
close system call.  The exit system call internally calls out
to close for each open file.  Unless some numskull changed it
since the last time I had my fingers in it.  (They've had plenty
of opportunity -- I haven't touched a non-BSD UNIX kernel in many
years.)

-- 
Tom Duff.  If all else fails, make it up.

Re: [9fans] AFS-client for Plan9 - ?

[Boyd Roberts]

From: Tom Duff <td@pixar.com>

> Unless some numskull changed it since the last time

sadly you're probably right -- between the sheets.

Re: [9fans] AFS-client for Plan9 - ?

[Alexander Viro]

On Tue, 14 Nov 2000, Tom Duff wrote:

> Wladimir Mutel <mwg@alkar.net> wrote:
> > If the local machine crashes or an application program exits without
> > issuing the close system call, it is possible that
> > the modifications are not recorded in the central copy of the file
> > maintained by the File Server.
> 
> I'm confused.  UNIX applications can't exit without issuing the
> close system call.  The exit system call internally calls out
> to close for each open file.  Unless some numskull changed it
> since the last time I had my fingers in it.  (They've had plenty
> of opportunity -- I haven't touched a non-BSD UNIX kernel in many
> years.)

Not bloody likely. _Many_ applications exit without calling close(), so
such system would die horribly under the leaks. OTOH... name a perversion
and somebody will find a place where it had been implemented.

Re: [9fans] AFS-client for Plan9 - ?

[Boyd Roberts]

From: Alexander Viro <viro@math.psu.edu>

> Not bloody likely. _Many_ applications exit without calling close

here we go.  the kernel closes the fd's as a part of exit,
like td said. 

duff knows what he's doing -- between the sheets.

you wanna outcode duff?

Re: [9fans] AFS-client for Plan9 - ?

[Tom Duff]

On Nov 14, 11:51pm, Boyd Roberts wrote:
> duff knows what he's doing -- between the sheets.
>
> you wanna outcode duff?

I am sitting in my workshop reading this,
and now my head won't fit through the door.

-- 
Tom Duff.  I spent an interesting evening recently with a grain of salt.

Re: [9fans] AFS-client for Plan9 - ?

[Chris Locke]

The kernel cannot close the fds if the machine has crashed.
The original post said "if the local machine crashes".
This can also be read as  "some numskul knocked
the reset button / kicked the power lead out /
raccoon sat on local power transformer etc. etc."

Chris.

Re: [9fans] AFS-client for Plan9 - ?

[Douglas A. Gwyn]

Boyd Roberts wrote:
> here we go.  the kernel closes the fd's as a part of exit,
> like td said.

Further, upon return from main(), all the exit() actions
are performed.

_exit(), on the other hand, does not perform any stdio.
(Its main use is in a child branch of a fork.)

Re: [9fans] AFS-client for Plan9 - ?

[Boyd Roberts]

From: Douglas A. Gwyn <gwyn@arl.army.mil>

> Boyd Roberts wrote:
> > here we go.  the kernel closes the fd's as a part of exit,
> > like td said.
> 
> Further, upon return from main(), all the exit() actions
> are performed.
> 
> _exit(), on the other hand, does not perform any stdio.
> (Its main use is in a child branch of a fork.)

absolutely correct.

Re: [9fans] AFS-client for Plan9 - ?

[Tom Duff]

On Nov 14,  5:41pm, Alexander Viro wrote:
> On Tue, 14 Nov 2000, Tom Duff wrote:
> > UNIX applications can't exit without issuing the
> > close system call.  The exit system call internally calls out
> > to close for each open file.
>
> Not bloody likely. _Many_ applications exit without calling close(), so
> such system would die horribly under the leaks.

No.
Read what I said.
The exit system call closes all open files.
You cannot exit without closing.  (By `cannot'
I do not mean that you are enjoined from so
doing, but rather that it is not possible to
do so.)

You can, however, exit without fclosing, although
the stdio library tries to make that difficult.

-- 
Tom Duff.  I spent an interesting evening recently with a grain of salt.

Re: [9fans] AFS-client for Plan9 - ?

[Alexander Viro]

On Tue, 14 Nov 2000, Tom Duff wrote:

> On Nov 14,  5:41pm, Alexander Viro wrote:
> > On Tue, 14 Nov 2000, Tom Duff wrote:
> > > UNIX applications can't exit without issuing the
> > > close system call.  The exit system call internally calls out
> > > to close for each open file.
> >
> > Not bloody likely. _Many_ applications exit without calling close(), so
> > such system would die horribly under the leaks.
> 
> No.
> Read what I said.

Erm... Ditto.

> The exit system call closes all open files.

Precisely. And any kernel that will try _not_ to do that will die under the
leaks produced by the $BIGNUM of applications expecting it do act as any
sane UNIX should. Thus "not bloody likely" - any attempt to change the
behaviour in that area is pretty much doomed.

Re: [9fans] AFS-client for Plan9 - ?

[Tom Duff]

> Precisely. And any kernel that will try _not_ to do that will die under the
> leaks produced by the $BIGNUM of applications expecting it do act as any
> sane UNIX should.
Which UNIX version lacks this behavior?
(Or are you trying to run UNIX programs
on some non-UNIX?)

-- 
Tom Duff.  I spent an interesting evening recently with a grain of salt.

Re: [9fans] AFS-client for Plan9 - ?

[Alexander Viro]

On Tue, 14 Nov 2000, Tom Duff wrote:

> > Precisely. And any kernel that will try _not_ to do that will die under the
> > leaks produced by the $BIGNUM of applications expecting it do act as any
> > sane UNIX should.
> Which UNIX version lacks this behavior?
> (Or are you trying to run UNIX programs
> on some non-UNIX?)

Grr... OK, let me rephrase it:

	I really doubt that there is (or ever was) a UNIX variant that
would not do close-on-exit. Reason: lots and lots of programs that would
kill such system with file leaks.

IOW, "not likely" was about "some numskull changed it" part - such attempt
would backfire immediately. We are in violent agreement...

Re: [9fans] AFS-client for Plan9 - ?

[Boyd Roberts]

From: Alexander Viro <viro@math.psu.edu>

> IOW, "not likely" was about "some numskull changed it" part - such attempt
> would backfire immediately. We are in violent agreement...

if some 'numskull' would stand up and call bullshit on dat
the world world would be a better place.

tell me why the BSD f/S was written?

quadratic rehash -- give me a break.

Re: [9fans] AFS-client for Plan9 - ?

[Wladimir Mutel]

> Are you volunteering?  I don't know of anyone else writing
> an AFS<->9P converter.

	Oh sorry, I do not feel excessive inspiration and (the worst) spare
	time. I often think I am only able to drop in an idea ...

--
mwg@alkar.net, 399916, 340044, 7442333, 7786458 - Владимир Мутель

Re: [9fans] AFS-client for Plan9 - ?

[presotto]

[-- Attachment #1: Type: text/plain, Size: 84 bytes --]

Are you volunteering?  I don't know of anyone else writing
an AFS<->9P converter.

[-- Attachment #2: Type: message/rfc822, Size: 1880 bytes --]

From: Wladimir Mutel <mwg@alkar.net>
To: 9fans@cse.psu.edu
Subject: [9fans] AFS-client for Plan9 - ?
Date: Mon, 13 Nov 2000 16:30:52 GMT
Message-ID: <8uoutp$als$1@pandora.alkar.net>

	IBM recently opened their AFS (distributed file system with local
	caching) -
	http://oss.software.ibm.com/developerworks/opensource/afs/

	Could there be any reason or use to make Plan9 AFS-client (like
	afssrv, to connect to afs-servers and map 9p to afs) ?

	Could Plan9 fs(8) get some afs features ? Local caching would be
	great, I think.

--
mwg@alkar.net, 399916, 340044, 7442333, 7786458 - Владимир Мутель

Re: [9fans] AFS-client for Plan9 - ?

[Boyd Roberts]

----- Original Message ----- 
From: <presotto@plan9.bell-labs.com>

> Are you volunteering?  I don't know of anyone else writing
> an AFS<->9P converter.

oooh, protocol conversion.  danger, will robinson.

although, 9P is sane.  whereas, NFS is not.

[9fans] AFS-client for Plan9 - ?

[Wladimir Mutel]

	IBM recently opened their AFS (distributed file system with local
	caching) -
	http://oss.software.ibm.com/developerworks/opensource/afs/

	Could there be any reason or use to make Plan9 AFS-client (like
	afssrv, to connect to afs-servers and map 9p to afs) ?

	Could Plan9 fs(8) get some afs features ? Local caching would be
	great, I think.

--
mwg@alkar.net, 399916, 340044, 7442333, 7786458 - Владимир Мутель