[9fans] Re: tracking file modifiers (was: home, end ^h^j^k^l)

[9fans] Re: tracking file modifiers (was: home, end ^h^j^k^l)

[Paul C Lustgarten]

At 10:14 AM 5/19/2001 -0400, rob pike wrote:
> The -m flag reports the muid (modifier uid) of the file, as reported
> in the new 9P protocol.  This is the person who most recently modified
> the file, instead of the person who created it.

If I had a collection of projects using Plan 9, and wanted
to bill back file store (WORM) usage to those projects,
would this muid provide enough information to do so?
I'm willing to assume that each user is a member of only
one project.  And I don't care about daily cache usage -
I just want to allocate responsibility for consumption
of the blocks on the WORM, while having all the other 
benefits of sharing that WORM amongst those projects.

I'm imagining that the muid would make it easy to
handle the simple case of files that are modified by
just a single user (like me editing my lib/profile).
What about files written by multiple users (e.g.,
several developers editing a shared project file,
all within the same day), or append-only (log,
mailbox) files that have been extended by several
different users (again, within a single day)?

Also, on the topic of managing WORM consumption,
is there some way to protect the WORM from excess
consumption, using the new muid or otherwise?
I'm thinking of scenarios such as programming mistakes
where a user accidentally writes a bunch of stuff that
they didn't intend and don't notice, or where they do
something stupid like copying over the entire project's
source tree in order to make a personal build with
a couple of modified files.  I keep thinking of being
able to impose per-user quotas that would be enforced
by the system itself (as part of the nightly dumps?),
as a safety measure complementing the economic
incentive created by charging the projects for what
their users do consume.

Re: [9fans] Re: tracking file modifiers (was: home, end ^h^j^k^l)

[Boyd Roberts]

From: "Paul C Lustgarten" <plus@cosym.net>
> Also, on the topic of managing WORM consumption,
> is there some way to protect the WORM from excess
> consumption, using the new muid or otherwise?

a mod may never hit the worm.  iirc there's a memory
cache, a magnetic cache and then the worm.  even munging
a block only gets a new copy of the block should it hit
the worm.

the file-server is cleverly constructed so that only
the tree and modified blocks chew up worm blocks.  put
the calculations in the chunk of code that burns worm
blocks.

> I'm thinking of scenarios such as programming mistakes
> where a user accidentally writes a bunch of stuff that
> they didn't intend and don't notice, or where they do
> something stupid like copying over the entire project's
> source tree in order to make a personal build with
> a couple of modified files.

you can't code against stupidity.  that is the 'knife school'
of design.  are you sure you want to cut that?

Re: [9fans] Worm consumption (was: Re: tracking file modifiers)

[Martin Harriss]

Paul C Lustgarten wrote:

>
> Also, on the topic of managing WORM consumption,
> is there some way to protect the WORM from excess
> consumption, using the new muid or otherwise?
> I'm thinking of scenarios such as programming mistakes
> where a user accidentally writes a bunch of stuff that
> they didn't intend and don't notice, or where they do
> something stupid like copying over the entire project's
> source tree in order to make a personal build with
> a couple of modified files.  I keep thinking of being
> able to impose per-user quotas that would be enforced
> by the system itself (as part of the nightly dumps?),
> as a safety measure complementing the economic
> incentive created by charging the projects for what
> their users do consume.

You might consider having a separate "scratch" file system not backed up by
a WORM.  Anything transient you put in scratch.  Anything precious you put
in the WORMed file system.

Martin

Re: [9fans] Worm consumption (was: Re: tracking file modifiers)

[Boyd Roberts]

From: "Martin Harriss" <martin@Princeton.EDU>
> You might consider having a separate "scratch" file system not backed up by
> a WORM.  Anything transient you put in scratch.  Anything precious you put
> in the WORMed file system.

that's such a bad idea.  the current design gives you that for free
if the mem/mag caches are large enough.  why complicated it?

Re: [9fans] Worm consumption (was: Re: tracking file modifiers)

[Paul C Lustgarten]

[-- Attachment #1: Type: text/plain, Size: 2662 bytes --]

> You might consider having a separate "scratch" file system not backed up by
> a WORM.  Anything transient you put in scratch.  Anything precious you put
> in the WORMed file system.

Yes, we had that on our system the years I was at Murray Hill,
and it worked pretty well for large things that we wanted to keep
arround indefinitely, but which we didn't want to waste WORM
space on, e.g., because they could be recreated if needed
(such as 50MB PDF files of someone's product manual,
or detailed (voluminous) logs that were collected continuously
of which only the newest couple weeks' worth were useful).
[To Boyd's comment - this is the difference from the regular
cache mechanism.  I can keep things in this explicitly un-WORMed
region indefinitely.  We had some files there for years.
The log files we'd reset every year or so.]

Such an arrangement is a voluntary mechanism that requires explicit
thought by the user.  Works pretty well for those who both (a) know
what they're doing and (b) are mindful of the WORM dynamics.

What I'm after now is a way to deal with users who are less informed
and/or less mindful.  (Which is to say, normal users.  :-)   How would
I (or anyone) manage a large Plan 9 installation with a large user base?

Best scheme I've been able to come up with so far is a two-pronged
approach:

	1. impose economic accountability for (disk) resource consumption
and
	2. provide technical constraints (guard rails) to protect the system
	    (and possibly the users themselves) against wayward users

I'm hopeful that the new muid capability in 9P2000 & associated 
fileserver may provide a crucial hook for building both of these
into the system.  But I don't have a complete picture (yet)
of how to do that.  Or whether this is the right way to tackle
the problem.

In general, I don't like systems to second guess the users and
try too intrusively to protect them from their own actions.
[Hmm - same goes for governments and *their* users . . .
but that's a matter for another forum.]  So the key issue is
probably more a matter of protecting the *system* from
the users.  Do we need disk quotas?  What other options
do we have?   Freely interacting with large numbers of 
users while having *no* protection?  [The CDC still
recommends against that, last I heard.]  A "morning-after"
command to recover from particularly egregious mistakes -
e.g., by removing stuff from the WORM *after* the dumps
(since we know it's not *really* a WORM any more)?
(Egads.  How could we ever live with ourselves again
after such a violation of the model?)

???


	Paul
	plus@cosym.net


[-- Attachment #2: Type: message/rfc822, Size: 3601 bytes --]

From: Martin Harriss <martin@Princeton.EDU>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] Worm consumption (was: Re: tracking file modifiers)
Date: Sat, 19 May 2001 23:12:02 -0400
Message-ID: <3B073602.5B439C05@princeton.edu>

Paul C Lustgarten wrote:

>
> Also, on the topic of managing WORM consumption,
> is there some way to protect the WORM from excess
> consumption, using the new muid or otherwise?
> I'm thinking of scenarios such as programming mistakes
> where a user accidentally writes a bunch of stuff that
> they didn't intend and don't notice, or where they do
> something stupid like copying over the entire project's
> source tree in order to make a personal build with
> a couple of modified files.  I keep thinking of being
> able to impose per-user quotas that would be enforced
> by the system itself (as part of the nightly dumps?),
> as a safety measure complementing the economic
> incentive created by charging the projects for what
> their users do consume.

You might consider having a separate "scratch" file system not backed up by
a WORM.  Anything transient you put in scratch.  Anything precious you put
in the WORMed file system.

Martin--upas-aqugepuaybkgltflddfcjxoojf--