Re: [9fans] union directories

Re: [9fans] union directories

[G. David Butler]

No, I did NOT pay him to say this! :)


"Douglas A. Gwyn" <DAGwyn@null.net> wrote:
__________
>rob pike wrote:
>> This is all a legacy of Unix's model that create means two different
>> things in the two situations.
>
>Well, there you go -- an attempt to create an existing file should
>fail.  That is a simpler and more useful property.
>

Re: [9fans] union directories

[Adrian Tritschler]

Jack Johnson wrote:
> Ronald G. Minnich wrote:
>> truthfully, how many of you out there started out using disk/kfscmd
>> allow when you wanted to modify the kernel source?

I'll admit it.  How many knew any other way?  How many yet know?  Maybe a
little mention on the Wiki at
http://plan9.bell-labs.com/wiki/plan9/Compiling_kernels would be helpful.

Several months into my plan9 learning and I'm still getting my head around the
plan9 way of doing some things... most things...

> I'll admit it, but I'm still a newbie so I have an extension of this
> question.
> 
> Is there any way to, say, recompile a source tree with subfolders
> without pre-creating those subfolders in the union directory?  Or,
> what's the correct way to do something like:
> 
> mkdir 9
> bind -bc 9 9
> bind -ac /sys/src/9 9
> cd 9
> mk
> 
> (though that's not what I'm doing, I'm just needed an easy example)
> 
> -Jack

	Adrian

---------------------------------------------------------------
Adrian Tritschler    mailto:Adrian.Tritschler@its.monash.edu.au
Latitude 38°S, Longitude 145°E, Altitude 50m,      Shoe size 44
---------------------------------------------------------------

Re: [9fans] union directories

[anyrhine]

...
> files obscured within a union mount are not accessible to me; in fact,
> apart from telling me that the directory probably contains a union
> mount, information on those files is of no use at all.

I like having the duplicates precisely for that information.
I tend to use a quite fixed set of bindings, and most of the time it's
just a question of wheter I've made them, not what I've bound
where. 

...
> acme the same - what's the point of showing me two of the same names
> next to each other?  it just causes clutter. clicking on either
> one performs exactly the same action.

Especially in acme, because there it's (imho) easier to look for the expected
duplicates than execute something, focus on the output window for
the results, and then focus back to the original window.

Re: [9fans] union directories

[Russ Cox]

> Just to check user/group/world w flags.

even if those are set the file system might
be read-only.  for example when you 9fs sources
you can't change things in /n/source/plan9/cron
despite it being mode 777 and not a union mount.

Re: [9fans] union directories

[Keith Nash]

>> Is there any way to test the writability
>> of a directory, other than by attempting
>> to create a file there, or by combining
>> ls -ld with an examination of the output of ns?

>No.

>I'm not even sure how ls -ld would help you.
Just to check user/group/world w flags.

>ns would give you a first approximation.

Re: [9fans] union directories

[Russ Cox]

> Is there any way to test the writability
> of a directory, other than by attempting
> to create a file there, or by combining
> ls -ld with an examination of the output of ns?

No.

I'm not even sure how ls -ld would help you.
ns would give you a first approximation.

Re: [9fans] union directories

[Keith Nash]

> Ls tells the truth!  I see no value in causing it to lie.

It tells the truth and nothing but the truth, but does it tell the whole truth?

I might believe from running ls -ld on a directory that I can create a new file in it; but I can't, if the directory is a union mount, and none of the elements was bound with -c.

Is there any way to test the writability of a directory, other than by attempting to create a file there, or by combining ls -ld with an examination of the output of ns?


Keith.

Re: [9fans] union directories

[rog]

[-- Attachment #1: Type: text/plain, Size: 1728 bytes --]

> Ls tells the truth!  I see no value in causing it to lie.

i'd like to be able to think of "ls" as showing me the files available
to me in a given directory.

files obscured within a union mount are not accessible to me; in fact,
apart from telling me that the directory probably contains a union
mount, information on those files is of no use at all.

there's a good implementation reason for having the duplicates show up
in raw reads (it only requires constant space in the kernel), but why
not get rid of the duplicates given that we're sorting the entries
anyway.

this isn't lying: it's showing a simplified version of the truth which
is not misleading (unlike the example in the lexnames paper).

> And then once ls lies, why doesn't ls *?  Why doesn't acme?

in fact, i think it's more important that * removes duplicates than ls.
i think it's a bug that:

	for (i in *) do something

can end up processing a single file multiple times.

acme the same - what's the point of showing me two of the same names
next to each other?  it just causes clutter. clicking on either
one performs exactly the same action.

> There might be good arguments that directory reads should
> reflect the information in the mount table, perhaps eliding
> duplicates and perhaps showing you the real stat info for 
> bound-over files (what you'd get with stat(2)).  Then again
> there are also good arguments for the way things are now.

mk was bitten by this at one stage.
it's a pity that
	ls -l | grep foo
doesn't necessarily give the same results as
	ls -ld foo

now that's a genuine case of ls (or at least unionread()) lying...
i agree about the arguments for the current behaviour though.

[-- Attachment #2: Type: message/rfc822, Size: 2798 bytes --]

From: "Russ Cox" <rsc@plan9.bell-labs.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] union directories
Date: Mon, 3 Mar 2003 13:05:41 -0500
Message-ID: <0a304f2dd88c5bdb8eb1ce403c53e037@plan9.bell-labs.com>

> Actually I have had more Unix-oriented folks ask me if in a union ls 
> there could be an option, per-file, to display the binding for that file 
> when there is more than one. 

Per-file options are a bad idea not even worth talking about.

> I got real complaints from people about seeing multiple files with the 
> same name on an ls. 

Ls tells the truth!  I see no value in causing it to lie.
And then once ls lies, why doesn't ls *?  Why doesn't acme?
Ls does one thing.  It does it well.  It's consistent with the
rest of the system.  If you want sort -u, you know where to
find it.

There might be good arguments that directory reads should
reflect the information in the mount table, perhaps eliding
duplicates and perhaps showing you the real stat info for 
bound-over files (what you'd get with stat(2)).  Then again
there are also good arguments for the way things are now.

Read the ksh example in the lexnames paper.  The last thing we
need is user-space programs that ``patch'' kernel behaviors.
Arguing that the kernel needs fixing might be reasonable.
Changing ls to hack around perceived kernel deficiencies
is the path to madness (or to Linux, take your pick).

Russ

Re: [9fans] union directories

[Russ Cox]

> Is there a better answer?

Running ls -l on the file will give you the stat info
for the actual file.  

Compare ls -l /dev | grep mouse and ls -l /dev/mouse.

Re: [9fans] union directories

[Ronald G. Minnich]

On Mon, 3 Mar 2003, Russ Cox wrote:

> Read the ksh example in the lexnames paper.  The last thing we
> need is user-space programs that ``patch'' kernel behaviors.
> Arguing that the kernel needs fixing might be reasonable.
> Changing ls to hack around perceived kernel deficiencies
> is the path to madness (or to Linux, take your pick).

I phrased it wrong. I did not mean a patch to 'ls'. 

Try again. 


I have showed folks union mounts, done an ls, and gotten complaints such
as: when you edit/cat/print a file, how do you know what you're editing?  
My only answer is weak: 'well you did the bind, so you know'.

Is there a better answer?

ron

Re: [9fans] union directories

[Russ Cox]

> Actually I have had more Unix-oriented folks ask me if in a union ls 
> there could be an option, per-file, to display the binding for that file 
> when there is more than one. 

Per-file options are a bad idea not even worth talking about.

> I got real complaints from people about seeing multiple files with the 
> same name on an ls. 

Ls tells the truth!  I see no value in causing it to lie.
And then once ls lies, why doesn't ls *?  Why doesn't acme?
Ls does one thing.  It does it well.  It's consistent with the
rest of the system.  If you want sort -u, you know where to
find it.

There might be good arguments that directory reads should
reflect the information in the mount table, perhaps eliding
duplicates and perhaps showing you the real stat info for 
bound-over files (what you'd get with stat(2)).  Then again
there are also good arguments for the way things are now.

Read the ksh example in the lexnames paper.  The last thing we
need is user-space programs that ``patch'' kernel behaviors.
Arguing that the kernel needs fixing might be reasonable.
Changing ls to hack around perceived kernel deficiencies
is the path to madness (or to Linux, take your pick).

Russ

Re: [9fans] union directories

[Ronald G. Minnich]

On Mon, 3 Mar 2003 rog@vitanuova.com wrote:

> come to that, it would be nice if ls -l used a stable sorting
> algorithm so that it was possible to tell which file attributes were
> likely to correspond to the file that would be opened under a
> particular name.

Actually I have had more Unix-oriented folks ask me if in a union ls 
there could be an option, per-file, to display the binding for that file 
when there is more than one. 

I got real complaints from people about seeing multiple files with the 
same name on an ls. 

Not that we want to put whiteouts in there, however.

ron

Re: [9fans] union directories

[rog]

> (Bind -bc 9 9 is like:
> 
> 	bind 9 9 
> 	bind -bc 9 9
> 
> hence everything appears twice.)

is there a good reason why ls doesn't eliminate duplicates when
showing union directories?

occasionally it might be useful, but as the default behaviour?

come to that, it would be nice if ls -l used a stable sorting
algorithm so that it was possible to tell which file attributes were
likely to correspond to the file that would be opened under a
particular name.

i did both of these things in inferno, with no obvious adverse
consequences.

Re: [9fans] union directories

[Geoff Collyer]

Yes, your example would have roughly the same effect as my second
example, except that mine uses /tmp/x as the mount point and yours
uses /sys/lib, and sometimes that matters.

The point to note is that

	mkdir x; bind -a /sys/lib x
and
	mkdir x; bind -ac /sys/lib x

do an implicit

	bind  /tmp/x /tmp/x

before the bind of /sys/lib, which leaves /tmp/x uncreateable unless
it is then bound into the union with -c, with something like

	bind -bc /tmp/x /tmp/x

after the bind of /sys/lib, or

	bind -c /tmp/x /tmp/x

before it.

Re: [9fans] union directories

[Jack Johnson]

Geoff Collyer wrote:
> `bind -c 9 9' remains useful, however, to retain createability of the
> directory in which a union mount is then performed:
> bind -c /tmp/x /tmp/x
> bind -a /sys/lib /tmp/x
> cd /tmp
> ; >x/FOO
> ; unmount x; ls x
> x/FOO

OK, but if I'm currently understanding this correctly, shouldn't it now be:

bind -bc /tmp/x /sys/lib
cd /sys/lib
 >FOO
cd /tmp
unmount /sys/lib; ls x
x/FOO

?

How does this differ from yours?  It *looks* like you end up with the
same scenario I originally presented.

-Jack

Re: [9fans] union directories

[Geoff Collyer]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

`bind -c 9 9' remains useful, however, to retain createability of the
directory in which a union mount is then performed:

; cd /tmp
; mkdir x
; bind -a /sys/lib x
; ns|tail -4
mount -b '#s/rio.geoff.73279' /dev
bind  /tmp/x /tmp/x
bind -a /sys/lib /tmp/x
cd /tmp
; >x/FOO
x/FOO: rc: can't open: mounted directory forbids creation

versus

; unmount x
; bind -c x x
; bind -a /sys/lib x
; ns|tail -4
mount -b '#s/rio.geoff.73279' /dev
bind -c /tmp/x /tmp/x
bind -a /sys/lib /tmp/x
cd /tmp
; >x/FOO
; unmount x; ls x
x/FOO

[-- Attachment #2: Type: message/rfc822, Size: 1984 bytes --]

From: "Russ Cox" <rsc@plan9.bell-labs.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] union directories
Date: Fri, 28 Feb 2003 13:56:18 -0500
Message-ID: <1f229e0b2b861ffeebd62738b74c9cd3@plan9.bell-labs.com>

You used to need to say "bind foo foo" to set up a mount
point for future union mounts.  (A long time ago.)
Now the system inserts these for you, though they're
still visible in the /proc/$pid/ns file (and thus the
output of ns).

Bind -bc 9 9 is a different thing, and almost
certainly not what is wanted, especially in the
example given -- it doesn't have any effect except
to make everything appear twice in the directory.
(Bind -bc 9 9 is like:

	bind 9 9
	bind -bc 9 9

hence everything appears twice.)

Russ

Re: [9fans] union directories

[Jack Johnson]

Russ Cox wrote:
> Bind -bc 9 9 is a different thing, and almost
> certainly not what is wanted

Aha!

It's a psychological mishap on my part.

I was thinking, "Create a union directory, add the writeable directory
to the beginning and the source directory to the end of the union
directory," rather than, "Unite the (new) writeable directory to the
beginning of the (old) source directory."

Which, in retrospect, makes the intent in the man page seem completely
obvious now.

Thanks!

-Jack

Re: [9fans] union directories

[Russ Cox]

You used to need to say "bind foo foo" to set up a mount
point for future union mounts.  (A long time ago.)
Now the system inserts these for you, though they're
still visible in the /proc/$pid/ns file (and thus the
output of ns).

Bind -bc 9 9 is a different thing, and almost
certainly not what is wanted, especially in the
example given -- it doesn't have any effect except
to make everything appear twice in the directory.
(Bind -bc 9 9 is like:

	bind 9 9
	bind -bc 9 9

hence everything appears twice.)

Russ

Re: [9fans] union directories

[Russ Cox]

there's no good way to do a recursive bind.
for the kernel you could piece it together with

t=/usr/you/9
bind -bc $t/pc /sys/src/9/pc
bind -bc $t/boot /sys/src/9/boot
cd /sys/src/9/pc
mk

[9fans] union directories

[Joel Salomon]

Jack Johnson wrote:

> mkdir 9
> bind -bc 9 9
> bind -ac /sys/src/9 9
> cd 9
> mk

What is the significance of bind -bc 9 9 ?
I've seen this bind foo foo elsewhere - what does it do?
--Joel

[9fans] union directories

[Jack Johnson]

Ronald G. Minnich wrote:
 > truthfully, how many of you out there started out using disk/kfscmd
 > allow when you wanted to modify the kernel source?

I'll admit it, but I'm still a newbie so I have an extension of this
question.

Is there any way to, say, recompile a source tree with subfolders
without pre-creating those subfolders in the union directory?  Or,
what's the correct way to do something like:

mkdir 9
bind -bc 9 9
bind -ac /sys/src/9 9
cd 9
mk

(though that's not what I'm doing, I'm just needed an easy example)

-Jack

Re: [9fans] union directories

[Douglas A. Gwyn]

rog@vitanuova.com wrote:
> if i type:
>         echo hello > /usr/rog/file
> i expect /usr/rog/file to be created if it isn't already there, but
> overwritten if it is.

What you expect is a strong function of what you are accustomed to.
Whether or not the above is desired behavior is about a 50-50
proposition, based on my own experience (with a lot of lost data).

My point about create is that it seems its primary function is to
write an entry into a directory.  Whether or not it should fail if
the entry is already there is debatable, but truncating the content
of the pointed-to file seems like a different function, indeed one
that is already available by open-for-writing.

I think the database people would support the position that the
basic actions need to be atomic, which pretty much means simple.

Re: [9fans] union directories

[rog]

> > This is all a legacy of Unix's model that create means two different
> > things in the two situations.
>
> Well, there you go -- an attempt to create an existing file should
> fail.  That is a simpler and more useful property.

it's not just a problem with the semantics of the create() system call,
but with the shell too.

if i type:

	echo hello > /usr/rog/file

i expect /usr/rog/file to be created if it isn't already there, but
overwritten if it is.

if create didn't overwrite an existing file, the shell would have to do
exactly as the kernel does currently, and for most intents and purposes
you'd still get the same "dual meaning".

i think under 9p2000 it's possible to call create() with an OEXCL
option and get the semantics you suggest.

  cheers,
    rog.

Re: [9fans] union directories

[Douglas A. Gwyn]

rob pike wrote:
> This is all a legacy of Unix's model that create means two different
> things in the two situations.

Well, there you go -- an attempt to create an existing file should
fail.  That is a simpler and more useful property.

Re: [9fans] union directories

[rob pike]

	The manual says "creation goes to the first element of the union that 
	permits creation". Doesn't that means that it shoud jump non_writable?.

Actually, the manual page bind(1) says,

	[-c] can be used in addition to any of the above to permit
	creation in a union directory.
	When a new file is created in a union directory,
	it is placed in the first element of the union that permits creation.

In this context, I don't feel it's ambiguous.

Bind(2) says,

	When a create system call (see open (2)) attempts to create in a union
	directory, and the file does not exist, the elements of the union are
	searched in order until one is found with MCREATE set.  The file is
	created in that directory; if that attempt fails, the create fails.

which is even clearer.

As to what it *should* do, this topic was one of the most discussed
in the early days of the system.  The current behavior obviously
reflects an easy implementation.  It's worked well in practice but
better designs may exist.

	Lookup should win whenever creation wins.

I'm not sure what this means.  There's no "winning" going on, and
lookup cannot work the same as creation if the file does not exist,
since there's nothing to look up.  If there is an existing file with that
name, in any element of the union, it is the one to which creation
will always apply.

This is all a legacy of Unix's model that create means two different
things in the two situations.

-rob

Re: [9fans] union directories

[Gorka Guardiola Muzquiz]

Russ> Actually that's the way it's supposed to work.
The manual says "creation goes to the first element of the union that 
permits creation". Doesn't that means that it should jump over non_writeble?
Even taking that in account, I can't see the logic behind this behaviour.
Lookup should win whenever creation wins. That would be symmetrical and
clean. with bind -ac n1 n2, bind -bc n1 n2, bind -ac n2 n1, bind -bc n2 n1
you would cover all the logical possibilities, and you save doing
bind -c n1 n2

Re: [9fans] union directories

[Gorka Guardiola Muzquiz]

>Actually that's the way it's supposed to work.
The manual says "creation goes to the first element of the union that 
permits creation". Doesn't that means that it shoud jump non_writable?.

Re: [9fans] union directories

[Gorka Guardiola Muzquiz]

Re: [9fans] union directories

[Russ Cox]

That's an unfortunate choice of words (in the manual).
"Permits creation" in that context means "was bound
into the union with -c" not "creates would succeed there
if tried".

Russ

Re: [9fans] union directories

[Russ Cox]

Actually that's the way it's supposed to work.

    Say you have two directories free_space and non_writable.
    If you write:

    bind -bc free_space non_writable 

    You add space to non_writable. Everything works fine. free_space is before
    non_writable in non_writable. If you create a file in non_writable you
    are actually creating int in free_space.

The union looks like

 * free_space
   non_writable

where the * means the directory was bound with -c.
Creates go to the topmost starred directory.

    bind -ac non_writable2 free_space2

    non_writable2 is after free_space2 in free_space2. You should be able to
    write in free_space. *you can't*. Even stranger:

Now you have

   free_space2
 * non_writable2

Creates go to non_writable2, but the directory
isn't writable.

    bind -ac fspace1 fspace2

    Any file in fspace2 supersedes any file in fspace1 with the same name,
    but if you create a file in fspace2 it is actually created *in fspace1*!!!???.
    fspace1 is *after* fspace2 for already existing files,
    but it is *before* when you create new ones.

Now you have

   fspace2
 * fspace1

Creates go to fspace1 but fspace2 wins for lookups.

It's just the way it's defined.  Note that if you did

bind -c fspace2 fspace2
bind -ac fspace1 fspace2

you'd have

 * fspace2
 * fspace1

so creates would go to fspace2 (the fact that fspace1
is starred is ignored), and fspace2 would win for lookups.

Russ

[9fans] union directories

[Gorka Guardiola Muzquiz]

I have encountered a behaviour in bind (well, actually in the fs) which
I find inconsistent. I may not be a bug, but it is counter intuitive.
Say you have two directories free_space and non_writable.
If you write:
bind -bc free_space non_writable 

You add space to non_writable. Everything works fine. free_space is before
non_writable in non_writable. If you create a file in non_writable you
are actually creating int in free_space.

bind -ac non_writable2 free_space2

non_writable2 is after free_space2 in free_space2. You should be able to
write in free_space. *you can't*. Even stranger:

bind -ac fspace1 fspace2

Any file in fspace2 supersedes any file in fspace1 with the same name,
but if you create a file in fspace2 it is actually created *in fspace1*!!!???.
fspace1 is *after* fspace2 for already existing files,
but it is *before* when you create new ones.

I find this weird and counter intuitive. It also seems to contradict
the manual page.  Is there any reason for this?. Is there something
I am not getting?.
-- 
                 Saludos,
                         Gorka

"Curiosity sKilled the cat"
-- 
    /"\
    \ /    ascii ribbon campaign - against html mail 
     X                           - against ms attachments
    / \