9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
* Re: [9fans] Python filesystem
@ 2001-11-29 23:07 geoff
  2001-11-29 23:26 ` George Michaelson
  0 siblings, 1 reply; 46+ messages in thread
From: geoff @ 2001-11-29 23:07 UTC (permalink / raw)
  To: 9fans

C News has the shell script that locks the news system and augments
PATH.  INN tends more to the monolithic end of the program-design
spectrum.

Version numbers in file names have some well known problems.  They
actually come from systems older than VMS, including at least
RS(u)X-11 and Tenex/TOPS-20, which takes us back about 30 years.  All
the implementations I've seen use a semi-colon to separate the real
file name from the version number.  Among other warts, this means you
have to quote them when referring to them from pretty much any shell:

	rm 'foo.c;1'
or
	rm 'foo.c;'*

In addition, the parsing of these names has traditionally been weird.
For example, foo.c means foo.c;0 (or foo.c;-1, I can't recall) which
means the most recent version of foo.c, perhaps foo.c;6.  I think the
other one (;-1 vs.  ;0) means the oldest version of foo.c still on
disk, perhaps foo.c;5.  Rob's and pjw's ``The Hideous Name'' paper
elaborates on these and other naming botches.

The reality is that you end up with a directory full of dreck rather
quickly, then the nightly PURGE command runs and clobbers all but the
most recent two or three versions (it's settable by your local
administrator and can be as low as one, if memory serves), so you
don't end up with a full history on disk, which is just as well
because you wouldn't be able to find anything.  Version numbers of
related files are not kept in synch, so you can't just

	cp *';6' /tmp/compile

to get a consistent snapshot.  File server dumps get this right:

; cd /n/dump/2001/1127/sys/src/cmd/disk/rkfs
; lc
all.h       dat.c       errno.h     ialloc.c    mkfile      sub.c
auth.c      dat.h       fcall.c     iobuf.c     portdat.h   uid.c
chk.c       dentry.c    fcallconv.c lib.h       portfns.h   utime.c
con.c       devmulti.c  fns.h       main.c      porttime.c
console.c   devwren.c   fs.c        misc.c      print.c
; mkdir /tmp/compile
; bind -bc /tmp/compile .
; mk
8c -FVw fs.c
8c -FVw sub.c
[...]
8c -FVw devwren.c
8l  -o 8.out fs.8 sub.8 porttime.8 fcall.8 iobuf.8 dat.8 main.8 dentry.8 fcallconv.8 ialloc.8 misc.8 con.8 console.8 chk.8 uid.8 auth.8 devwren.8

It's possible that Dan's comment about granularity referred to space,
not time.  The various source control systems just store diffs between
versions, but if you edit (and thus rewrite) a source file, the whole
file gets written to the dump.  Given the low cost of storage, that
seems quite acceptable.  Storing diffs to save space probably mattered
more in the mid-1970s when SCCS was written.



^ permalink raw reply	[flat|nested] 46+ messages in thread
* [9fans] Python filesystem
@ 2001-12-08 19:58 Doug McIlroy
  0 siblings, 0 replies; 46+ messages in thread
From: Doug McIlroy @ 2001-12-08 19:58 UTC (permalink / raw)
  To: rob, 9fans

I
I don't know what it has to do with Python, but
here's a little about the half-baked idea referenced in
the attached message.
In its simplest form, the idea was for the metadata
to be a single line of text, with all the metadata
in a file system unioned into a single file in which
each line begins with the (or a) name of the
associated data file.  What descriptors might
go in the metadata is left arbitrary.
To ask about a file, one would just look at the metadata.
To find a file given some descriptors, one could grep
the union file.
I never thought of an appealing way to maintain the
union file.  How, for example, can a useful name
be determined for a file, short of some ugly AI
like pwd?  What happens to the metadata when
a file gets recreated?  Assuming only one copy
of metadata exists for a file with two names, how
do you treat the metadata on unlink?
One can come up with answers to these questions, but
can one come up with a solution that will run fast
and be accurate?
Glimpse, for example, provides (a different sort of)
index to the contents of a file system by periodically
reading the whole file system.  Hence it cannot be
accurate.
The way metadata is stored matters, too.  Imagine
trying to grep the union of 10,000 files, were
each metadata item in a separate file.
If anybody thinks up a good way to overcome the
problems, I'd love to hear about it.

Doug McIlroy


>From: skipt@real.com
>Date: Thu Nov 29 14:50:26 EST 2001
>To: 9fans@cse.psu.edu
>Subject: Re: [9fans] Python filesystem
>
>In a thread long ago, Rob mentioned an idea proposed by Doug McIlroy for an
>indexed/annotated filesystem that would keep an annotation file for each
>regular file. It would seem like the right idea for keeping the mod
>documentation, etc.
>
>BTW, I've not been able to find any reference to the annotation/fs idea.
>Was anything written up (that could be shared)?
>
>>What do we lack then?  Locking and management of metadata?  There's
>>probably a way around those, as well.  The revision control FS isn't
>>well formed for saving; maybe a better solution would be to echo a
>>revision number into a ctl or new file, and then have that create a new
>>delta.  Write the file into it, and let the FS take care of picking out
>>the delta and storing it.  Perhaps a metadata file could be associated
>>with every file.  eg, foo.c;meta, foo.c;1.0, etc.
>


^ permalink raw reply	[flat|nested] 46+ messages in thread
* Re: [9fans] Python filesystem
@ 2001-11-30  0:11 geoff
  2001-11-30  3:10 ` William Josephson
  0 siblings, 1 reply; 46+ messages in thread
From: geoff @ 2001-11-30  0:11 UTC (permalink / raw)
  To: 9fans

An afterthought on Tenex-style version numbers: I don't believe any
implementations of file version numbers permit directories to have
versions, only plain files.  If directories had versions, and they
were kept around for a lot longer than a day, these systems would be
more usable (except for the added clutter of the directory versions)
and you probably wouldn't want nor need version numbers on plain
files.

Further, if the directory version number were incremented (and a new
directory created) only upon user request (ick), these systems would
be closer to the file server dumps:

	cd '.;3'
or
	cd '/sys/src/cmd;57'

though still rather nasty.  These systems cut the interaction of time
and the file system namespace one way, putting the
not-terribly-meaningful version information at the end using odd
syntax, which results in clutter between PURGEs and possibly after.
The file server dumps cut another way, with time-based version
information at the start, using uniform syntax, and gathering related
versions of files together (which minimises clutter), rather than
gathering related names (file;*) in a cluttered directory.

Imagine trying to go back and gather up related source files using
file version numbers as they currently stand.  You are in a cluttered
directory of files and versions.  You can't see far.  You get to pick
up files, examine them, put them down or in your backpack, and wander
around, looking at dates or diffing adjacent versions, trying to
figure out which files and versions form a consistent set, and keeping
a list as you go.  Let's see: it's fs.h;203, main.c;173, cfs.c;159,
9p.c;2, mkfile;17.  Bletch.

I just don't see the attraction.  You get ugly file names with odd
syntax, no convenient way to back up to a given time, and your
directories are always full of trash.  Such a deal.  Is there
*anything* that people find attractive about file version numbers?

I understand that people are proposing active file servers based on
Tenex-style version numbers, but I would propose to start from a solid
foundation rather than the shakey house of cards that is file version
numbers.



^ permalink raw reply	[flat|nested] 46+ messages in thread
* Re: [9fans] Python filesystem
@ 2001-11-29 20:01 Russ Cox
  2001-11-29 20:37 ` Dan Cross
  0 siblings, 1 reply; 46+ messages in thread
From: Russ Cox @ 2001-11-29 20:01 UTC (permalink / raw)
  To: 9fans

> cvshell?  I've noticed a trend in Plan 9 to write compressed word
> sequences using all letters, eg, cvsshell.

As opposed to all numbers?  I'm confused.
I suppose I could have written cvs.shell or cvs_shell
but that removes the ambiguity and that was most
of the fun.

> Now the question arises, is
> this a Freudian slip, or an intentional jab at CVS?  :-)

I ported CVS to Plan 9.  Does that make me more
or less likely to jab at it?  </unhelpful>

A constructive note: the biggest problem with making CVS
a file system is that it's not at all clear how to make
it present _useful_ views of the world.  The dump is nice
because the versioning is at the top of the tree, so that
each directory is internally consistent.  If you did the
same with CVS, I can easily see being overwhelemed with
needless detail in order to present consistent pictures.

Russ


^ permalink raw reply	[flat|nested] 46+ messages in thread
* Re: [9fans] Python filesystem
@ 2001-11-29 14:49 Russ Cox
  2001-11-29 19:28 ` Dan Cross
  0 siblings, 1 reply; 46+ messages in thread
From: Russ Cox @ 2001-11-29 14:49 UTC (permalink / raw)
  To: 9fans

#!/bin/rc

if(! ~ $#* 0){
	echo 'usage: cvshell' >[1=2]
	exit usage
}

while(echo -n 'cvs> '; r=`{read})
	rc -c 'cvs '^$r

sorry.  couldn't help myself.


^ permalink raw reply	[flat|nested] 46+ messages in thread
* Re: [9fans] Python filesystem
@ 2001-11-29 10:23 rog
  2001-11-29 10:54 ` Boyd Roberts
  0 siblings, 1 reply; 46+ messages in thread
From: rog @ 2001-11-29 10:23 UTC (permalink / raw)
  To: 9fans

> the meta anti-spam RFC 822 header parser version

cool! do they do these in Woolworth's?



^ permalink raw reply	[flat|nested] 46+ messages in thread
* Re: [9fans] Python filesystem
@ 2001-11-28 20:36 David Gordon Hogan
  0 siblings, 0 replies; 46+ messages in thread
From: David Gordon Hogan @ 2001-11-28 20:36 UTC (permalink / raw)
  To: 9fans

> I'm surprised more of the command line tools aren't daemonised actually.

Perl is frequently demonized.



^ permalink raw reply	[flat|nested] 46+ messages in thread
* Re: [9fans] Python filesystem
@ 2001-11-28 18:54 Russ Cox
  2001-11-28 19:09 ` Matt
  0 siblings, 1 reply; 46+ messages in thread
From: Russ Cox @ 2001-11-28 18:54 UTC (permalink / raw)
  To: 9fans

why not just post a connection to a real shell
in /srv and open it each time you get a request?
then you have a persistent environment.

if you were worried about concurrent access
then i guess i could see something like /net/cs
that just serves conversations, but otherwise
a file system seems like overkill.



^ permalink raw reply	[flat|nested] 46+ messages in thread
* Re: [9fans] Python filesystem
@ 2001-11-28 18:21 Russ Cox
  2001-11-28 18:34 ` Matt
  0 siblings, 1 reply; 46+ messages in thread
From: Russ Cox @ 2001-11-28 18:21 UTC (permalink / raw)
  To: 9fans

> python is not just a script interpreter but it is also a kind of shell
> fire it up without a script and you can execute commands 'interactively'
> which means it's quite a candidate for a file system

you lost me here.  rc is not a file system.

russ


^ permalink raw reply	[flat|nested] 46+ messages in thread
* [9fans] Python filesystem
@ 2001-11-28 18:18 Matt
  2001-11-30  5:44 ` Quinn Dunkan
  0 siblings, 1 reply; 46+ messages in thread
From: Matt @ 2001-11-28 18:18 UTC (permalink / raw)
  To: 9fans

I know the answer to this mail should be :

"send us the code when it's finished"

but I've been musing the past couple of days about a python filesystem

python is not just a script interpreter but it is also a kind of shell

fire it up without a script and you can execute commands 'interactively'

which means it's quite a candidate for a file system

is it a technique that is used anywhere else?

the ability to embed a programming language that an application can send code
to progressively sounds quite interesting

any thoughts?

M


^ permalink raw reply	[flat|nested] 46+ messages in thread

end of thread, other threads:[~2001-12-08 19:58 UTC | newest]

Thread overview: 46+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-11-29 23:07 [9fans] Python filesystem geoff
2001-11-29 23:26 ` George Michaelson
  -- strict thread matches above, loose matches on Subject: below --
2001-12-08 19:58 Doug McIlroy
2001-11-30  0:11 geoff
2001-11-30  3:10 ` William Josephson
2001-11-29 20:01 Russ Cox
2001-11-29 20:37 ` Dan Cross
2001-11-29 20:49   ` Boyd Roberts
2001-11-30 16:01     ` plan9
2001-12-01  4:44       ` Boyd Roberts
2001-11-29 14:49 Russ Cox
2001-11-29 19:28 ` Dan Cross
2001-11-29 20:02   ` Skip Tavakkolian
2001-11-29 20:05   ` Boyd Roberts
2001-11-29 10:23 rog
2001-11-29 10:54 ` Boyd Roberts
2001-11-28 20:36 David Gordon Hogan
2001-11-28 18:54 Russ Cox
2001-11-28 19:09 ` Matt
2001-11-28 21:46   ` Boyd Roberts
2001-11-29 12:24     ` Matt
2001-11-29  5:49   ` Lucio De Re
2001-11-29  6:30     ` Boyd Roberts
2001-11-29  6:31       ` George Michaelson
2001-11-29  7:10         ` Boyd Roberts
2001-11-29 11:26           ` Sam Holden
2001-12-06 16:56           ` Ralph Corderoy
2001-12-06 17:32             ` Boyd Roberts
2001-11-29 10:50       ` Lucio De Re
2001-11-29 11:06         ` Boyd Roberts
2001-12-06 15:59           ` Ralph Corderoy
2001-11-29  7:21     ` Skip Tavakkolian
2001-11-29  7:32       ` Steve Kilbane
2001-12-03 22:39         ` Laura Creighton
2001-12-07  9:36           ` Ralph Corderoy
2001-12-07 14:07             ` Laura Creighton
2001-11-29  7:37       ` Boyd Roberts
2001-11-29 11:10         ` Christopher Nielsen
2001-11-29 19:51         ` Skip Tavakkolian
2001-11-29 10:08     ` John Murdie
2001-11-29 10:37       ` Boyd Roberts
2001-11-29 12:03       ` Lucio De Re
2001-11-28 18:21 Russ Cox
2001-11-28 18:34 ` Matt
2001-11-28 18:18 Matt
2001-11-30  5:44 ` Quinn Dunkan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).