[9fans] some shell scripts

[9fans] some shell scripts

[Russ Cox]

First, a nice little clock for acme.
I "ported" this over to the Unix acme today
and realized I'd never posted it.  The ... around
the time is so that when you Sort it ends up at the top.

t23=; cat plan9/bin/rc/Clock
#!/bin/rc
cd /mnt/acme/new
while (date | sed 's/.* (..:..).*/name ...\1.../' >ctl)
	sleep 60

t23=; cat bin/Clock
#!/usr/local/plan9/bin/rc
n=`{9p read acme/new/ctl}
echo $n
n=$n(1)
while (date | 9sed 's/.* (..:..).*/name ...\1.../' | 9p write acme/$n/ctl)
	sleep 60
t23=; 

Second, inspired by reading about Nemo's tags,
I realized that, on Unix, I use locate | grep way too much
and I'm sick of typing really long path names.
This script L prints all names on the system 
matching the argument except that / can expand
to any number of elements.  So:

t23=; L plan9/acme
/usr/local/plan9/acid/acme
/usr/local/plan9/bin/acme
/usr/local/plan9/src/cmd/acme
t23=; L plan9/download.c
/usr/local/plan9/src/cmd/postscript/download/download.c
t23=; 

Plan 9 users will have to substitute locate with something
like du -a / in cron to update /tmp/locate-slash.

t23=; cat bin/L
#!/usr/local/plan9/bin/rc

TMP=/tmp

if(~ $#* 0){
	echo 'usage: L pth ...' >[1=2]
	exit usage
}

fn dogrep {
	if(~ $#* 1)
		grep /$1
	if not{
		x=$1
		shift
		grep /$x | dogrep $*
	}
}

# it is noticeably faster (.09s vs .55s) to just
# save the raw locate database somewhere [sic]
if(! test -f $TMP/locate-slash
|| test `{mtime $TMP/locate-slash | awk '{print $1}'} -lt `{hoc -e
`{$PLAN9/bin/date -n}^-86400})
	locate / >$TMP/locate-slash

nl='
'
for(i){
	ifs=$nl { pattern=`{echo $i | 9sed 's;\*;[^/]*;g; s;/;/(.*/)?;g;
s;^;(.*/)?;; s;$;$;'} }
	egrep $pattern $TMP/locate-slash
}
t23=;

and then the obvious next step is BL:

t23=; cat bin/BL
#!/usr/local/plan9/bin/rc

B `{L $*}
t23=;

Re: [9fans] some shell scripts

[Martin C.Atkins]

Hi,

On Wed, 13 Oct 2004 18:38:15 -0400 Russ Cox <russcox@gmail.com> wrote:
>...
> Second, inspired by reading about Nemo's tags,
> I realized that, on Unix, I use locate | grep way too much
> and I'm sick of typing really long path names.

Reading about Nemo's tags, and Russ' L, I wonder what people here think
about Beagle? (see:
	http://www.gnome.org/projects/beagle/
)

I saw a demo a couple of weeks ago, and it was quite impressive.
Especially impressive was the fact that search results were updated
dynamically (within ~1/2s) when (matching) files were
created/destroyed.

This pretty-much requires a mechanism for the kernel to tell a user
process about changes to the filesystem, which sounds horrible, but
seems to work (they use inotify. Windows, apparently, has had such a
mechanism for years). Obviously, one does not want this to recurse :-(

What would be the "Plan 9 way" of achieving this? I can't see how
binding a fileserver on / (or even on my home directory) could have
reasonable performance, even if it could be made to have the right
semantics! Running a full filesystem scan every 0.1s would not be
very attractive! :-)

Note that Beagle searches more than just the filesystem, but that's
another story. All in all, Beagle-like facilities do look rather
attractive - it's sort of a super-google (i.e. uses google to search
the web. It can also search all/only the webpages that you have
visited - now that's something I really need :-)!

Martin
-- 
Martin C. Atkins				martin@parvat.com
Parvat Infotech (Private) Limited		http://www.parvat.com

Re: [9fans] some shell scripts

[geoff]

I think performance could be acceptable: I run lnfs over both file
systems (main and other) on my old ken file server and it doesn't seem
to slow things noticeably, and this should be similar: the file server
would just pass requests through, taking note of creations and
removals and updating its database.  It sounds pretty straightforward
actually.

Re: [9fans] some shell scripts

[Fco. J. Ballesteros]

> This pretty-much requires a mechanism for the kernel to tell a user
> process about changes to the filesystem, which sounds horrible, but

One of the things in my todo is doing something to notify changes
in the fs. It could be either a change in fossil or a wrapper fs to notify
events. Somehow I managed to skip this issue so far.

Re: [9fans] some shell scripts

[Martin C.Atkins]

OK - I'm glad it isn't so hard! :-)

Martin
-- 
Martin C. Atkins			martin_ml@parvat.com
Parvat Infotech Private Limited		http://www.parvat.com{/,/martin}

Re: [9fans] some shell scripts

[Nigel Roles]

geoff@collyer.net wrote:

>I think performance could be acceptable: I run lnfs over both file
>systems (main and other) on my old ken file server and it doesn't seem
>to slow things noticeably, and this should be similar: the file server
>would just pass requests through, taking note of creations and
>removals and updating its database.  It sounds pretty straightforward
>actually.
>
>  
>
I think you're right about the load implied by lnfs. You only need to 
check the
.longnames file when opening or closing a file. This works with the lnfs 
running
on the workstation and is therefore kenfs comaptible.

Surely for watching files, you want to know if other clients connected 
to the fileserver
have changed things? Wouldn't this require a client to update a database 
on every change
in case another client is interested in the changes it has made to a 
particular
file?

This doesn't sound too different from simply statting every file you are 
interested
in every second, which Martin reasonably suggested might be a little 
heavyweight
when trying to build indexes for a large proportion of files on a file 
system.

The bright new shiny inotify device in Linux (note it's a device so 
accessibility over a
network might be a problem) sources events when inodes are updated and copes
with dismounts, which is attractive with so many hot-pluggable-widgets these
days. Within the system-centric Unix world, this is relatively tidy 
compared to
SIGIO.

Seems like it might need some kenfs/fossil hacks for the Plan 9 world.

Re: [9fans] some shell scripts

[geoff]

One could start lnfs early in cpurc and termrc, have it post a /srv
file, and have /lib/namespace* mount that /srv file.  I think that for
the sort of thing you described (with Beagle), creations and removals
are of primary interest, though one could watch for writes too.
Keeping a database or log of such changes (perhaps served from the
watcher file server, permitting some in-memory caching) is vastly
cheaper than statting a potentially huge set of files every second
(though the difference would be even bigger on (l)unix, with the
directory entries separated from the i-nodes).  One should be able to
just note the new file name, size and perhaps mtime of each write, so
the volume of data should be tolerable.  If you're building indices, a
watcher client would probably want to wait for a given file to be
closed and perhaps wait a little longer in case it's rewritten, then
index it.  Maybe index every changed file each hour, or something of
the sort.

I don't think the need kernel mods for this.

Re: [9fans] some shell scripts

[Russ Cox]

> I don't think the need kernel mods for this.

I think it makes a lot more sense to do it on the file server though.
Have some sort of changes file that you can block reading.
Any local solution fails to account for changes by others.

Russ

Re: [9fans] some shell scripts

[geoff]

I guess I don't see how it fails to account for changes by others.  If
you connect to a file server, you're either running its cpurc or
termrc (well, barring the case of cross-authentication-domain mounts)
or connecting to processes that run its /lib/namespace* files (e.g.,
drawterm).

Thinking about it some more, on a busy file server, you're probably
going to want to limit the volume of data you have to read to see the
changes to the files you care about, which suggests a watcher process
feeding each client only the changes they care about.