[9fans] Multi-dimensional filesystem

[9fans] Multi-dimensional filesystem

[tlaronde]

Hello,

This is mainly a theoretical question.

While playing with the representation of mathematical definitions as a
file hierarchy (at dot you find a DESC or whatever named file with the
description, and the subdirs are simply more restrictive instances of
the thing; say : collection -> magma -> monoïde -> group etc.), it is
soon obvious that a filesystem is a one dimension thing: you only follow
one string. Multiple "parents" at the same level are not there.

One could trick partly using hard or soft links. But with always the
same problem: who is dot-dot, in a case where multiple parents are here?
And multiple parents are not, to my knowledge, supported by kernel
filesystem code. Manipulating the namespace is not the same.

Has someone ever played with the notion of a multidimensional
filesystem, where '/' is the origin, the nodes would be some 
representation of (a, b, c,...) (even negatives perhaps), each node 
having a name (user defined one by the way), and if
a node is, say (3, 0, 1,...) this means that it is to be found as the
third subdir of the (1, 0, 0,...) path etc., (In this scheme, if there 
is no link (no path) from another notion, it is another dimension).

Just for intellectual curiosity.

Best,
-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Skip Tavakkolian]

if i understand correctly, this is one way it could be done (i think):

* built a graph representing the structure
* create a file server that given a graph and a root node, synthesizes
a hierarchy,  AND
* on every walk to a node launches a copy of itself with the same
graph but the new node as the root AND
* mounts the newly launched copy of it self under that node (like exportfs).

-Skip

On Fri, Aug 3, 2012 at 10:18 AM,  <tlaronde@polynum.com> wrote:
> Hello,
>
> This is mainly a theoretical question.
>
> While playing with the representation of mathematical definitions as a
> file hierarchy (at dot you find a DESC or whatever named file with the
> description, and the subdirs are simply more restrictive instances of
> the thing; say : collection -> magma -> monoïde -> group etc.), it is
> soon obvious that a filesystem is a one dimension thing: you only follow
> one string. Multiple "parents" at the same level are not there.
>
> One could trick partly using hard or soft links. But with always the
> same problem: who is dot-dot, in a case where multiple parents are here?
> And multiple parents are not, to my knowledge, supported by kernel
> filesystem code. Manipulating the namespace is not the same.
>
> Has someone ever played with the notion of a multidimensional
> filesystem, where '/' is the origin, the nodes would be some
> representation of (a, b, c,...) (even negatives perhaps), each node
> having a name (user defined one by the way), and if
> a node is, say (3, 0, 1,...) this means that it is to be found as the
> third subdir of the (1, 0, 0,...) path etc., (In this scheme, if there
> is no link (no path) from another notion, it is another dimension).
>
> Just for intellectual curiosity.
>
> Best,
> --
>         Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
>                       http://www.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Fri, Aug 03, 2012 at 11:58:08AM -0700, Skip Tavakkolian wrote:
> if i understand correctly, this is one way it could be done (i think):
>
> * built a graph representing the structure
> * create a file server that given a graph and a root node, synthesizes
> a hierarchy,  AND
> * on every walk to a node launches a copy of itself with the same
> graph but the new node as the root AND
> * mounts the newly launched copy of it self under that node (like exportfs).

Interesting. I guess the "children"/"parents" problem could be tricked by this
translation (changing the origin on each node, and in fact presenting
"parents" as one subdirs tree [reverse hierarchy] and the real "children"
classically the other part (the negative could be precisely parents)).
But in this case the equivalent of "cd .." on root would indeed goes in
a subdir, unless root is absolute root...

What is typical is that n 9P, walk takes whether a subdir or the
previous, that is the ".." is really a local variable meaning
"whatever was before" and not something hard encoded in the current
file.

The problem of implementing something efficient for the storage
(not serving it) is another matter.

The "lack" of links (whether hard or symbolic) on Plan9 could seem
to suppress some facilities. But since this does not give all, even not
a lot (this does not address multidimensional), it happens that it
could be easier to implement something like this in a Plan9 world...

Thanks!
--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Burton Samograd]

> Has someone ever played with the notion of a multidimensional filesystem

David Korn did some research on a 3d file system called 3d:

David G. Korn, Eduardo Krell, The 3-D File System, pp147-156, USENIX Conference Proceedings, Summer 1989, Baltimore, MD

And also  at behind a paywall:

http://onlinelibrary.wiley.com/doi/10.1002/spe.4380201304/abstract

I seem to remember it being available through ast: http://www2.research.att.com/sw/download/

--
Burton Samograd


This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page

Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide.

Re: [9fans] Multi-dimensional filesystem

[Kurt H Maier]

On Fri, Aug 03, 2012 at 05:08:52PM -0400, Burton Samograd wrote:
>
> This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page
>
> Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide.
>

Are you kidding

Re: [9fans] Multi-dimensional filesystem

[Burton Samograd]

It's automatically added to my mails.   Sorry, I forget about it because I don't add it, the mail server does.

-----Original Message-----
From: 9fans-bounces@9fans.net [mailto:9fans-bounces@9fans.net] On Behalf Of Kurt H Maier
Sent: Friday, August 03, 2012 3:13 PM
To: Fans of the OS Plan 9 from Bell Labs
Subject: Re: [9fans] Multi-dimensional filesystem

On Fri, Aug 03, 2012 at 05:08:52PM -0400, Burton Samograd wrote:
>
> This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page
>
> Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide.
>

Are you kidding


This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page

Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide.

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

Hello,

On Fri, Aug 03, 2012 at 05:08:52PM -0400, Burton Samograd wrote:
>
> > Has someone ever played with the notion of a multidimensional filesystem
>
> David Korn did some research on a 3d file system called 3d:
>
> David G. Korn, Eduardo Krell, The 3-D File System, pp147-156, USENIX Conference Proceedings, Summer 1989, Baltimore, MD
>
> And also  at behind a paywall:
>
> http://onlinelibrary.wiley.com/doi/10.1002/spe.4380201304/abstract
>
> I seem to remember it being available through ast: http://www2.research.att.com/sw/download/

Thank you for the reference!

Best,
--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Aram Hăvărneanu]

I'm pretty sure there's an isomorphism between your multidimensional
filesystem and the set of potential namespaces. A dimension in your
multidimensional fileysystem is just an arbitrary set of filesystems
bounded in a particular namespace...

-- 
Aram Hăvărneanu

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Sat, Aug 04, 2012 at 01:56:07PM +0300, Aram H?v?rneanu wrote:
> I'm pretty sure there's an isomorphism between your multidimensional
> filesystem and the set of potential namespaces. A dimension in your
> multidimensional fileysystem is just an arbitrary set of filesystems
> bounded in a particular namespace...

I'm not quite sure about the bijective property but at least, since
mathematics has recognized that the linear relationship is fundamental,
and since base and dimensions are easily approached by linear Algebra,
it seems probable enough that as long as the dot-dot thing is
implemented by a fileserver---no semantics enforced at the
kernel level---once you have the primitive for an oriented linearity
(children/parent), multidimension is achievable.

As sketched in a previous mail, a fileserver could present a "classical"
file hierarchy, but with ".", "..", and, say ".+" for the hierarchical
children, and ".-" for the reverse hierarchical parents (these being
only a _view_; storage is another story...). So this could be achieved
but by not attempting to provide an ubicuitous view, but only a local
view from the file considered.

--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Nicolas Bercher]

On 03/08/2012 19:18, tlaronde@polynum.com wrote:
> Hello,
>
> This is mainly a theoretical question.
>
> While playing with the representation of mathematical definitions as
> a file hierarchy (at dot you find a DESC or whatever named file with
> the description, and the subdirs are simply more restrictive
> instances of the thing; say : collection ->  magma ->  monoïde ->
> group etc.), it is soon obvious that a filesystem is a one dimension
> thing: you only follow one string. Multiple "parents" at the same
> level are not there.
>
> One could trick partly using hard or soft links. But with always the
> same problem: who is dot-dot, in a case where multiple parents are
> here? And multiple parents are not, to my knowledge, supported by
> kernel filesystem code. Manipulating the namespace is not the same.
>
> Has someone ever played with the notion of a multidimensional
> filesystem, where '/' is the origin, the nodes would be some
> representation of (a, b, c,...) (even negatives perhaps), each node
> having a name (user defined one by the way), and if a node is, say
> (3, 0, 1,...) this means that it is to be found as the third subdir
> of the (1, 0, 0,...) path etc., (In this scheme, if there is no link
> (no path) from another notion, it is another dimension).
>
> Just for intellectual curiosity.
>
> Best,

Hi,

As far as I understand, it seems you are interested in the idea of views
over your files. Something that has been approached as "non-hierarchical
file systems". But the complexity of handling such graphs often seems to
be the reason why these projects failed. Not to mention how the pain to
adapt them to the existing systems that are strictly hierarchical.

In a project we presented in iwp9 6e (2011), we introduced the design of
a filesystem that stores files (stream or hierarchy of files) in
/records/ on a WORM file system (Venti). Records are stored in sequence,
as they arrive (in respect to their recording order). Each record is
identified by your username 'u', the repository name 'h' (for host) and
an index related to time 't' (that is not time, but a self incremented
interger linked to the date in the current calendar). Each triplet
{u,h,t} is uniquely linked to a Venti SHA-1 score. That is the ground level.

On top of that, you are free to point files to build any view on them
and store these views again in new records, using any naming convention
you like at this moment. A non intrusive hypertext language can be
used to write a log book from were you point the files you stored in
records. From this log book, you can do literate programming, run
scripts on files, etc. (For the moment, Emacs org-mode seems to be a
good hyper-text language to start from.)

A third component, a "triplet" indexer, helps you to find which records
points to other records, an vice versa. A plain text search engine
helps you to retrieve text from the past. All is centered on traceability.

We spent a lot of time studying reference bibliography from the 60s to
today and already have plans for implementation of the ground level
file system on top of Venti.

Hope this will interest you, at least "just for intellectual curiosity"!
;-)

Nicolas

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Sat, Aug 04, 2012 at 02:16:33PM +0200, Nicolas Bercher wrote:
>
> As far as I understand, it seems you are interested in the idea of views
> over your files. Something that has been approached as "non-hierarchical
> file systems". But the complexity of handling such graphs often seems to
> be the reason why these projects failed. Not to mention how the pain to
> adapt them to the existing systems that are strictly hierarchical.

It seems clearer (and this is in accordance with notions expressed
further in your message), that the file hierarchy is a view of the data.
The way the data is stored can be totally different from the classical
one dimension tree structure.

The multidimension can (seen from far) be tricked with (as explained in
another message), from the current file, a ".", "..", ".+" for children
and ".-" for parents; keeping in mind that the paths are only the next
neighbours (so this is a limited local view), as if say in a 2D grid
(discrete coordinates) you will only consider the next objects with
coordinates differing only by +/- 1.

>
> In a project we presented in iwp9 6e (2011), we introduced the design of
> a filesystem that stores files (stream or hierarchy of files) in
> /records/ on a WORM file system (Venti). Records are stored in sequence,
> as they arrive (in respect to their recording order). Each record is
> identified by your username 'u', the repository name 'h' (for host) and
> an index related to time 't' (that is not time, but a self incremented
> interger linked to the date in the current calendar). Each triplet
> {u,h,t} is uniquely linked to a Venti SHA-1 score. That is the ground level.
>
> On top of that, you are free to point files to build any view on them
> and store these views again in new records, using any naming convention
> you like at this moment. A non intrusive hypertext language can be
> used to write a log book from were you point the files you stored in
> records. From this log book, you can do literate programming, run
> scripts on files, etc. (For the moment, Emacs org-mode seems to be a
> good hyper-text language to start from.)
>
> A third component, a "triplet" indexer, helps you to find which records
> points to other records, an vice versa. A plain text search engine
> helps you to retrieve text from the past. All is centered on traceability.
>
> We spent a lot of time studying reference bibliography from the 60s to
> today and already have plans for implementation of the ground level
> file system on top of Venti.
>

My "application case" was a little different. This was to present a
database of mathematical definitions with a file hierarchy, the
relationship between the notions being represented by the file hierarchy
(but this obviously needs multidimension since not everything is on the
same deducing string of notions).

In this case, there will be some lookup function, taking a name or a
more complex criteria, and returning a file with the definition
represented by a n-uple (a, b, c, ...). The advantage is that the
identifier can be of fixed length or by convention, if one more dimension is
added, it is added last, and shorter uples have all zeroes
coordinates for non expressed dimensions. But this encodes not only the
definition search, but all the relationships since parent notions or
descendant notions are immediately deduced from this.

So there could be a WORM but with a typical copy-on-write block strategy
for backup and history (but this is orthogonal to the multidimension).
The implementation of the multidimension per se (as soon as there is
a matrix representation, there is an obvious way to do). And the
representation (view) of the data as a "normal" file structure with
the ".+" and ".-".

(This "explanation" does not hide the vagueness of the thing---but I'm
just wandering "intellectually" at the moment).

> Hope this will interest you, at least "just for intellectual curiosity"!
> ;-)

Yes ;-)
--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 62 bytes --]

There is also http://lsub.org/iwp9/cready/abheyshahplan9.pdf

[-- Attachment #2: Type: text/html, Size: 139 bytes --]

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Sun, Aug 05, 2012 at 11:29:19AM -0400, Charles Forsyth wrote:
> There is also http://lsub.org/iwp9/cready/abheyshahplan9.pdf

Thank you! Indeed, this is related.
--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Eugene Gorodinsky]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

I'm sure I must not understand the problem fully and am confused because of
that, but how is this idea of multidimensionality different from a
relational filesystem approach such as befs (
http://www.nobius.org/~dbg/practical-file-system-design.pdf)?

2012/8/5 <tlaronde@polynum.com>

> On Sun, Aug 05, 2012 at 11:29:19AM -0400, Charles Forsyth wrote:
> > There is also http://lsub.org/iwp9/cready/abheyshahplan9.pdf
>
> Thank you! Indeed, this is related.
> --
>         Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
>                       http://www.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>
>

[-- Attachment #2: Type: text/html, Size: 1225 bytes --]

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Wed, Aug 15, 2012 at 07:04:28PM +0300, Eugene Gorodinsky wrote:
> I'm sure I must not understand the problem fully and am confused because of
> that, but how is this idea of multidimensionality different from a
> relational filesystem approach such as befs (
> http://www.nobius.org/~dbg/practical-file-system-design.pdf)?

One can obviously mimick what I have described with a database, fields,
the relationships being built by indexing.

The hierarchical (but with multiple parents as well as multiple
children) is a representation of the indexing in this case.

So the BeFS could---from a cursory look about the combination of
database, indexing and hierachy representation---offer a solution,
except that the multiple parents, is not there (this could be, as well
as with other systems, be done with .+ and .- presentation).

The main differences from what I have in mind are:

1) There is no general relational database concept: the relationships of
the "records" (files, that can have both a text content [the definition
in my example] and be a directory node) is exclusively isomorphic to
coordinates: (i, j, k, ...).

2) There is no constraint in the size of the "fields": the dimension can
grow with time (no given dimensions coordinates being, by convention,
equal to 0 to reach the actual dimension) ; there is no limit (except
implementation one) for the size of an enumeration.

3) The hierarchy is the user interface, to view and to add the data: a
new file (record) is added by placing it in the hierarchy; while in a
relational database, the indexing is deduced from the actual records;
here, so to say, the data is entered through the indexing.

4) And the problem was also thought through 9P: is there something in 9P
that would prevent, at least theoritically, such a view of data to be
presented? With the convention of ".+" and ".-", my answer is no: 9P has
no hardcoded knowledge of ".." if I'm not mistaken.

--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Bakul Shah]

On Wed, 15 Aug 2012 19:33:27 +0200 tlaronde@polynum.com  wrote:
>
> The main differences from what I have in mind are:
>
> 1) There is no general relational database concept: the relationships of
> the "records" (files, that can have both a text content [the definition
> in my example] and be a directory node) is exclusively isomorphic to
> coordinates: (i, j, k, ...).

Sounds like you want to represent a node with a directory,
that has a "content" file that stores content associated with
this node and a bunch of links to other nodes.  You are most
interested in parent/child relationships so for instance you
can store links to all parents in a "parent" file and links to
all children in "children" file. Not suggesting this is how
you implement it; I am just trying to understand.  A concrete
example from you would help.

> 2) There is no constraint in the size of the "fields": the dimension can
> grow with time (no given dimensions coordinates being, by convention,
> equal to 0 to reach the actual dimension) ; there is no limit (except
> implementation one) for the size of an enumeration.
>
> 3) The hierarchy is the user interface, to view and to add the data: a
> new file (record) is added by placing it in the hierarchy; while in a
> relational database, the indexing is deduced from the actual records;
> here, so to say, the data is entered through the indexing.
>
> 4) And the problem was also thought through 9P: is there something in 9P
> that would prevent, at least theoritically, such a view of data to be
> presented? With the convention of ".+" and ".-", my answer is no: 9P has
> no hardcoded knowledge of ".." if I'm not mistaken.

Note that in Unix ".." came for free since each dir stored the
inode number (a "link" to an anonymous inode) along with a
name.  The scheme could represent any graph but the kernel
restricted it to a tree by disallowing links to directories.
Expanding this back to multiple parents in effect makes a
poset (partially ordered set) but the unix trick or storing
(parent-inode, "..") fails. Treating ".." as an /operator/
that cancels the previous component in a path also fails.

You can perhaps extend a symlink object to store multiple
links as I described above. The other difficulty is that
something like 9p will only yield zero or one object as you
traverse a path. How do you handle multiple parents?

A more general idea is to consider a path component a "graph
expression" component that is interpreted by the current node
and may yield *any number of* new nodes.

In other words

	x = open("a/b/c", mode)

can yield a vector of file descriptors!  Leaving component
interpretation to the current node makes this a very dynamic
and powerful system (for example, one can think of a node that
maps to a list of network nodes -- so something like

    echo "date" > /net/my-nodes/foo
    chmod +x /net/my-nodes/foo
    /net/my-nodes/foo

can run date on all my nodes!  This would be a very compact
expression of how to distribute data or work.

Your original query is vague enough (& confusing, at least to
me) that one can take it in a number of directions :-)

Re: [9fans] Multi-dimensional filesystem

[erik quanstrom]

> 	x = open("a/b/c", mode)
>
> can yield a vector of file descriptors!  Leaving component
> interpretation to the current node makes this a very dynamic
> and powerful system (for example, one can think of a node that
> maps to a list of network nodes -- so something like
>
>     echo "date" > /net/my-nodes/foo
>     chmod +x /net/my-nodes/foo
>     /net/my-nodes/foo

one would think that a distributing file server would be
a better abstraction as the normal tools could be brought
to bear on the problem.

- erik

Re: [9fans] Multi-dimensional filesystem

[Bakul Shah]

On Wed, 15 Aug 2012 16:17:28 EDT erik quanstrom <quanstro@labs.coraid.com>  wrote:
> > 	x = open("a/b/c", mode)
> >
> > can yield a vector of file descriptors!  Leaving component
> > interpretation to the current node makes this a very dynamic
> > and powerful system (for example, one can think of a node that
> > maps to a list of network nodes -- so something like
> >
> >     echo "date" > /net/my-nodes/foo
> >     chmod +x /net/my-nodes/foo
> >     /net/my-nodes/foo
>
> one would think that a distributing file server would be
> a better abstraction as the normal tools could be brought
> to bear on the problem.

I was just exploring the APLish nature of the idea.  I used
shell syntax to get the idea across -- here my-nodes would map
to a set of fileserver nodes that interpret the remaining path
so this path actually maps to a whole bunch of files. No idea
if this is better, worse or even sensible.  There are some
graph languages that could possibly map here.

What is a "distributing" file server? If you mean something
like cxfs, luster, glusterfs, ceph, etc they wouldn't do the
same thing. The above would in fact be kind of a distributed
filesystem!

Re: [9fans] Multi-dimensional filesystem

[erik quanstrom]

> I was just exploring the APLish nature of the idea.  I used
> shell syntax to get the idea across -- here my-nodes would map
> to a set of fileserver nodes that interpret the remaining path
> so this path actually maps to a whole bunch of files. No idea
> if this is better, worse or even sensible.  There are some
> graph languages that could possibly map here.
>
> What is a "distributing" file server? If you mean something
> like cxfs, luster, glusterfs, ceph, etc they wouldn't do the
> same thing. The above would in fact be kind of a distributed
> filesystem!

i was thinking of file server in the traditional (ahem) plan 9 sense,
a network service that responds to 9p rather than a traditional (boring)
unix-style store.

and as such, i was thinking of a server that simply distributed requests
among a set of servers.  so that

> > >     echo "date" > /net/my-nodes/foo
> > >     chmod +x /net/my-nodes/foo

would work with the normal tools on a normal kernel.  all the distribution
would be part of a purpose-built fs.

- erik

Re: [9fans] Multi-dimensional filesystem

[Bakul Shah]

On Wed, 15 Aug 2012 21:38:43 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
>
> i was thinking of file server in the traditional (ahem) plan 9 sense,
> a network service that responds to 9p rather than a traditional (boring)
> unix-style store.

To me the plan9 model is the "boring" one. Boringly simple.
And I like it that way!  Regardless, both can be used to
distribute files.

> and as such, i was thinking of a server that simply distributed requests
> among a set of servers.  so that
>
> > > >     echo "date" > /net/my-nodes/foo
> > > >     chmod +x /net/my-nodes/foo
>
> would work with the normal tools on a normal kernel.  all the distribution
> would be part of a purpose-built fs.

How would this work? Someone has to map "/net/my-nodes/foo" to
a set of files. Either a program iterates over the list or you
push this list processing into the "purpose built" filesystem.
[Actually the latter is what I was thinking of -- A "9p*"
protocol would return a list of file handles, and "rc*" would
be an APLish version of rc]

Re: [9fans] Multi-dimensional filesystem

[erik quanstrom]

> > and as such, i was thinking of a server that simply distributed requests
> > among a set of servers.  so that
> >
> > > > >     echo "date" > /net/my-nodes/foo
> > > > >     chmod +x /net/my-nodes/foo
> >
> > would work with the normal tools on a normal kernel.  all the distribution
> > would be part of a purpose-built fs.
>
> How would this work? Someone has to map "/net/my-nodes/foo" to
> a set of files. Either a program iterates over the list or you
> push this list processing into the "purpose built" filesystem.
> [Actually the latter is what I was thinking of -- A "9p*"
> protocol would return a list of file handles, and "rc*" would
> be an APLish version of rc]

i was also thinking of the latter.  one could set up a set of parallel
trees so that "date -n > /n/parallel/nodes/dev/date" could set the
time do some gross level on a set of pre-selected nodes.  selection
could be fairly obvious, as in "echo create notes $machines>/n/parallel/ctl".

- erik

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Wed, Aug 15, 2012 at 01:09:49PM -0700, Bakul Shah wrote:
> 
> Sounds like you want to represent a node with a directory,
> that has a "content" file that stores content associated with
> this node and a bunch of links to other nodes.  You are most
> interested in parent/child relationships so for instance you
> can store links to all parents in a "parent" file and links to
> all children in "children" file. Not suggesting this is how
> you implement it; I am just trying to understand.  A concrete
> example from you would help.
> 

I will give two (DeLuxe(R) version!) examples.

First one, related to what I was wandering about, is mathematical
definitions and relationships. Take the picture of the first volume of
van der Waerden's Albebra (I have the german edition and will keep the
german words). We speak about links between notions presented in a
linear order: the order of the chapters:

Galois theory chapter is the descendant of concepts from
Vektorräume, Fortsetzung der Gruppentheorie and Körpertheorie chapters;
Vektorräume and Gruppentheory being siblings ("children" of Ringe und
Körper) while Körpertheorie is a grand-child of Vektorräume.

Example of multiple parents, and not at the same level.

Second example: field parcel (surveyor work). One can divide a parcel,
leading to several children. But one can also reunite several adjacent
parcels leading to one uniq new, all the inner borders being
suppressed. This new parcel has several parents. And the "child" is
bigger than every "parent" taken alone...

> > 
> > 4) And the problem was also thought through 9P: is there something in 9P
> > that would prevent, at least theoritically, such a view of data to be
> > presented? With the convention of ".+" and ".-", my answer is no: 9P has
> > no hardcoded knowledge of ".." if I'm not mistaken.
> 
> Note that in Unix ".." came for free since each dir stored the
> inode number (a "link" to an anonymous inode) along with a
> name.  The scheme could represent any graph but the kernel
> restricted it to a tree by disallowing links to directories.
> Expanding this back to multiple parents in effect makes a
> poset (partially ordered set) but the unix trick or storing
> (parent-inode, "..") fails. Treating ".." as an /operator/
> that cancels the previous component in a path also fails.

Yes.

> 
> You can perhaps extend a symlink object to store multiple
> links as I described above. The other difficulty is that
> something like 9p will only yield zero or one object as you
> traverse a path. How do you handle multiple parents?
> 
> A more general idea is to consider a path component a "graph
> expression" component that is interpreted by the current node
> and may yield *any number of* new nodes.
> 
> In other words
> 
> 	x = open("a/b/c", mode)
> 
> can yield a vector of file descriptors!  Leaving component
> interpretation to the current node makes this a very dynamic
> and powerful system (for example, one can think of a node that
> maps to a list of network nodes -- so something like
> 
>     echo "date" > /net/my-nodes/foo
>     chmod +x /net/my-nodes/foo
>     /net/my-nodes/foo
> 
> can run date on all my nodes!  This would be a very compact
> expression of how to distribute data or work.
> 
> Your original query is vague enough (& confusing, at least to
> me) that one can take it in a number of directions :-)

I don't claim that my question was "definite" and it was open. So every
idea "wandering" around this fuzzy concept is OK :-)

But in my idea, there would be several distinct things (and I think
plan9 has already split the things making alternative view possible):

1) There is the storage of the "things". This is mostly orthogonal---the
trick being, as always, to be able to retrieve efficiently the stuff but
really starting from the coordinates (i, j, k, ...).

2) In 9P, the ".." if I'm not mistaken is not here. One can request a
file, or go back to the previous one. The ".." including "path removing"
is not implemented in 9P if I read correctly the manpage. So a client
can speak 9P, but the way the stuff are presented will be mainly in the
server, the trick to allow existing clients to browse being this ".+"
and ".-": ".+" leads to children; ".-" leads to parents and a DESC file
being perhaps a text description, and some fake parenting allowing a
request of ".." to trigger a special action from the server---but all 
these are artefacts of the server.

3) This is the server that offers a "view" of the data. And in this
case, indeed, a request for "something" can trigger "bizarre" actions.

>From a very cursory look, it seems to me that it could be simpler to
implement this with Plan9, because it is relatively agnostic about what
is a filesystem, than with an Unix if I take into account the
difficulties Unix kernels have with userland filesystems---probably 
because a lot of semantics are assumed in the vnode subsystem.

By conception, Plan9 makes a distinction between viewing the data
(terminal), processing the data (CPU) and serving the data (fileserver).
This is the case here, and this is not a surprise since these
distinctions are quite natural---the terminal/CPU/fileserver being
simply the special case where the three happen to be the very same
machine. But this is a coincidence, not an obligation.

-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Bakul Shah]

On Wed, 15 Aug 2012 23:27:34 +0200 tlaronde@polynum.com  wrote:
>
> I will give two (DeLuxe(R) version!) examples.

Thanks. Sounds like you are taking about IS-A and HAS-A
relationships.

> 2) In 9P, the ".." if I'm not mistaken is not here. One can request a
> file, or go back to the previous one. The ".." including "path removing"
> is not implemented in 9P if I read correctly the manpage. So a client
> can speak 9P, but the way the stuff are presented will be mainly in the
> server, the trick to allow existing clients to browse being this ".+"
> and ".-": ".+" leads to children; ".-" leads to parents and a DESC file
> being perhaps a text description, and some fake parenting allowing a
> request of ".." to trigger a special action from the server---but all
> these are artefacts of the server.

Looks like you are treating ".+" & ".-" specially *somewhere*.
Are foo/.- and foo/bar treated differently?  What would
"foo/.-" even mean? What would open("foo/.-", mode) do?

When I try to work out an example I don't get how you can
implement this.  May be you can use your special syntax to
present an example or two as shell scripts?

In any case since multiple relationships are possible, why
single out a specific case? That is, it seems to me you can
either use the current 9p as is and implement what you want by
convention in your programs or extend the model in a major way
(which is where I was going).

> >From a very cursory look, it seems to me that it could be simpler to
> implement this with Plan9, because it is relatively agnostic about what
> is a filesystem, than with an Unix if I take into account the
> difficulties Unix kernels have with userland filesystems---probably
> because a lot of semantics are assumed in the vnode subsystem.

You can use fusefs to implement a plan9 like synthetic
filesystems. But compared to plan9 the unix model is indeed
far more constrained and a lot more complicated -- you should
see Linux's /sys synthetic filesystem -- it is also a good
exmaple of what multiple inheritance wreak.

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Wed, Aug 15, 2012 at 08:47:47PM -0700, Bakul Shah wrote:
>
> > 2) In 9P, the ".." if I'm not mistaken is not here. One can request a
> > file, or go back to the previous one. The ".." including "path removing"
> > is not implemented in 9P if I read correctly the manpage. So a client
> > can speak 9P, but the way the stuff are presented will be mainly in the
> > server, the trick to allow existing clients to browse being this ".+"
> > and ".-": ".+" leads to children; ".-" leads to parents and a DESC file
> > being perhaps a text description, and some fake parenting allowing a
> > request of ".." to trigger a special action from the server---but all
> > these are artefacts of the server.
>
> Looks like you are treating ".+" & ".-" specially *somewhere*.
> Are foo/.- and foo/bar treated differently?  What would
> "foo/.-" even mean? What would open("foo/.-", mode) do?

They are treated specially by the server serving 9P requests. The aim
would be to present a classical file hierarchy, even with special
conventional names, implementing multiple parents, by convention,
under .-/, and (classical) multiple children by .+/.

.+/ holds the children (if the requested node has ones).

.-/ holds the parents (in reverse order i.e. the nearest parents up in
every dimension).

./* being the siblings of the node (siblings, whether files or
directories appear by name not hidden under .+/ or .-/).

What is more bizarre, with my scheme, is how to implement the meaning
of ".."? If classical clients have to be able to be used, the server
must create a fake name (as the penultimate component of the
dirpath), that triggers the correct answer from the server.

But the result is that going "up" means going down (from the view
served) to the next hierarchy level in .-/.

>
> When I try to work out an example I don't get how you can
> implement this.  May be you can use your special syntax to
> present an example or two as shell scripts?
>
> In any case since multiple relationships are possible, why
> single out a specific case? That is, it seems to me you can
> either use the current 9p as is and implement what you want by
> convention in your programs or extend the model in a major way
> (which is where I was going).

The two approaches are not mutually exclusive. I mean one can
implement using the current 9p (to see if it works, what are the
limits, and the benefits...); and later tackle a more refined and
perhaps more general answer the way you are describing.

Plus, for a more broader view, I am---as I think a lot of
users/programmers---not absolutely happy with relational databases.
For example, for KerGIS, I have implemented, for attributes linked
to geometry, a library, with a SQL backend (for use with PostgreSQL),
a DBF (mainly for ESRI(R) Shape support), and a stdout and a stdin
to be able to have a text format and simply pipe to utilities to
save, fill or convert between databases.

Mostly, PostgreSQL for a normal use (of KerGIS) is overkill. And
the SQL type does not allow the kind of relationships I want---multiple
parents are difficult, or rely on processing the data for some indexing;
to have a list in one field, without specifying a length is not there
etc.

Having a file system organizing the relationships between
the data, and implementing, one can say, the indexing by its very
structure, and allowing to retrieve "records" or "fields" by the
file system would be ideal (a small loss of time is not a problem).

Not to say that once ownership, locking etc. is made by a fileserver,
you have a distributed system for free---a lot of GIS softwares
are using relational database even to store geometry! (a disaster
from an efficiency standpoint when one is making geometrical
manipulations---processing spaghettis and so on), simply because
this relieves the developers/programmers from the distributed
problems: they let the database server handles this; IMHO if
databases are so pervasive nowadays, this is because of this.

--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[erik quanstrom]

> What is more bizarre, with my scheme, is how to implement the meaning
> of ".."? If classical clients have to be able to be used, the server
> must create a fake name (as the penultimate component of the
> dirpath), that triggers the correct answer from the server.

see defmnt.c:/^fixdotdotname for where this is handled by the kernel,
not the file server.

- erik

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Thu, Aug 16, 2012 at 09:40:22AM -0400, erik quanstrom wrote:
> > What is more bizarre, with my scheme, is how to implement the meaning
> > of ".."? If classical clients have to be able to be used, the server
> > must create a fake name (as the penultimate component of the
> > dirpath), that triggers the correct answer from the server.
>
> see defmnt.c:/^fixdotdotname for where this is handled by the kernel,
> not the file server.

Well, I find cleanname (libc/port/cleanname.c) doing the .. dance with
compression. But nothing in the kernel proper..

Except by allowing a new syntax .../{a,b,c,d}/foo meaning that foo has
a, b, c and d as parents, the only way I see things working with the
utilities and the ".." treatment is to insert a special name ensuring
that a ".." suppression leading to this name will trigger the correct
answer from the fileserver.

I don't say that the fileserver has to treat directly the ".." in
a path specification: the cleaning is made on the client side before
the 9P request is sent. I mean that for this to work, the fileserver,
from the first mount, will have to create special names (a uniq
name, or the {a,b,c,d} lexeme if the user can have a chance to have
a representation of the graphs that bear some meaning) so that a
request of the content of this special name delivers the correct
view of the data (if "classical" utilities are used to browse the
file hierarchy).

And by the way, since the parents a, b, c, d can be in several
dimensions, and not having the same scalar value as a coordinate
(`a' being direct child of /; `b' being deep in multiple directions
etc., going up is not going up 1 in every dimension...), going to
the "parents" would not mean anything, and the best to do would be
to present a fake "middle" node where {a, b, c, d} appear in .-/
(parents), the children of {a,b,c,d} in .+/, and nothing in ./
(user to decide precisely where in a named node, he wants to go,
standing in between for the moment).

In a more general term, as projective geometry gives rules to
represent a 3D objects on a 2D plane, is it possible to represent
multiple dimensions in a single dimension string? The answer is
yes, since this is what the n-uples coordinates representation is and
since our linear languages "speak" about multidimensional features even
when we have strictly no intuition about these "spaces".
Can this be user friendly? No, if there are a lot of links, and
long enumerations. But the naming of the nodes is always a problem
(lengthy names, spaces, discrimining characters being put last and
not first bringing identical long prefixes etc.).

--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[erik quanstrom]

On Thu Aug 16 11:44:01 EDT 2012, tlaronde@polynum.com wrote:
> On Thu, Aug 16, 2012 at 09:40:22AM -0400, erik quanstrom wrote:
> > > What is more bizarre, with my scheme, is how to implement the meaning
> > > of ".."? If classical clients have to be able to be used, the server
> > > must create a fake name (as the penultimate component of the
> > > dirpath), that triggers the correct answer from the server.
> >
> > see defmnt.c:/^fixdotdotname for where this is handled by the kernel,
> > not the file server.
>
> Well, I find cleanname (libc/port/cleanname.c) doing the .. dance with
> compression. But nothing in the kernel proper..

cleanname is called by the function i indicated in devmnt.

>
> Except by allowing a new syntax .../{a,b,c,d}/foo meaning that foo has
> a, b, c and d as parents, the only way I see things working with the
> utilities and the ".." treatment is to insert a special name ensuring
> that a ".." suppression leading to this name will trigger the correct
> answer from the fileserver.

this is already the case due to union directories (as bakul points out),
and the solution has been widely discussed.

- erik

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Thu, Aug 16, 2012 at 12:06:40PM -0400, erik quanstrom wrote:
>
> >
> > Except by allowing a new syntax .../{a,b,c,d}/foo meaning that foo has
> > a, b, c and d as parents, the only way I see things working with the
> > utilities and the ".." treatment is to insert a special name ensuring
> > that a ".." suppression leading to this name will trigger the correct
> > answer from the fileserver.
>
> this is already the case due to union directories (as bakul points out),
> and the solution has been widely discussed.

I gather. But here, there is an union on request (descending), and a
split once you go up, since it is not the purpose to hide the
"explication" of the union...

--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Bakul Shah]

On Thu, 16 Aug 2012 09:40:22 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> > What is more bizarre, with my scheme, is how to implement the meaning
> > of ".."? If classical clients have to be able to be used, the server
> > must create a fake name (as the penultimate component of the
> > dirpath), that triggers the correct answer from the server.
>
> see defmnt.c:/^fixdotdotname for where this is handled by the kernel,
> not the file server.

It pretty much has to.  Consider what happens when you do
something like

% x=`{pwd}
% bind /sys/src tmp
% cd tmp
% cd ..

This gets you back to $x.  If you leave ".." upto the
fileserver, you'd get back to /sys not $x. The server can't
know the right context.

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Thu, Aug 16, 2012 at 08:59:01AM -0700, Bakul Shah wrote:
> On Thu, 16 Aug 2012 09:40:22 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> > > What is more bizarre, with my scheme, is how to implement the meaning
> > > of ".."? If classical clients have to be able to be used, the server
> > > must create a fake name (as the penultimate component of the
> > > dirpath), that triggers the correct answer from the server.
> >
> > see defmnt.c:/^fixdotdotname for where this is handled by the kernel,
> > not the file server.
>
> It pretty much has to.  Consider what happens when you do
> something like
>
> % x=`{pwd}
> % bind /sys/src tmp
> % cd tmp
> % cd ..
>
> This gets you back to $x.  If you leave ".." upto the
> fileserver, you'd get back to /sys not $x. The server can't
> know the right context.

And this is why the pathname has to encode, one way or the other, the
hierarchy, to let the fileserver be context ignorant, and the user
go wherever he wants.
--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Burton Samograd]

> It pretty much has to.  Consider what happens when you do something like
>
> % x=`{pwd}
> % bind /sys/src tmp
> % cd tmp
> % cd ..
>
> This gets you back to $x.  If you leave ".." upto the fileserver, you'd get back to /sys not $x. The server can't know the right context.

I thought this problem was solved in plan9:

Lexical File Names in Plan 9, or, Getting Dot-Dot Right -  Rob Pike

A vexing old problem solved: how to make pwd get the right answer in the face of multiply-bound directories.

http://plan9.bell-labs.com/sys/doc/lexnames.html

--
Burton Samograd

This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page

Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide.

Re: [9fans] Multi-dimensional filesystem

[Eugene Gorodinsky]

[-- Attachment #1: Type: text/plain, Size: 1519 bytes --]

>
> First one, related to what I was wandering about, is mathematical
> definitions and relationships. Take the picture of the first volume of
> van der Waerden's Albebra (I have the german edition and will keep the
> german words). We speak about links between notions presented in a
> linear order: the order of the chapters:
>
> Galois theory chapter is the descendant of concepts from
> Vektorräume, Fortsetzung der Gruppentheorie and Körpertheorie chapters;
> Vektorräume and Gruppentheory being siblings ("children" of Ringe und
> Körper) while Körpertheorie is a grand-child of Vektorräume.
>
> Example of multiple parents, and not at the same level.
>
> This seems to be possible to recreate with befs, but I have to say: this
particular problem is too specific to be solved with something like a
filesystem. One should write a fileserver only when that would allow the
user to use multiple familiar tools to work on the task. With your example
it looks like the only thing the user would need here is the list of
related topics along with the description of their relationship so that a
related topic could be chosen and read. You can use a relational database
for storage of the content along with the list of related topics and a
program that can browse that database and follow the links. A user would
never even think about doing something like 'cp ./Galors theory/parents
./Galors theory/children, even though the approach of implementing that in
a fileserver allows that.

[-- Attachment #2: Type: text/html, Size: 1722 bytes --]

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Thu, Aug 16, 2012 at 08:02:52PM +0300, Eugene Gorodinsky wrote:
> >
> > First one, related to what I was wandering about, is mathematical
> > definitions and relationships. Take the picture of the first volume of
> > van der Waerden's Albebra (I have the german edition and will keep the
> > german words). We speak about links between notions presented in a
> > linear order: the order of the chapters:
> >
> > Galois theory chapter is the descendant of concepts from
> > Vektorräume, Fortsetzung der Gruppentheorie and Körpertheorie chapters;
> > Vektorräume and Gruppentheory being siblings ("children" of Ringe und
> > Körper) while Körpertheorie is a grand-child of Vektorräume.
> >
> > Example of multiple parents, and not at the same level.
> >
> > This seems to be possible to recreate with befs, but I have to say: this
> particular problem is too specific to be solved with something like a
> filesystem. One should write a fileserver only when that would allow the
> user to use multiple familiar tools to work on the task. With your example
> it looks like the only thing the user would need here is the list of
> related topics along with the description of their relationship so that a
> related topic could be chosen and read. You can use a relational database
> for storage of the content along with the list of related topics and a
> program that can browse that database and follow the links. A user would
> never even think about doing something like 'cp ./Galors theory/parents
> ./Galors theory/children, even though the approach of implementing that in
> a fileserver allows that.

I think you missed the point. What I have given is an example, what
indeed made me wonder, initially, about a way to simply store
definitions as a text file, the relationships between the notions
being described by a directory structure. It was obvious rapidly
that this won't do in a classical hierarchical filesystem.

Say it is a "thought experiment". I don't plan to write a fileserver
just to implement this example. But multidimensional data is more widely
needed than this example. I have indeed cases where a fileserver like
that would allow storage of some data already stored in a database
(served by a database SQL engine, for example), and for which the
classical relational database is simply not a match.

Files have the benefits that you can use the text tools on them to grep
or whatever. And the file hierarchy reflects what is given by indexing
in a relational database.

Last, I always find it useful when I program or when I learn to ask me
questions about "what if" (as long as I don't spend the bulk of time
beating around the bush). One learns with exercices, questions, reading
a book a pen in the hand, simply because this is the only way to leave a
track in one's mind. Even dead ends are useful, since you know what is
possible and what is not. 

And I may do strictly nothing, at the moment, with some multidimensional
stuff, but use some of the solutions sketched, or something thought
about passing,  or part of a detail, to solve a non directly related 
problem later.

There is a paper by D.E. Knuth, IIRC, speaking about the usefulness of
"toy programming". This is it.

-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[erik quanstrom]

> I think you missed the point. What I have given is an example, what
> indeed made me wonder, initially, about a way to simply store
> definitions as a text file, the relationships between the notions
> being described by a directory structure. It was obvious rapidly
> that this won't do in a classical hierarchical filesystem.

i think this can be done with a traditional file system as long as
you don't insist that a file belong to exactly one directory.  (that is
messing with ".." is the hard way to go.)

(note that plan 9 file servers already do this in a limited way since
the dump file system allows an unchanging file or directory to be
a member of as many /n/dump/yyyy/mmdd[.seq] heirarchies as there are
dumps between changes.)

- erik

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Thu, Aug 16, 2012 at 04:11:11PM -0400, erik quanstrom wrote:
> > I think you missed the point. What I have given is an example, what
> > indeed made me wonder, initially, about a way to simply store
> > definitions as a text file, the relationships between the notions
> > being described by a directory structure. It was obvious rapidly
> > that this won't do in a classical hierarchical filesystem.
>
> i think this can be done with a traditional file system as long as
> you don't insist that a file belong to exactly one directory.  (that is
> messing with ".." is the hard way to go.)
>
> (note that plan 9 file servers already do this in a limited way since
> the dump file system allows an unchanging file or directory to be
> a member of as many /n/dump/yyyy/mmdd[.seq] heirarchies as there are
> dumps between changes.)

Yes, the multiplicity of an entry is possible on various systems
with various means that are hard or soft links, unions, pointing on
shared blocks etc. What is not here is to obtain "naturally" a view
of all the relations of an entry, specially for the parents. The
"getting dot-dot right" is precisely the problem that there are
multiple paths, and only one is valid for a classical dir tree and
you got to follow this one correct path back. Here, this multiplicity
is simultaneously valid, and all the paths are the correct answer.

--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Multi-dimensional filesystem

[Lucio De Re]

> The
> "getting dot-dot right" is precisely the problem that there are
> multiple paths, and only one is valid for a classical dir tree and
> you got to follow this one correct path back. Here, this multiplicity
> is simultaneously valid, and all the paths are the correct answer.

The question here is how to enhance the Unix-like hierarchical
directory to embrace multi-dimensionality.  I have no doubt that the
changes to 9P that will almost certainly be required for this will be
fairly obvious and from there building synthetic fileservers will not
be difficult.  I have a feeling that assuming that Plan 9 already has
the required silver bullet is a bad starting point.

++L

Re: [9fans] Multi-dimensional filesystem

[tlaronde]

On Fri, Aug 17, 2012 at 09:48:33AM +0200, Lucio De Re wrote:
>
> The question here is how to enhance the Unix-like hierarchical
> directory to embrace multi-dimensionality.  I have no doubt that the
> changes to 9P that will almost certainly be required for this will be
> fairly obvious and from there building synthetic fileservers will not
> be difficult.  I have a feeling that assuming that Plan 9 already has
> the required silver bullet is a bad starting point.

I don't assume that "everything is OK already" ;) But I assume what has
been (re-)discovered by modern mathematics: that the linearity is
fundamental, because it is the way we think. Mathematical language is
sequential linear, can speak about multidimensional objects without
being itself, lexicaly or syntactically, multidimensional (there have
been attempts, by Frege, and even in some early computer languages). So:
it is better to be sure that the core, kernel, whatever is agnostic
about the dimensions (making no assumptions about, for this, dot-dot),
than trying to inject multdimensional features in the core.

And one can play with the idea without changing 9P at the moment, just
to gather more informations.

--
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                      http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C