* [9fans] Multi-dimensional filesystem @ 2012-08-03 17:18 tlaronde 2012-08-03 18:58 ` Skip Tavakkolian ` (2 more replies) 0 siblings, 3 replies; 37+ messages in thread From: tlaronde @ 2012-08-03 17:18 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Hello, This is mainly a theoretical question. While playing with the representation of mathematical definitions as a file hierarchy (at dot you find a DESC or whatever named file with the description, and the subdirs are simply more restrictive instances of the thing; say : collection -> magma -> monoïde -> group etc.), it is soon obvious that a filesystem is a one dimension thing: you only follow one string. Multiple "parents" at the same level are not there. One could trick partly using hard or soft links. But with always the same problem: who is dot-dot, in a case where multiple parents are here? And multiple parents are not, to my knowledge, supported by kernel filesystem code. Manipulating the namespace is not the same. Has someone ever played with the notion of a multidimensional filesystem, where '/' is the origin, the nodes would be some representation of (a, b, c,...) (even negatives perhaps), each node having a name (user defined one by the way), and if a node is, say (3, 0, 1,...) this means that it is to be found as the third subdir of the (1, 0, 0,...) path etc., (In this scheme, if there is no link (no path) from another notion, it is another dimension). Just for intellectual curiosity. Best, -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-03 17:18 [9fans] Multi-dimensional filesystem tlaronde @ 2012-08-03 18:58 ` Skip Tavakkolian 2012-08-03 19:25 ` tlaronde 2012-08-03 21:08 ` Burton Samograd 2012-08-04 12:16 ` Nicolas Bercher 2 siblings, 1 reply; 37+ messages in thread From: Skip Tavakkolian @ 2012-08-03 18:58 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs if i understand correctly, this is one way it could be done (i think): * built a graph representing the structure * create a file server that given a graph and a root node, synthesizes a hierarchy, AND * on every walk to a node launches a copy of itself with the same graph but the new node as the root AND * mounts the newly launched copy of it self under that node (like exportfs). -Skip On Fri, Aug 3, 2012 at 10:18 AM, <tlaronde@polynum.com> wrote: > Hello, > > This is mainly a theoretical question. > > While playing with the representation of mathematical definitions as a > file hierarchy (at dot you find a DESC or whatever named file with the > description, and the subdirs are simply more restrictive instances of > the thing; say : collection -> magma -> monoïde -> group etc.), it is > soon obvious that a filesystem is a one dimension thing: you only follow > one string. Multiple "parents" at the same level are not there. > > One could trick partly using hard or soft links. But with always the > same problem: who is dot-dot, in a case where multiple parents are here? > And multiple parents are not, to my knowledge, supported by kernel > filesystem code. Manipulating the namespace is not the same. > > Has someone ever played with the notion of a multidimensional > filesystem, where '/' is the origin, the nodes would be some > representation of (a, b, c,...) (even negatives perhaps), each node > having a name (user defined one by the way), and if > a node is, say (3, 0, 1,...) this means that it is to be found as the > third subdir of the (1, 0, 0,...) path etc., (In this scheme, if there > is no link (no path) from another notion, it is another dimension). > > Just for intellectual curiosity. > > Best, > -- > Thierry Laronde <tlaronde +AT+ polynum +dot+ com> > http://www.kergis.com/ > Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-03 18:58 ` Skip Tavakkolian @ 2012-08-03 19:25 ` tlaronde 0 siblings, 0 replies; 37+ messages in thread From: tlaronde @ 2012-08-03 19:25 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Fri, Aug 03, 2012 at 11:58:08AM -0700, Skip Tavakkolian wrote: > if i understand correctly, this is one way it could be done (i think): > > * built a graph representing the structure > * create a file server that given a graph and a root node, synthesizes > a hierarchy, AND > * on every walk to a node launches a copy of itself with the same > graph but the new node as the root AND > * mounts the newly launched copy of it self under that node (like exportfs). Interesting. I guess the "children"/"parents" problem could be tricked by this translation (changing the origin on each node, and in fact presenting "parents" as one subdirs tree [reverse hierarchy] and the real "children" classically the other part (the negative could be precisely parents)). But in this case the equivalent of "cd .." on root would indeed goes in a subdir, unless root is absolute root... What is typical is that n 9P, walk takes whether a subdir or the previous, that is the ".." is really a local variable meaning "whatever was before" and not something hard encoded in the current file. The problem of implementing something efficient for the storage (not serving it) is another matter. The "lack" of links (whether hard or symbolic) on Plan9 could seem to suppress some facilities. But since this does not give all, even not a lot (this does not address multidimensional), it happens that it could be easier to implement something like this in a Plan9 world... Thanks! -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-03 17:18 [9fans] Multi-dimensional filesystem tlaronde 2012-08-03 18:58 ` Skip Tavakkolian @ 2012-08-03 21:08 ` Burton Samograd 2012-08-03 21:12 ` Kurt H Maier 2012-08-04 6:13 ` tlaronde 2012-08-04 12:16 ` Nicolas Bercher 2 siblings, 2 replies; 37+ messages in thread From: Burton Samograd @ 2012-08-03 21:08 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > Has someone ever played with the notion of a multidimensional filesystem David Korn did some research on a 3d file system called 3d: David G. Korn, Eduardo Krell, The 3-D File System, pp147-156, USENIX Conference Proceedings, Summer 1989, Baltimore, MD And also at behind a paywall: http://onlinelibrary.wiley.com/doi/10.1002/spe.4380201304/abstract I seem to remember it being available through ast: http://www2.research.att.com/sw/download/ -- Burton Samograd This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-03 21:08 ` Burton Samograd @ 2012-08-03 21:12 ` Kurt H Maier 2012-08-03 21:17 ` Burton Samograd 2012-08-04 6:13 ` tlaronde 1 sibling, 1 reply; 37+ messages in thread From: Kurt H Maier @ 2012-08-03 21:12 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Fri, Aug 03, 2012 at 05:08:52PM -0400, Burton Samograd wrote: > > This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page > > Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide. > Are you kidding ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-03 21:12 ` Kurt H Maier @ 2012-08-03 21:17 ` Burton Samograd 0 siblings, 0 replies; 37+ messages in thread From: Burton Samograd @ 2012-08-03 21:17 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs It's automatically added to my mails. Sorry, I forget about it because I don't add it, the mail server does. -----Original Message----- From: 9fans-bounces@9fans.net [mailto:9fans-bounces@9fans.net] On Behalf Of Kurt H Maier Sent: Friday, August 03, 2012 3:13 PM To: Fans of the OS Plan 9 from Bell Labs Subject: Re: [9fans] Multi-dimensional filesystem On Fri, Aug 03, 2012 at 05:08:52PM -0400, Burton Samograd wrote: > > This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page > > Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide. > Are you kidding This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-03 21:08 ` Burton Samograd 2012-08-03 21:12 ` Kurt H Maier @ 2012-08-04 6:13 ` tlaronde 2012-08-04 10:56 ` Aram Hăvărneanu 1 sibling, 1 reply; 37+ messages in thread From: tlaronde @ 2012-08-04 6:13 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Hello, On Fri, Aug 03, 2012 at 05:08:52PM -0400, Burton Samograd wrote: > > > Has someone ever played with the notion of a multidimensional filesystem > > David Korn did some research on a 3d file system called 3d: > > David G. Korn, Eduardo Krell, The 3-D File System, pp147-156, USENIX Conference Proceedings, Summer 1989, Baltimore, MD > > And also at behind a paywall: > > http://onlinelibrary.wiley.com/doi/10.1002/spe.4380201304/abstract > > I seem to remember it being available through ast: http://www2.research.att.com/sw/download/ Thank you for the reference! Best, -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-04 6:13 ` tlaronde @ 2012-08-04 10:56 ` Aram Hăvărneanu 2012-08-04 15:00 ` tlaronde 0 siblings, 1 reply; 37+ messages in thread From: Aram Hăvărneanu @ 2012-08-04 10:56 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs I'm pretty sure there's an isomorphism between your multidimensional filesystem and the set of potential namespaces. A dimension in your multidimensional fileysystem is just an arbitrary set of filesystems bounded in a particular namespace... -- Aram Hăvărneanu ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-04 10:56 ` Aram Hăvărneanu @ 2012-08-04 15:00 ` tlaronde 0 siblings, 0 replies; 37+ messages in thread From: tlaronde @ 2012-08-04 15:00 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sat, Aug 04, 2012 at 01:56:07PM +0300, Aram H?v?rneanu wrote: > I'm pretty sure there's an isomorphism between your multidimensional > filesystem and the set of potential namespaces. A dimension in your > multidimensional fileysystem is just an arbitrary set of filesystems > bounded in a particular namespace... I'm not quite sure about the bijective property but at least, since mathematics has recognized that the linear relationship is fundamental, and since base and dimensions are easily approached by linear Algebra, it seems probable enough that as long as the dot-dot thing is implemented by a fileserver---no semantics enforced at the kernel level---once you have the primitive for an oriented linearity (children/parent), multidimension is achievable. As sketched in a previous mail, a fileserver could present a "classical" file hierarchy, but with ".", "..", and, say ".+" for the hierarchical children, and ".-" for the reverse hierarchical parents (these being only a _view_; storage is another story...). So this could be achieved but by not attempting to provide an ubicuitous view, but only a local view from the file considered. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-03 17:18 [9fans] Multi-dimensional filesystem tlaronde 2012-08-03 18:58 ` Skip Tavakkolian 2012-08-03 21:08 ` Burton Samograd @ 2012-08-04 12:16 ` Nicolas Bercher 2012-08-04 15:20 ` tlaronde 2 siblings, 1 reply; 37+ messages in thread From: Nicolas Bercher @ 2012-08-04 12:16 UTC (permalink / raw) To: 9fans; +Cc: vincent douzal On 03/08/2012 19:18, tlaronde@polynum.com wrote: > Hello, > > This is mainly a theoretical question. > > While playing with the representation of mathematical definitions as > a file hierarchy (at dot you find a DESC or whatever named file with > the description, and the subdirs are simply more restrictive > instances of the thing; say : collection -> magma -> monoïde -> > group etc.), it is soon obvious that a filesystem is a one dimension > thing: you only follow one string. Multiple "parents" at the same > level are not there. > > One could trick partly using hard or soft links. But with always the > same problem: who is dot-dot, in a case where multiple parents are > here? And multiple parents are not, to my knowledge, supported by > kernel filesystem code. Manipulating the namespace is not the same. > > Has someone ever played with the notion of a multidimensional > filesystem, where '/' is the origin, the nodes would be some > representation of (a, b, c,...) (even negatives perhaps), each node > having a name (user defined one by the way), and if a node is, say > (3, 0, 1,...) this means that it is to be found as the third subdir > of the (1, 0, 0,...) path etc., (In this scheme, if there is no link > (no path) from another notion, it is another dimension). > > Just for intellectual curiosity. > > Best, Hi, As far as I understand, it seems you are interested in the idea of views over your files. Something that has been approached as "non-hierarchical file systems". But the complexity of handling such graphs often seems to be the reason why these projects failed. Not to mention how the pain to adapt them to the existing systems that are strictly hierarchical. In a project we presented in iwp9 6e (2011), we introduced the design of a filesystem that stores files (stream or hierarchy of files) in /records/ on a WORM file system (Venti). Records are stored in sequence, as they arrive (in respect to their recording order). Each record is identified by your username 'u', the repository name 'h' (for host) and an index related to time 't' (that is not time, but a self incremented interger linked to the date in the current calendar). Each triplet {u,h,t} is uniquely linked to a Venti SHA-1 score. That is the ground level. On top of that, you are free to point files to build any view on them and store these views again in new records, using any naming convention you like at this moment. A non intrusive hypertext language can be used to write a log book from were you point the files you stored in records. From this log book, you can do literate programming, run scripts on files, etc. (For the moment, Emacs org-mode seems to be a good hyper-text language to start from.) A third component, a "triplet" indexer, helps you to find which records points to other records, an vice versa. A plain text search engine helps you to retrieve text from the past. All is centered on traceability. We spent a lot of time studying reference bibliography from the 60s to today and already have plans for implementation of the ground level file system on top of Venti. Hope this will interest you, at least "just for intellectual curiosity"! ;-) Nicolas ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-04 12:16 ` Nicolas Bercher @ 2012-08-04 15:20 ` tlaronde 2012-08-05 15:29 ` Charles Forsyth 0 siblings, 1 reply; 37+ messages in thread From: tlaronde @ 2012-08-04 15:20 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sat, Aug 04, 2012 at 02:16:33PM +0200, Nicolas Bercher wrote: > > As far as I understand, it seems you are interested in the idea of views > over your files. Something that has been approached as "non-hierarchical > file systems". But the complexity of handling such graphs often seems to > be the reason why these projects failed. Not to mention how the pain to > adapt them to the existing systems that are strictly hierarchical. It seems clearer (and this is in accordance with notions expressed further in your message), that the file hierarchy is a view of the data. The way the data is stored can be totally different from the classical one dimension tree structure. The multidimension can (seen from far) be tricked with (as explained in another message), from the current file, a ".", "..", ".+" for children and ".-" for parents; keeping in mind that the paths are only the next neighbours (so this is a limited local view), as if say in a 2D grid (discrete coordinates) you will only consider the next objects with coordinates differing only by +/- 1. > > In a project we presented in iwp9 6e (2011), we introduced the design of > a filesystem that stores files (stream or hierarchy of files) in > /records/ on a WORM file system (Venti). Records are stored in sequence, > as they arrive (in respect to their recording order). Each record is > identified by your username 'u', the repository name 'h' (for host) and > an index related to time 't' (that is not time, but a self incremented > interger linked to the date in the current calendar). Each triplet > {u,h,t} is uniquely linked to a Venti SHA-1 score. That is the ground level. > > On top of that, you are free to point files to build any view on them > and store these views again in new records, using any naming convention > you like at this moment. A non intrusive hypertext language can be > used to write a log book from were you point the files you stored in > records. From this log book, you can do literate programming, run > scripts on files, etc. (For the moment, Emacs org-mode seems to be a > good hyper-text language to start from.) > > A third component, a "triplet" indexer, helps you to find which records > points to other records, an vice versa. A plain text search engine > helps you to retrieve text from the past. All is centered on traceability. > > We spent a lot of time studying reference bibliography from the 60s to > today and already have plans for implementation of the ground level > file system on top of Venti. > My "application case" was a little different. This was to present a database of mathematical definitions with a file hierarchy, the relationship between the notions being represented by the file hierarchy (but this obviously needs multidimension since not everything is on the same deducing string of notions). In this case, there will be some lookup function, taking a name or a more complex criteria, and returning a file with the definition represented by a n-uple (a, b, c, ...). The advantage is that the identifier can be of fixed length or by convention, if one more dimension is added, it is added last, and shorter uples have all zeroes coordinates for non expressed dimensions. But this encodes not only the definition search, but all the relationships since parent notions or descendant notions are immediately deduced from this. So there could be a WORM but with a typical copy-on-write block strategy for backup and history (but this is orthogonal to the multidimension). The implementation of the multidimension per se (as soon as there is a matrix representation, there is an obvious way to do). And the representation (view) of the data as a "normal" file structure with the ".+" and ".-". (This "explanation" does not hide the vagueness of the thing---but I'm just wandering "intellectually" at the moment). > Hope this will interest you, at least "just for intellectual curiosity"! > ;-) Yes ;-) -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-04 15:20 ` tlaronde @ 2012-08-05 15:29 ` Charles Forsyth 2012-08-05 17:36 ` tlaronde 0 siblings, 1 reply; 37+ messages in thread From: Charles Forsyth @ 2012-08-05 15:29 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 62 bytes --] There is also http://lsub.org/iwp9/cready/abheyshahplan9.pdf [-- Attachment #2: Type: text/html, Size: 139 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-05 15:29 ` Charles Forsyth @ 2012-08-05 17:36 ` tlaronde 2012-08-15 16:04 ` Eugene Gorodinsky 0 siblings, 1 reply; 37+ messages in thread From: tlaronde @ 2012-08-05 17:36 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sun, Aug 05, 2012 at 11:29:19AM -0400, Charles Forsyth wrote: > There is also http://lsub.org/iwp9/cready/abheyshahplan9.pdf Thank you! Indeed, this is related. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-05 17:36 ` tlaronde @ 2012-08-15 16:04 ` Eugene Gorodinsky 2012-08-15 17:33 ` tlaronde 0 siblings, 1 reply; 37+ messages in thread From: Eugene Gorodinsky @ 2012-08-15 16:04 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 661 bytes --] I'm sure I must not understand the problem fully and am confused because of that, but how is this idea of multidimensionality different from a relational filesystem approach such as befs ( http://www.nobius.org/~dbg/practical-file-system-design.pdf)? 2012/8/5 <tlaronde@polynum.com> > On Sun, Aug 05, 2012 at 11:29:19AM -0400, Charles Forsyth wrote: > > There is also http://lsub.org/iwp9/cready/abheyshahplan9.pdf > > Thank you! Indeed, this is related. > -- > Thierry Laronde <tlaronde +AT+ polynum +dot+ com> > http://www.kergis.com/ > Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C > > [-- Attachment #2: Type: text/html, Size: 1225 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-15 16:04 ` Eugene Gorodinsky @ 2012-08-15 17:33 ` tlaronde 2012-08-15 20:09 ` Bakul Shah 0 siblings, 1 reply; 37+ messages in thread From: tlaronde @ 2012-08-15 17:33 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, Aug 15, 2012 at 07:04:28PM +0300, Eugene Gorodinsky wrote: > I'm sure I must not understand the problem fully and am confused because of > that, but how is this idea of multidimensionality different from a > relational filesystem approach such as befs ( > http://www.nobius.org/~dbg/practical-file-system-design.pdf)? One can obviously mimick what I have described with a database, fields, the relationships being built by indexing. The hierarchical (but with multiple parents as well as multiple children) is a representation of the indexing in this case. So the BeFS could---from a cursory look about the combination of database, indexing and hierachy representation---offer a solution, except that the multiple parents, is not there (this could be, as well as with other systems, be done with .+ and .- presentation). The main differences from what I have in mind are: 1) There is no general relational database concept: the relationships of the "records" (files, that can have both a text content [the definition in my example] and be a directory node) is exclusively isomorphic to coordinates: (i, j, k, ...). 2) There is no constraint in the size of the "fields": the dimension can grow with time (no given dimensions coordinates being, by convention, equal to 0 to reach the actual dimension) ; there is no limit (except implementation one) for the size of an enumeration. 3) The hierarchy is the user interface, to view and to add the data: a new file (record) is added by placing it in the hierarchy; while in a relational database, the indexing is deduced from the actual records; here, so to say, the data is entered through the indexing. 4) And the problem was also thought through 9P: is there something in 9P that would prevent, at least theoritically, such a view of data to be presented? With the convention of ".+" and ".-", my answer is no: 9P has no hardcoded knowledge of ".." if I'm not mistaken. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-15 17:33 ` tlaronde @ 2012-08-15 20:09 ` Bakul Shah 2012-08-15 20:17 ` erik quanstrom 2012-08-15 21:27 ` tlaronde 0 siblings, 2 replies; 37+ messages in thread From: Bakul Shah @ 2012-08-15 20:09 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, 15 Aug 2012 19:33:27 +0200 tlaronde@polynum.com wrote: > > The main differences from what I have in mind are: > > 1) There is no general relational database concept: the relationships of > the "records" (files, that can have both a text content [the definition > in my example] and be a directory node) is exclusively isomorphic to > coordinates: (i, j, k, ...). Sounds like you want to represent a node with a directory, that has a "content" file that stores content associated with this node and a bunch of links to other nodes. You are most interested in parent/child relationships so for instance you can store links to all parents in a "parent" file and links to all children in "children" file. Not suggesting this is how you implement it; I am just trying to understand. A concrete example from you would help. > 2) There is no constraint in the size of the "fields": the dimension can > grow with time (no given dimensions coordinates being, by convention, > equal to 0 to reach the actual dimension) ; there is no limit (except > implementation one) for the size of an enumeration. > > 3) The hierarchy is the user interface, to view and to add the data: a > new file (record) is added by placing it in the hierarchy; while in a > relational database, the indexing is deduced from the actual records; > here, so to say, the data is entered through the indexing. > > 4) And the problem was also thought through 9P: is there something in 9P > that would prevent, at least theoritically, such a view of data to be > presented? With the convention of ".+" and ".-", my answer is no: 9P has > no hardcoded knowledge of ".." if I'm not mistaken. Note that in Unix ".." came for free since each dir stored the inode number (a "link" to an anonymous inode) along with a name. The scheme could represent any graph but the kernel restricted it to a tree by disallowing links to directories. Expanding this back to multiple parents in effect makes a poset (partially ordered set) but the unix trick or storing (parent-inode, "..") fails. Treating ".." as an /operator/ that cancels the previous component in a path also fails. You can perhaps extend a symlink object to store multiple links as I described above. The other difficulty is that something like 9p will only yield zero or one object as you traverse a path. How do you handle multiple parents? A more general idea is to consider a path component a "graph expression" component that is interpreted by the current node and may yield *any number of* new nodes. In other words x = open("a/b/c", mode) can yield a vector of file descriptors! Leaving component interpretation to the current node makes this a very dynamic and powerful system (for example, one can think of a node that maps to a list of network nodes -- so something like echo "date" > /net/my-nodes/foo chmod +x /net/my-nodes/foo /net/my-nodes/foo can run date on all my nodes! This would be a very compact expression of how to distribute data or work. Your original query is vague enough (& confusing, at least to me) that one can take it in a number of directions :-) ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-15 20:09 ` Bakul Shah @ 2012-08-15 20:17 ` erik quanstrom 2012-08-15 21:00 ` Bakul Shah 2012-08-15 21:27 ` tlaronde 1 sibling, 1 reply; 37+ messages in thread From: erik quanstrom @ 2012-08-15 20:17 UTC (permalink / raw) To: 9fans > x = open("a/b/c", mode) > > can yield a vector of file descriptors! Leaving component > interpretation to the current node makes this a very dynamic > and powerful system (for example, one can think of a node that > maps to a list of network nodes -- so something like > > echo "date" > /net/my-nodes/foo > chmod +x /net/my-nodes/foo > /net/my-nodes/foo one would think that a distributing file server would be a better abstraction as the normal tools could be brought to bear on the problem. - erik ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-15 20:17 ` erik quanstrom @ 2012-08-15 21:00 ` Bakul Shah 2012-08-16 1:38 ` erik quanstrom 0 siblings, 1 reply; 37+ messages in thread From: Bakul Shah @ 2012-08-15 21:00 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, 15 Aug 2012 16:17:28 EDT erik quanstrom <quanstro@labs.coraid.com> wrote: > > x = open("a/b/c", mode) > > > > can yield a vector of file descriptors! Leaving component > > interpretation to the current node makes this a very dynamic > > and powerful system (for example, one can think of a node that > > maps to a list of network nodes -- so something like > > > > echo "date" > /net/my-nodes/foo > > chmod +x /net/my-nodes/foo > > /net/my-nodes/foo > > one would think that a distributing file server would be > a better abstraction as the normal tools could be brought > to bear on the problem. I was just exploring the APLish nature of the idea. I used shell syntax to get the idea across -- here my-nodes would map to a set of fileserver nodes that interpret the remaining path so this path actually maps to a whole bunch of files. No idea if this is better, worse or even sensible. There are some graph languages that could possibly map here. What is a "distributing" file server? If you mean something like cxfs, luster, glusterfs, ceph, etc they wouldn't do the same thing. The above would in fact be kind of a distributed filesystem! ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-15 21:00 ` Bakul Shah @ 2012-08-16 1:38 ` erik quanstrom 2012-08-16 4:06 ` Bakul Shah 0 siblings, 1 reply; 37+ messages in thread From: erik quanstrom @ 2012-08-16 1:38 UTC (permalink / raw) To: 9fans > I was just exploring the APLish nature of the idea. I used > shell syntax to get the idea across -- here my-nodes would map > to a set of fileserver nodes that interpret the remaining path > so this path actually maps to a whole bunch of files. No idea > if this is better, worse or even sensible. There are some > graph languages that could possibly map here. > > What is a "distributing" file server? If you mean something > like cxfs, luster, glusterfs, ceph, etc they wouldn't do the > same thing. The above would in fact be kind of a distributed > filesystem! i was thinking of file server in the traditional (ahem) plan 9 sense, a network service that responds to 9p rather than a traditional (boring) unix-style store. and as such, i was thinking of a server that simply distributed requests among a set of servers. so that > > > echo "date" > /net/my-nodes/foo > > > chmod +x /net/my-nodes/foo would work with the normal tools on a normal kernel. all the distribution would be part of a purpose-built fs. - erik ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 1:38 ` erik quanstrom @ 2012-08-16 4:06 ` Bakul Shah 2012-08-16 13:45 ` erik quanstrom 0 siblings, 1 reply; 37+ messages in thread From: Bakul Shah @ 2012-08-16 4:06 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, 15 Aug 2012 21:38:43 EDT erik quanstrom <quanstro@quanstro.net> wrote: > > i was thinking of file server in the traditional (ahem) plan 9 sense, > a network service that responds to 9p rather than a traditional (boring) > unix-style store. To me the plan9 model is the "boring" one. Boringly simple. And I like it that way! Regardless, both can be used to distribute files. > and as such, i was thinking of a server that simply distributed requests > among a set of servers. so that > > > > > echo "date" > /net/my-nodes/foo > > > > chmod +x /net/my-nodes/foo > > would work with the normal tools on a normal kernel. all the distribution > would be part of a purpose-built fs. How would this work? Someone has to map "/net/my-nodes/foo" to a set of files. Either a program iterates over the list or you push this list processing into the "purpose built" filesystem. [Actually the latter is what I was thinking of -- A "9p*" protocol would return a list of file handles, and "rc*" would be an APLish version of rc] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 4:06 ` Bakul Shah @ 2012-08-16 13:45 ` erik quanstrom 0 siblings, 0 replies; 37+ messages in thread From: erik quanstrom @ 2012-08-16 13:45 UTC (permalink / raw) To: 9fans > > and as such, i was thinking of a server that simply distributed requests > > among a set of servers. so that > > > > > > > echo "date" > /net/my-nodes/foo > > > > > chmod +x /net/my-nodes/foo > > > > would work with the normal tools on a normal kernel. all the distribution > > would be part of a purpose-built fs. > > How would this work? Someone has to map "/net/my-nodes/foo" to > a set of files. Either a program iterates over the list or you > push this list processing into the "purpose built" filesystem. > [Actually the latter is what I was thinking of -- A "9p*" > protocol would return a list of file handles, and "rc*" would > be an APLish version of rc] i was also thinking of the latter. one could set up a set of parallel trees so that "date -n > /n/parallel/nodes/dev/date" could set the time do some gross level on a set of pre-selected nodes. selection could be fairly obvious, as in "echo create notes $machines>/n/parallel/ctl". - erik ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-15 20:09 ` Bakul Shah 2012-08-15 20:17 ` erik quanstrom @ 2012-08-15 21:27 ` tlaronde 2012-08-16 3:47 ` Bakul Shah 2012-08-16 17:02 ` Eugene Gorodinsky 1 sibling, 2 replies; 37+ messages in thread From: tlaronde @ 2012-08-15 21:27 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, Aug 15, 2012 at 01:09:49PM -0700, Bakul Shah wrote: > > Sounds like you want to represent a node with a directory, > that has a "content" file that stores content associated with > this node and a bunch of links to other nodes. You are most > interested in parent/child relationships so for instance you > can store links to all parents in a "parent" file and links to > all children in "children" file. Not suggesting this is how > you implement it; I am just trying to understand. A concrete > example from you would help. > I will give two (DeLuxe(R) version!) examples. First one, related to what I was wandering about, is mathematical definitions and relationships. Take the picture of the first volume of van der Waerden's Albebra (I have the german edition and will keep the german words). We speak about links between notions presented in a linear order: the order of the chapters: Galois theory chapter is the descendant of concepts from Vektorräume, Fortsetzung der Gruppentheorie and Körpertheorie chapters; Vektorräume and Gruppentheory being siblings ("children" of Ringe und Körper) while Körpertheorie is a grand-child of Vektorräume. Example of multiple parents, and not at the same level. Second example: field parcel (surveyor work). One can divide a parcel, leading to several children. But one can also reunite several adjacent parcels leading to one uniq new, all the inner borders being suppressed. This new parcel has several parents. And the "child" is bigger than every "parent" taken alone... > > > > 4) And the problem was also thought through 9P: is there something in 9P > > that would prevent, at least theoritically, such a view of data to be > > presented? With the convention of ".+" and ".-", my answer is no: 9P has > > no hardcoded knowledge of ".." if I'm not mistaken. > > Note that in Unix ".." came for free since each dir stored the > inode number (a "link" to an anonymous inode) along with a > name. The scheme could represent any graph but the kernel > restricted it to a tree by disallowing links to directories. > Expanding this back to multiple parents in effect makes a > poset (partially ordered set) but the unix trick or storing > (parent-inode, "..") fails. Treating ".." as an /operator/ > that cancels the previous component in a path also fails. Yes. > > You can perhaps extend a symlink object to store multiple > links as I described above. The other difficulty is that > something like 9p will only yield zero or one object as you > traverse a path. How do you handle multiple parents? > > A more general idea is to consider a path component a "graph > expression" component that is interpreted by the current node > and may yield *any number of* new nodes. > > In other words > > x = open("a/b/c", mode) > > can yield a vector of file descriptors! Leaving component > interpretation to the current node makes this a very dynamic > and powerful system (for example, one can think of a node that > maps to a list of network nodes -- so something like > > echo "date" > /net/my-nodes/foo > chmod +x /net/my-nodes/foo > /net/my-nodes/foo > > can run date on all my nodes! This would be a very compact > expression of how to distribute data or work. > > Your original query is vague enough (& confusing, at least to > me) that one can take it in a number of directions :-) I don't claim that my question was "definite" and it was open. So every idea "wandering" around this fuzzy concept is OK :-) But in my idea, there would be several distinct things (and I think plan9 has already split the things making alternative view possible): 1) There is the storage of the "things". This is mostly orthogonal---the trick being, as always, to be able to retrieve efficiently the stuff but really starting from the coordinates (i, j, k, ...). 2) In 9P, the ".." if I'm not mistaken is not here. One can request a file, or go back to the previous one. The ".." including "path removing" is not implemented in 9P if I read correctly the manpage. So a client can speak 9P, but the way the stuff are presented will be mainly in the server, the trick to allow existing clients to browse being this ".+" and ".-": ".+" leads to children; ".-" leads to parents and a DESC file being perhaps a text description, and some fake parenting allowing a request of ".." to trigger a special action from the server---but all these are artefacts of the server. 3) This is the server that offers a "view" of the data. And in this case, indeed, a request for "something" can trigger "bizarre" actions. >From a very cursory look, it seems to me that it could be simpler to implement this with Plan9, because it is relatively agnostic about what is a filesystem, than with an Unix if I take into account the difficulties Unix kernels have with userland filesystems---probably because a lot of semantics are assumed in the vnode subsystem. By conception, Plan9 makes a distinction between viewing the data (terminal), processing the data (CPU) and serving the data (fileserver). This is the case here, and this is not a surprise since these distinctions are quite natural---the terminal/CPU/fileserver being simply the special case where the three happen to be the very same machine. But this is a coincidence, not an obligation. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-15 21:27 ` tlaronde @ 2012-08-16 3:47 ` Bakul Shah 2012-08-16 5:34 ` tlaronde 2012-08-16 17:02 ` Eugene Gorodinsky 1 sibling, 1 reply; 37+ messages in thread From: Bakul Shah @ 2012-08-16 3:47 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, 15 Aug 2012 23:27:34 +0200 tlaronde@polynum.com wrote: > > I will give two (DeLuxe(R) version!) examples. Thanks. Sounds like you are taking about IS-A and HAS-A relationships. > 2) In 9P, the ".." if I'm not mistaken is not here. One can request a > file, or go back to the previous one. The ".." including "path removing" > is not implemented in 9P if I read correctly the manpage. So a client > can speak 9P, but the way the stuff are presented will be mainly in the > server, the trick to allow existing clients to browse being this ".+" > and ".-": ".+" leads to children; ".-" leads to parents and a DESC file > being perhaps a text description, and some fake parenting allowing a > request of ".." to trigger a special action from the server---but all > these are artefacts of the server. Looks like you are treating ".+" & ".-" specially *somewhere*. Are foo/.- and foo/bar treated differently? What would "foo/.-" even mean? What would open("foo/.-", mode) do? When I try to work out an example I don't get how you can implement this. May be you can use your special syntax to present an example or two as shell scripts? In any case since multiple relationships are possible, why single out a specific case? That is, it seems to me you can either use the current 9p as is and implement what you want by convention in your programs or extend the model in a major way (which is where I was going). > >From a very cursory look, it seems to me that it could be simpler to > implement this with Plan9, because it is relatively agnostic about what > is a filesystem, than with an Unix if I take into account the > difficulties Unix kernels have with userland filesystems---probably > because a lot of semantics are assumed in the vnode subsystem. You can use fusefs to implement a plan9 like synthetic filesystems. But compared to plan9 the unix model is indeed far more constrained and a lot more complicated -- you should see Linux's /sys synthetic filesystem -- it is also a good exmaple of what multiple inheritance wreak. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 3:47 ` Bakul Shah @ 2012-08-16 5:34 ` tlaronde 2012-08-16 13:40 ` erik quanstrom 0 siblings, 1 reply; 37+ messages in thread From: tlaronde @ 2012-08-16 5:34 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, Aug 15, 2012 at 08:47:47PM -0700, Bakul Shah wrote: > > > 2) In 9P, the ".." if I'm not mistaken is not here. One can request a > > file, or go back to the previous one. The ".." including "path removing" > > is not implemented in 9P if I read correctly the manpage. So a client > > can speak 9P, but the way the stuff are presented will be mainly in the > > server, the trick to allow existing clients to browse being this ".+" > > and ".-": ".+" leads to children; ".-" leads to parents and a DESC file > > being perhaps a text description, and some fake parenting allowing a > > request of ".." to trigger a special action from the server---but all > > these are artefacts of the server. > > Looks like you are treating ".+" & ".-" specially *somewhere*. > Are foo/.- and foo/bar treated differently? What would > "foo/.-" even mean? What would open("foo/.-", mode) do? They are treated specially by the server serving 9P requests. The aim would be to present a classical file hierarchy, even with special conventional names, implementing multiple parents, by convention, under .-/, and (classical) multiple children by .+/. .+/ holds the children (if the requested node has ones). .-/ holds the parents (in reverse order i.e. the nearest parents up in every dimension). ./* being the siblings of the node (siblings, whether files or directories appear by name not hidden under .+/ or .-/). What is more bizarre, with my scheme, is how to implement the meaning of ".."? If classical clients have to be able to be used, the server must create a fake name (as the penultimate component of the dirpath), that triggers the correct answer from the server. But the result is that going "up" means going down (from the view served) to the next hierarchy level in .-/. > > When I try to work out an example I don't get how you can > implement this. May be you can use your special syntax to > present an example or two as shell scripts? > > In any case since multiple relationships are possible, why > single out a specific case? That is, it seems to me you can > either use the current 9p as is and implement what you want by > convention in your programs or extend the model in a major way > (which is where I was going). The two approaches are not mutually exclusive. I mean one can implement using the current 9p (to see if it works, what are the limits, and the benefits...); and later tackle a more refined and perhaps more general answer the way you are describing. Plus, for a more broader view, I am---as I think a lot of users/programmers---not absolutely happy with relational databases. For example, for KerGIS, I have implemented, for attributes linked to geometry, a library, with a SQL backend (for use with PostgreSQL), a DBF (mainly for ESRI(R) Shape support), and a stdout and a stdin to be able to have a text format and simply pipe to utilities to save, fill or convert between databases. Mostly, PostgreSQL for a normal use (of KerGIS) is overkill. And the SQL type does not allow the kind of relationships I want---multiple parents are difficult, or rely on processing the data for some indexing; to have a list in one field, without specifying a length is not there etc. Having a file system organizing the relationships between the data, and implementing, one can say, the indexing by its very structure, and allowing to retrieve "records" or "fields" by the file system would be ideal (a small loss of time is not a problem). Not to say that once ownership, locking etc. is made by a fileserver, you have a distributed system for free---a lot of GIS softwares are using relational database even to store geometry! (a disaster from an efficiency standpoint when one is making geometrical manipulations---processing spaghettis and so on), simply because this relieves the developers/programmers from the distributed problems: they let the database server handles this; IMHO if databases are so pervasive nowadays, this is because of this. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 5:34 ` tlaronde @ 2012-08-16 13:40 ` erik quanstrom 2012-08-16 15:41 ` tlaronde 2012-08-16 15:59 ` Bakul Shah 0 siblings, 2 replies; 37+ messages in thread From: erik quanstrom @ 2012-08-16 13:40 UTC (permalink / raw) To: 9fans > What is more bizarre, with my scheme, is how to implement the meaning > of ".."? If classical clients have to be able to be used, the server > must create a fake name (as the penultimate component of the > dirpath), that triggers the correct answer from the server. see defmnt.c:/^fixdotdotname for where this is handled by the kernel, not the file server. - erik ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 13:40 ` erik quanstrom @ 2012-08-16 15:41 ` tlaronde 2012-08-16 16:06 ` erik quanstrom 2012-08-16 15:59 ` Bakul Shah 1 sibling, 1 reply; 37+ messages in thread From: tlaronde @ 2012-08-16 15:41 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, Aug 16, 2012 at 09:40:22AM -0400, erik quanstrom wrote: > > What is more bizarre, with my scheme, is how to implement the meaning > > of ".."? If classical clients have to be able to be used, the server > > must create a fake name (as the penultimate component of the > > dirpath), that triggers the correct answer from the server. > > see defmnt.c:/^fixdotdotname for where this is handled by the kernel, > not the file server. Well, I find cleanname (libc/port/cleanname.c) doing the .. dance with compression. But nothing in the kernel proper.. Except by allowing a new syntax .../{a,b,c,d}/foo meaning that foo has a, b, c and d as parents, the only way I see things working with the utilities and the ".." treatment is to insert a special name ensuring that a ".." suppression leading to this name will trigger the correct answer from the fileserver. I don't say that the fileserver has to treat directly the ".." in a path specification: the cleaning is made on the client side before the 9P request is sent. I mean that for this to work, the fileserver, from the first mount, will have to create special names (a uniq name, or the {a,b,c,d} lexeme if the user can have a chance to have a representation of the graphs that bear some meaning) so that a request of the content of this special name delivers the correct view of the data (if "classical" utilities are used to browse the file hierarchy). And by the way, since the parents a, b, c, d can be in several dimensions, and not having the same scalar value as a coordinate (`a' being direct child of /; `b' being deep in multiple directions etc., going up is not going up 1 in every dimension...), going to the "parents" would not mean anything, and the best to do would be to present a fake "middle" node where {a, b, c, d} appear in .-/ (parents), the children of {a,b,c,d} in .+/, and nothing in ./ (user to decide precisely where in a named node, he wants to go, standing in between for the moment). In a more general term, as projective geometry gives rules to represent a 3D objects on a 2D plane, is it possible to represent multiple dimensions in a single dimension string? The answer is yes, since this is what the n-uples coordinates representation is and since our linear languages "speak" about multidimensional features even when we have strictly no intuition about these "spaces". Can this be user friendly? No, if there are a lot of links, and long enumerations. But the naming of the nodes is always a problem (lengthy names, spaces, discrimining characters being put last and not first bringing identical long prefixes etc.). -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 15:41 ` tlaronde @ 2012-08-16 16:06 ` erik quanstrom 2012-08-16 16:28 ` tlaronde 0 siblings, 1 reply; 37+ messages in thread From: erik quanstrom @ 2012-08-16 16:06 UTC (permalink / raw) To: 9fans On Thu Aug 16 11:44:01 EDT 2012, tlaronde@polynum.com wrote: > On Thu, Aug 16, 2012 at 09:40:22AM -0400, erik quanstrom wrote: > > > What is more bizarre, with my scheme, is how to implement the meaning > > > of ".."? If classical clients have to be able to be used, the server > > > must create a fake name (as the penultimate component of the > > > dirpath), that triggers the correct answer from the server. > > > > see defmnt.c:/^fixdotdotname for where this is handled by the kernel, > > not the file server. > > Well, I find cleanname (libc/port/cleanname.c) doing the .. dance with > compression. But nothing in the kernel proper.. cleanname is called by the function i indicated in devmnt. > > Except by allowing a new syntax .../{a,b,c,d}/foo meaning that foo has > a, b, c and d as parents, the only way I see things working with the > utilities and the ".." treatment is to insert a special name ensuring > that a ".." suppression leading to this name will trigger the correct > answer from the fileserver. this is already the case due to union directories (as bakul points out), and the solution has been widely discussed. - erik ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 16:06 ` erik quanstrom @ 2012-08-16 16:28 ` tlaronde 0 siblings, 0 replies; 37+ messages in thread From: tlaronde @ 2012-08-16 16:28 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, Aug 16, 2012 at 12:06:40PM -0400, erik quanstrom wrote: > > > > > Except by allowing a new syntax .../{a,b,c,d}/foo meaning that foo has > > a, b, c and d as parents, the only way I see things working with the > > utilities and the ".." treatment is to insert a special name ensuring > > that a ".." suppression leading to this name will trigger the correct > > answer from the fileserver. > > this is already the case due to union directories (as bakul points out), > and the solution has been widely discussed. I gather. But here, there is an union on request (descending), and a split once you go up, since it is not the purpose to hide the "explication" of the union... -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 13:40 ` erik quanstrom 2012-08-16 15:41 ` tlaronde @ 2012-08-16 15:59 ` Bakul Shah 2012-08-16 16:31 ` tlaronde 2012-08-16 16:48 ` Burton Samograd 1 sibling, 2 replies; 37+ messages in thread From: Bakul Shah @ 2012-08-16 15:59 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, 16 Aug 2012 09:40:22 EDT erik quanstrom <quanstro@quanstro.net> wrote: > > What is more bizarre, with my scheme, is how to implement the meaning > > of ".."? If classical clients have to be able to be used, the server > > must create a fake name (as the penultimate component of the > > dirpath), that triggers the correct answer from the server. > > see defmnt.c:/^fixdotdotname for where this is handled by the kernel, > not the file server. It pretty much has to. Consider what happens when you do something like % x=`{pwd} % bind /sys/src tmp % cd tmp % cd .. This gets you back to $x. If you leave ".." upto the fileserver, you'd get back to /sys not $x. The server can't know the right context. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 15:59 ` Bakul Shah @ 2012-08-16 16:31 ` tlaronde 2012-08-16 16:48 ` Burton Samograd 1 sibling, 0 replies; 37+ messages in thread From: tlaronde @ 2012-08-16 16:31 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, Aug 16, 2012 at 08:59:01AM -0700, Bakul Shah wrote: > On Thu, 16 Aug 2012 09:40:22 EDT erik quanstrom <quanstro@quanstro.net> wrote: > > > What is more bizarre, with my scheme, is how to implement the meaning > > > of ".."? If classical clients have to be able to be used, the server > > > must create a fake name (as the penultimate component of the > > > dirpath), that triggers the correct answer from the server. > > > > see defmnt.c:/^fixdotdotname for where this is handled by the kernel, > > not the file server. > > It pretty much has to. Consider what happens when you do > something like > > % x=`{pwd} > % bind /sys/src tmp > % cd tmp > % cd .. > > This gets you back to $x. If you leave ".." upto the > fileserver, you'd get back to /sys not $x. The server can't > know the right context. And this is why the pathname has to encode, one way or the other, the hierarchy, to let the fileserver be context ignorant, and the user go wherever he wants. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 15:59 ` Bakul Shah 2012-08-16 16:31 ` tlaronde @ 2012-08-16 16:48 ` Burton Samograd 1 sibling, 0 replies; 37+ messages in thread From: Burton Samograd @ 2012-08-16 16:48 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > It pretty much has to. Consider what happens when you do something like > > % x=`{pwd} > % bind /sys/src tmp > % cd tmp > % cd .. > > This gets you back to $x. If you leave ".." upto the fileserver, you'd get back to /sys not $x. The server can't know the right context. I thought this problem was solved in plan9: Lexical File Names in Plan 9, or, Getting Dot-Dot Right - Rob Pike A vexing old problem solved: how to make pwd get the right answer in the face of multiply-bound directories. http://plan9.bell-labs.com/sys/doc/lexnames.html -- Burton Samograd This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-15 21:27 ` tlaronde 2012-08-16 3:47 ` Bakul Shah @ 2012-08-16 17:02 ` Eugene Gorodinsky 2012-08-16 19:48 ` tlaronde 1 sibling, 1 reply; 37+ messages in thread From: Eugene Gorodinsky @ 2012-08-16 17:02 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1519 bytes --] > > First one, related to what I was wandering about, is mathematical > definitions and relationships. Take the picture of the first volume of > van der Waerden's Albebra (I have the german edition and will keep the > german words). We speak about links between notions presented in a > linear order: the order of the chapters: > > Galois theory chapter is the descendant of concepts from > Vektorräume, Fortsetzung der Gruppentheorie and Körpertheorie chapters; > Vektorräume and Gruppentheory being siblings ("children" of Ringe und > Körper) while Körpertheorie is a grand-child of Vektorräume. > > Example of multiple parents, and not at the same level. > > This seems to be possible to recreate with befs, but I have to say: this particular problem is too specific to be solved with something like a filesystem. One should write a fileserver only when that would allow the user to use multiple familiar tools to work on the task. With your example it looks like the only thing the user would need here is the list of related topics along with the description of their relationship so that a related topic could be chosen and read. You can use a relational database for storage of the content along with the list of related topics and a program that can browse that database and follow the links. A user would never even think about doing something like 'cp ./Galors theory/parents ./Galors theory/children, even though the approach of implementing that in a fileserver allows that. [-- Attachment #2: Type: text/html, Size: 1722 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 17:02 ` Eugene Gorodinsky @ 2012-08-16 19:48 ` tlaronde 2012-08-16 20:11 ` erik quanstrom 0 siblings, 1 reply; 37+ messages in thread From: tlaronde @ 2012-08-16 19:48 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, Aug 16, 2012 at 08:02:52PM +0300, Eugene Gorodinsky wrote: > > > > First one, related to what I was wandering about, is mathematical > > definitions and relationships. Take the picture of the first volume of > > van der Waerden's Albebra (I have the german edition and will keep the > > german words). We speak about links between notions presented in a > > linear order: the order of the chapters: > > > > Galois theory chapter is the descendant of concepts from > > Vektorräume, Fortsetzung der Gruppentheorie and Körpertheorie chapters; > > Vektorräume and Gruppentheory being siblings ("children" of Ringe und > > Körper) while Körpertheorie is a grand-child of Vektorräume. > > > > Example of multiple parents, and not at the same level. > > > > This seems to be possible to recreate with befs, but I have to say: this > particular problem is too specific to be solved with something like a > filesystem. One should write a fileserver only when that would allow the > user to use multiple familiar tools to work on the task. With your example > it looks like the only thing the user would need here is the list of > related topics along with the description of their relationship so that a > related topic could be chosen and read. You can use a relational database > for storage of the content along with the list of related topics and a > program that can browse that database and follow the links. A user would > never even think about doing something like 'cp ./Galors theory/parents > ./Galors theory/children, even though the approach of implementing that in > a fileserver allows that. I think you missed the point. What I have given is an example, what indeed made me wonder, initially, about a way to simply store definitions as a text file, the relationships between the notions being described by a directory structure. It was obvious rapidly that this won't do in a classical hierarchical filesystem. Say it is a "thought experiment". I don't plan to write a fileserver just to implement this example. But multidimensional data is more widely needed than this example. I have indeed cases where a fileserver like that would allow storage of some data already stored in a database (served by a database SQL engine, for example), and for which the classical relational database is simply not a match. Files have the benefits that you can use the text tools on them to grep or whatever. And the file hierarchy reflects what is given by indexing in a relational database. Last, I always find it useful when I program or when I learn to ask me questions about "what if" (as long as I don't spend the bulk of time beating around the bush). One learns with exercices, questions, reading a book a pen in the hand, simply because this is the only way to leave a track in one's mind. Even dead ends are useful, since you know what is possible and what is not. And I may do strictly nothing, at the moment, with some multidimensional stuff, but use some of the solutions sketched, or something thought about passing, or part of a detail, to solve a non directly related problem later. There is a paper by D.E. Knuth, IIRC, speaking about the usefulness of "toy programming". This is it. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 19:48 ` tlaronde @ 2012-08-16 20:11 ` erik quanstrom 2012-08-17 6:48 ` tlaronde 0 siblings, 1 reply; 37+ messages in thread From: erik quanstrom @ 2012-08-16 20:11 UTC (permalink / raw) To: 9fans > I think you missed the point. What I have given is an example, what > indeed made me wonder, initially, about a way to simply store > definitions as a text file, the relationships between the notions > being described by a directory structure. It was obvious rapidly > that this won't do in a classical hierarchical filesystem. i think this can be done with a traditional file system as long as you don't insist that a file belong to exactly one directory. (that is messing with ".." is the hard way to go.) (note that plan 9 file servers already do this in a limited way since the dump file system allows an unchanging file or directory to be a member of as many /n/dump/yyyy/mmdd[.seq] heirarchies as there are dumps between changes.) - erik ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-16 20:11 ` erik quanstrom @ 2012-08-17 6:48 ` tlaronde 2012-08-17 7:48 ` Lucio De Re 0 siblings, 1 reply; 37+ messages in thread From: tlaronde @ 2012-08-17 6:48 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, Aug 16, 2012 at 04:11:11PM -0400, erik quanstrom wrote: > > I think you missed the point. What I have given is an example, what > > indeed made me wonder, initially, about a way to simply store > > definitions as a text file, the relationships between the notions > > being described by a directory structure. It was obvious rapidly > > that this won't do in a classical hierarchical filesystem. > > i think this can be done with a traditional file system as long as > you don't insist that a file belong to exactly one directory. (that is > messing with ".." is the hard way to go.) > > (note that plan 9 file servers already do this in a limited way since > the dump file system allows an unchanging file or directory to be > a member of as many /n/dump/yyyy/mmdd[.seq] heirarchies as there are > dumps between changes.) Yes, the multiplicity of an entry is possible on various systems with various means that are hard or soft links, unions, pointing on shared blocks etc. What is not here is to obtain "naturally" a view of all the relations of an entry, specially for the parents. The "getting dot-dot right" is precisely the problem that there are multiple paths, and only one is valid for a classical dir tree and you got to follow this one correct path back. Here, this multiplicity is simultaneously valid, and all the paths are the correct answer. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-17 6:48 ` tlaronde @ 2012-08-17 7:48 ` Lucio De Re 2012-08-20 15:17 ` tlaronde 0 siblings, 1 reply; 37+ messages in thread From: Lucio De Re @ 2012-08-17 7:48 UTC (permalink / raw) To: 9fans > The > "getting dot-dot right" is precisely the problem that there are > multiple paths, and only one is valid for a classical dir tree and > you got to follow this one correct path back. Here, this multiplicity > is simultaneously valid, and all the paths are the correct answer. The question here is how to enhance the Unix-like hierarchical directory to embrace multi-dimensionality. I have no doubt that the changes to 9P that will almost certainly be required for this will be fairly obvious and from there building synthetic fileservers will not be difficult. I have a feeling that assuming that Plan 9 already has the required silver bullet is a bad starting point. ++L ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Multi-dimensional filesystem 2012-08-17 7:48 ` Lucio De Re @ 2012-08-20 15:17 ` tlaronde 0 siblings, 0 replies; 37+ messages in thread From: tlaronde @ 2012-08-20 15:17 UTC (permalink / raw) To: lucio, Fans of the OS Plan 9 from Bell Labs On Fri, Aug 17, 2012 at 09:48:33AM +0200, Lucio De Re wrote: > > The question here is how to enhance the Unix-like hierarchical > directory to embrace multi-dimensionality. I have no doubt that the > changes to 9P that will almost certainly be required for this will be > fairly obvious and from there building synthetic fileservers will not > be difficult. I have a feeling that assuming that Plan 9 already has > the required silver bullet is a bad starting point. I don't assume that "everything is OK already" ;) But I assume what has been (re-)discovered by modern mathematics: that the linearity is fundamental, because it is the way we think. Mathematical language is sequential linear, can speak about multidimensional objects without being itself, lexicaly or syntactically, multidimensional (there have been attempts, by Frege, and even in some early computer languages). So: it is better to be sure that the core, kernel, whatever is agnostic about the dimensions (making no assumptions about, for this, dot-dot), than trying to inject multdimensional features in the core. And one can play with the idea without changing 9P at the moment, just to gather more informations. -- Thierry Laronde <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2012-08-20 15:17 UTC | newest] Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-08-03 17:18 [9fans] Multi-dimensional filesystem tlaronde 2012-08-03 18:58 ` Skip Tavakkolian 2012-08-03 19:25 ` tlaronde 2012-08-03 21:08 ` Burton Samograd 2012-08-03 21:12 ` Kurt H Maier 2012-08-03 21:17 ` Burton Samograd 2012-08-04 6:13 ` tlaronde 2012-08-04 10:56 ` Aram Hăvărneanu 2012-08-04 15:00 ` tlaronde 2012-08-04 12:16 ` Nicolas Bercher 2012-08-04 15:20 ` tlaronde 2012-08-05 15:29 ` Charles Forsyth 2012-08-05 17:36 ` tlaronde 2012-08-15 16:04 ` Eugene Gorodinsky 2012-08-15 17:33 ` tlaronde 2012-08-15 20:09 ` Bakul Shah 2012-08-15 20:17 ` erik quanstrom 2012-08-15 21:00 ` Bakul Shah 2012-08-16 1:38 ` erik quanstrom 2012-08-16 4:06 ` Bakul Shah 2012-08-16 13:45 ` erik quanstrom 2012-08-15 21:27 ` tlaronde 2012-08-16 3:47 ` Bakul Shah 2012-08-16 5:34 ` tlaronde 2012-08-16 13:40 ` erik quanstrom 2012-08-16 15:41 ` tlaronde 2012-08-16 16:06 ` erik quanstrom 2012-08-16 16:28 ` tlaronde 2012-08-16 15:59 ` Bakul Shah 2012-08-16 16:31 ` tlaronde 2012-08-16 16:48 ` Burton Samograd 2012-08-16 17:02 ` Eugene Gorodinsky 2012-08-16 19:48 ` tlaronde 2012-08-16 20:11 ` erik quanstrom 2012-08-17 6:48 ` tlaronde 2012-08-17 7:48 ` Lucio De Re 2012-08-20 15:17 ` tlaronde
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).