Re: [9fans] blanks in file names

Re: [9fans] blanks in file names

[Sam]

Yeah, 5, 7, 5, iirc.

Sam

On Fri, 5 Jul 2002, Andrew Simmons wrote:

> > Change space! Change the file delimiter!
> > The shell will never recover.  The system will break.
> > I will mourn.
>
> Isn't a haiku supposed to have 17 syllables?
>

Re: [9fans] blanks in file names

[Sam]

Unless that was an attempt at levity, in which case I've just been
caught responding to a thread I haven't been following.

heh,

sam

On Thu,
4 Jul 2002, Sam wrote:

> Yeah, 5, 7, 5, iirc.
>
> Sam
>
> On Fri, 5 Jul 2002, Andrew Simmons wrote:
>
> > > Change space! Change the file delimiter!
> > > The shell will never recover.  The system will break.
> > > I will mourn.
> >
> > Isn't a haiku supposed to have 17 syllables?
> >
>

Re: [9fans] blanks in file names

[Andrew Simmons]

> Change space! Change the file delimiter!
> The shell will never recover.  The system will break.
> I will mourn.

Isn't a haiku supposed to have 17 syllables?

Re: [9fans] blanks in file names

[ozan s yigit]

> Isn't a haiku supposed to have 17 syllables?

if you are writing in english, not necessarily. yes if you are writing in
japanese. in that case, it should have seventeen (five/seven/five) *onji*
which is not a syllable, but a sound symbol. these sound symbols are shorter
and more uniform than english syllables. this is is a roundabout way of
saying something many translators and english haiku poets (as i have in
reading and composing haiku for sometime now) found: english haiku needs
only about ten to twelve syllables to have nearly the same duration and
rhythm as a japanese haiku. so don't feel the need to pad a haiku if you
find it sounds right to your ears in ten or thirteen syllables. remember
haiku is unforced, without frills and literary flourishes.

	winter moonlight -
	little laptop
	warm with plan9.		[oz/2002]

oz
---
Never let your job get in the way of your work.
			  -- Sir Isaac Newton, Master of the Royal Mint

Re: [9fans] blanks in file names

[Dave]

I happen to dislike much about rio, but that's not a fundamental part
of the OS; it's simply a shell (and a very good one, too, which is why
you're not likely to see me rushing to replace it).  FWIW, though, I can
easily imagine a vi-like or emacs-like environment with a framebuffer
back-end doing the job just fine, without rodents, and without losing
any functionality.

 - Dave


Jack Johnson wrote:
> 
> Geoff Collyer wrote:
> > One downside of APE is that it's made some people think of Plan 9 as
> > Just Another Goddamned Unix Implementation.
> 
> Is this what Dave meant when he wrote:
>  > /me shudders at the thought that GUIs might someday take over the
>  > world :-(
> 
> ?
> 
> For the record, I think Plan 9 is a textual GUI.  Though you could 
> probably swing it with a bare rc shell, unless you can run it on a 
> braille term (or hear it through your speakers) it's probably safest to 
> call it a GUI.
> 
> At this point, I guess I'm thankful that there are no Olfactory User 
> Interfaces (though paradoxically I've seen a few that stank).
> 
> I did read somewhere that there's a braille term that can draw out 
> window regions.  A variation of acme on something like that could be 
> interesting.  Mouse actions would take some serious rethought, though.
> 
> -Jack
> 

Re: [9fans] blanks in file names

[Dave]

I just noticed that there were a few more messages in this thread that
got lost in my email (and I can't really blame elm this time, either) :-(

I know we'd all like to kill this thread, but I can't look at this post
without replying; it's just too misguided to ignore.

The reason I want a new open is _not_ that the current open isn't
efficient enough for my tastes.  The reason I want a new open is that the
current one was lifted straight out of the old UNIX design, and miserably
fails to accomodate the modular approach that UNIX no longer provides,
but Plan 9 has found some brilliant ways of recreating (which is why I
got my original Plan9 installation in the first place).  The fact that
the new open happens to be more efficient than the old one for many
applications is merely a neat side effect, one that I only brought up
when I was accused of making open _less_ efficient (and I didn't see you
pipe up then to complain that cost doesn't matter).  In UNIX, paths never
became real objects in the sense that Plan 9 has promoted them to, because
UNIX has very primitive filesystem serving.  In Plan 9, it becomes a lot
more interesting to have commands simply create temporary filesystems
and populate them with the results (e.g., ls creating a directory with
each file being one of the entries requested, and the contents of each
of those files being information about the entry).  UNIX tried to make
everything a file, but it got bogged down with plain-text IO.  Plan 9
aligned itself squarely with the file philosophy, and I see no reason
to stop short when filenames are concerned.

 - Dave

BTW - I don't know if it's worth noting that while strlen (sequential
memory accesses) is an order of magnitude faster than DES (repeated
computations + sequential memory accesses), simply getting the length
from a structure (direct memory access) is an order of magnitude faster
than strlen.


Geoff Collyer wrote:
> 
> My apologies for dragging this out, but I think this gets to the heart
> of the matter.
> 
> The cost of strlen doesn't matter, it really doesn't matter.  I can't
> recall seeing a real-life program where the cost of strlen was more
> than trivia.
> 
> A quick comparison on Plan 9 shows that strlen runs at 19 times the
> speed of DES, using the same 64-byte string as input each time, which
> is almost certainly longer than the average string.  DES is really
> slow, by design; it's not just a question of a few more memory
> accesses; it's just hard to do the bit-swizzling quickly in software
> (though it can be done relatively quickly in hardware).
> 
> One of the big achievements of Unix was to get people to stop worrying
> about the microseconds and look at the slightly larger picture of what
> could be done if your first concern were not microseconds, and that
> was a quarter-century ago!  With processor cycles being so cheap and
> available now, it's generally not worth worrying about expenses until
> they cause a real (measurable, reproducible) problem.  Proposals based
> on supposedly greater efficiency for a new open(2) interface are not
> worth considering, particularly when open's efficiency isn't currently
> a problem, and the new interface is ugly, incompatible with the
> existing clean and simple one, and purports to solve non-problems like
> allowing NUL and slash in file name components.
> 
> 
> One downside of APE is that it's made some people think of Plan 9 as
> Just Another Goddamned Unix Implementation.  If you're not interested
> in exploring what's new in Plan 9, and are offended that realloc isn't
> provably optimal or that GNU configure doesn't just run out the box,
> why are you using Plan 9?  FreeBSD, OpenBSD, NetBSD and Linux are all
> available at no monetary cost.
> 

Re: [9fans] blanks in file names

[Jack Johnson]

Geoff Collyer wrote:
> One downside of APE is that it's made some people think of Plan 9 as
> Just Another Goddamned Unix Implementation.

Is this what Dave meant when he wrote:
 > /me shudders at the thought that GUIs might someday take over the
 > world :-(

?

For the record, I think Plan 9 is a textual GUI.  Though you could 
probably swing it with a bare rc shell, unless you can run it on a 
braille term (or hear it through your speakers) it's probably safest to 
call it a GUI.

At this point, I guess I'm thankful that there are no Olfactory User 
Interfaces (though paradoxically I've seen a few that stank).

I did read somewhere that there's a braille term that can draw out 
window regions.  A variation of acme on something like that could be 
interesting.  Mouse actions would take some serious rethought, though.

-Jack

Re: [9fans] blanks in file names

[Geoff Collyer]

My apologies for dragging this out, but I think this gets to the heart
of the matter.

The cost of strlen doesn't matter, it really doesn't matter.  I can't
recall seeing a real-life program where the cost of strlen was more
than trivia.

A quick comparison on Plan 9 shows that strlen runs at 19 times the
speed of DES, using the same 64-byte string as input each time, which
is almost certainly longer than the average string.  DES is really
slow, by design; it's not just a question of a few more memory
accesses; it's just hard to do the bit-swizzling quickly in software
(though it can be done relatively quickly in hardware).

One of the big achievements of Unix was to get people to stop worrying
about the microseconds and look at the slightly larger picture of what
could be done if your first concern were not microseconds, and that
was a quarter-century ago!  With processor cycles being so cheap and
available now, it's generally not worth worrying about expenses until
they cause a real (measurable, reproducible) problem.  Proposals based
on supposedly greater efficiency for a new open(2) interface are not
worth considering, particularly when open's efficiency isn't currently
a problem, and the new interface is ugly, incompatible with the
existing clean and simple one, and purports to solve non-problems like
allowing NUL and slash in file name components.


One downside of APE is that it's made some people think of Plan 9 as
Just Another Goddamned Unix Implementation.  If you're not interested
in exploring what's new in Plan 9, and are offended that realloc isn't
provably optimal or that GNU configure doesn't just run out the box,
why are you using Plan 9?  FreeBSD, OpenBSD, NetBSD and Linux are all
available at no monetary cost.

Re: [9fans] blanks in file names

[arisawa]

Hello,

Dave, you should be aware that you started from wrong place.
As I said, your new_open is not required.
You speek too much about that is not required.
And I am surprised to see:
>then user programs and libraries
>are free to use URL, HTML, quoting, C-style escaping,
>or any other system that suits the application at hand

Kenji Arisawa

Re: [9fans] blanks in file names

[Jim Choate]

On Sun, 14 Jul 2002, Dave wrote:

> /me shudders at the thought that GUIs might someday take over the
> world :-(

I also, it means that mankind will have missed the entire point of
computing.


 --
    ____________________________________________________________________

              When I die, I would like to be born again as me.

                                            Hugh Hefner
     ravage@ssz.com                                         www.ssz.com
     jchoate@open-forge.org                          www.open-forge.org

    --------------------------------------------------------------------

Re: [9fans] blanks in file names

[Dave]

Okay, I think we might have enough votes to finally kill the thread ;-)

 - Dave


Ish Rattan wrote:
> 
> Probably it is time for this thread to go away.
> 
> An OS design is a reflection of what the designer considers
> important -- who gives a hoot if it not a clean design. My
> response: if you do not like it either desgin your "own" OR
> use one that you consider a clean OS!!
> 
> -ishwar
> 
> On Wed, 10 Jul 2002, Dave wrote:
> 
> > Reply inline:
> >
> >  - Dave
> >
> > Dan Cross wrote:
> > >
> > > > The "offending" program in this case is the program that's so much simpler
> > > > because it's using the new kernel interface rather than the old one.
> > >
> > > I don't think it would be simpler; I think it would be more
> > > complicated.  You're replacing a simple, textual representation of an
> > > object with a binary representation; you have to have some way to do
> > > canonicalization in the common case, but even that path is thrwat with
> > > danger.
> > Manipulating text with all sorts of dynamic buffers is substantially
> > more complicated than simply replacing a node in a linked list.
> > The canonicalization is all being done by the kernel, or a library.
> >
> > >
> > > > Nothing has to be rethought because all my proposed changes do is restore
> > > > the strict seperation between nodes in a filename (something UNIX -
> > > > and therefore Plan 9 - likes to rely on).
> > >
> > > But they change an already well-established interface.  Have you
> > > thought through the implications of this, in all their macabre glory?
> > > What you propose--changing the most basic interface for opening a file
> > > in a system where everything looks more or less like a file--has huge
> > > implications.  And all this just to support a strange edge-case, which
> > > is adequately solved by substitutions in the filename.  Sure, it's not
> > > perfect in some weird pathological case, but how often is this going to
> > > come up in practice?  Remember: Optimize for the common case.
> > Optimization for the common case is good, but creating a system where the
> > uncommon case will cause major mayhem at the system level is evidence
> > of a very unclean approach.  (When you consider the reasoning behind
> > the problem (namely, spaces and slashes in filenames kill our ability
> > to seperate nodes easily), it makes perfect sense that our solution
> > isn't very clean.  The only clean solution is to restore the ancient
> > UNIX ideal of being able to easily seperate nodes.  In other words,
> > either kill spaces altogether and damn interoperability, or promote
> > spaces to full citizenship.)
> >
> > >
> > > > There's plenty of experience with other systems working on linked lists
> > > > (including a huge amount of kernel code in my Linux box that I'm typing
> > > > from, ATM).  Most of the problems with linked lists have been pretty
> > > > well documented, by now.
> > >
> > > It's the huge amount of kernel code that Plan 9 is trying to avoid.
> > String manipulation is more complex than linked list manipulation.
> >
> > >
> > > Being forced to conform to a lot of external interfaces *will* kill the
> > > system.
> > I don't dispute that point, but the interface I propose is most unlike
> > any other interface currently known to man (not trying to conform to any
> > external interface).  I'm simply pointing out that failing to provide
> > at least a 1-1 mapping with capabilities that are already widely used
> > in external systems that must interoperate with ours *will* kill us.
> >
> > >
> > > Besides, the point Nemo was trying to make umpteen posts ago was that,
> > > yes, you can roll back changes using the dump filesystem, which gives
> > > you temporal mobility.  He is right.
> > You can do a lot of things if you're prepared to get involved in the
> > functions that your OS should be doing automatically.  Try running an FTP
> > mirror to a busy site that way, though, and you'll quickly discover why
> > automation is a good thing.  The worst part about our system is that the
> > "solution" you eventually find for an FTP mirror will be useless on an
> > HTTP proxy.  When "solutions" need to be modified for each individual
> > application, you know that the system isn't clean.
> >
> > >
> > > 	- Dan C.
> > >
> >  - Dave C. :-)
> >
> 

Re: [9fans] blanks in file names

[Dave]

/me shudders at the thought that GUIs might someday take over the
world :-(

 - Dave

BTW - The point I was trying to make is that if the kernel doesn't use
any particular filename-rewriting rule, then user programs and libraries
are free to use URL, HTML, quoting, C-style escaping, or any other
system that suits the application at hand, without having to rewrite
their versions into whatever rewriting rule the kernel decides to use.
(I guess you can say I'm "pro choice.")


arisawa@ar.aichi-u.ac.jp wrote:
> 
> Hello,
> 
> > One of the beautiful things about Plan 9 is that it provides a  
> solution
> > that's workable with little effort.  The various substitution file
> > systems provide a workable solution without introducing any  
> additional
> > complexity.  If you want a total--100% complete--solution, then a
> > `urlifyfs' can be written that uses URL escaping as a canonical
> > representation, or something similar.  The system interface  
> doesn't
> > have to be changed, though.  *That* is the mark of a clean system
> > design.
> The idea was born from the discussion between Dave and I.
> Dave proposed '\' escape.
> Until that time, I had been tied to quoting strategy.
> %XX looks ugly as Dave says, but it works cleanly.
> That is the beauty.
> We must be patient with this ugliness because we are in textual
> environment. Full GUI environment, I believe Plan9 also have it
> in future, will remove this ugliness.
> Thanks Dave.
> 
> Kenji Arisawa
> 

Re: [9fans] blanks in file names

[rob pike, esq.]

> > strlen() is an expensive operation.
> 
> No, it's not terribly expensive.

The strlen() of this thread is certainly an expensive operation.

-rob

Re: [9fans] blanks in file names

[Dave]

Reply inline:

 - Dave

Geoff Collyer wrote:
> 
> dave@dave.tj,
> 
> Have you actually used Plan 9 or even read its manual?  I ask because
> you've made a succession of odd statements:
Yup, I had Release 3 on an old system (which I tried to upgrade to Release
4, to no avail), although I must admit that it wasn't my preferred
platform.  Everything from the directory structure to the process of
trying to compile standard UNIX-compatible software was extremely awkward.
I was hoping to give Release 4 a try, but until I can get the Millenium
(or the Blade 3D, my only other card) to work with Release 4, I'm out
of luck.  As for the manual, no way ... I never read the manual until
I run into trouble ;-P

> 
> > strlen() is an expensive operation.
> 
> No, it's not terribly expensive.  Even if it were, most strings are
> short, so it's not much of an issue.  DES encryption is expensive.
It must examine every byte in the array, and that's an expensive
operation.  DES must examine every byte multiple times, so it's a very
expensive operation.  The structure method doesn't require access to
every byte in the array, since the length is counted.

> 
> > realloc() sucks in a multithreaded environment.
> 
> You don't define ``sucks'', but the whole malloc family use internal
> locks on Plan 9 so that they *do* work well (or at least don't corrupt
> memory) in multiprocess or multithread programs.
Yup, I should've defined "sucks" - here goes:
suck - v. being rediculously inefficient, due to the need to move the
realloc(3)ed chunk almost constantly, because realloc(3) isn't smart
enough to know that another realloc(3) in another thread is also growing
a block, and realloc(3) hates stepping on other realloc(3)s' toes

(BTW - realloc(3) can also suck in a single-threaded environment (two
arrays that are growing in the same thread), but it's more common
for it to be used in a way that causes it to suck in multi-threaded
environments.)

> 
> > Also, I'd like to mention again that I'm not asking the kernel to
> > allocate memory.
> 
> The kernel already allocates memory, including string memory.  What's
> the big deal?
AFAIK, the kernel does _not_ allocate userland memory with anything
but the brk(2) call - which is only called by malloc(3), realloc(3),
and operator new().  Please correct me if I'm wrong.

> 

Re: [9fans] blanks in file names

[Dave]

Lucio De Re wrote:
> 
> On Thu, Jul 11, 2002 at 02:14:59AM -0400, Dave wrote:
> > 
> > Reply inline:
> > 
> You make some valid points, but I think they are well in excess of
> present requirements.  Not to say that a future OS/GUI will not (have
> to) take them into consideration.
Okay, fair enough ... I'll just wait a few more years until some OS
agrees to go out on a limb with me (Release 5, maybe?) ;-)

> 
> You do however confuse URI and HTML and you really shouldn't:
> 
> 
> > The only way to have the urlifyfs concept providing a 100% complete
> > solution is to use it as the default filesystem for your own stuff.
> > The reason?  imagine downloading a file "blah%apos;" from an FTP server;
> > now, you download a file "blah'" from an FTP server (which your urlifyfs
> > faithfully translates into "blah%apos;" without realizing that it's
> > destroying a different file).  Guess what?  You've just clobbered your
> > original.  Now, if you're going to use urlifyfs for your own stuff
> 
> URI: blah%apos; -> blah%25pos%3b
Whoopsies ... my brain must've been half mush when I typed that :-(

> 
> ++L
> 

Re: [9fans] blanks in file names

[arisawa]

Hello,

I should have added to my previous thread:
I think `urlifyfs' or similar stuff is good idea.
I didn't have that idea.

Kenji Arisawa

Re: [9fans] blanks in file names

[okamoto]

>The idea was born from the discussion between Dave and I.
>Dave proposed '\' escape.

No, both are completely misunderstanding the point which Rob pointed.
You can read once more his and nemo's remarks before.

Kenji

Re: [9fans] blanks in file names

[Dan Cross]

I want to let this thread die, but I just have one last thing to say:

> Getting the kernel away from plain text doesn't mean getting the shell

But....  It took 20 years to get the kernel to plain text.  I don't want
to go back.

	- Dan C.

Re: [9fans] blanks in file names

[arisawa]

Hello,

> One of the beautiful things about Plan 9 is that it provides a  
solution
> that's workable with little effort.  The various substitution file
> systems provide a workable solution without introducing any  
additional
> complexity.  If you want a total--100% complete--solution, then a
> `urlifyfs' can be written that uses URL escaping as a canonical
> representation, or something similar.  The system interface  
doesn't
> have to be changed, though.  *That* is the mark of a clean system
> design.
The idea was born from the discussion between Dave and I.
Dave proposed '\' escape.
Until that time, I had been tied to quoting strategy.
%XX looks ugly as Dave says, but it works cleanly.
That is the beauty.
We must be patient with this ugliness because we are in textual
environment. Full GUI environment, I believe Plan9 also have it
in future, will remove this ugliness.
Thanks Dave.

Kenji Arisawa

Re: [9fans] blanks in file names

[Ish Rattan]

Probably it is time for this thread to go away.

An OS design is a reflection of what the designer considers
important -- who gives a hoot if it not a clean design. My
response: if you do not like it either desgin your "own" OR
use one that you consider a clean OS!!

-ishwar

On Wed, 10 Jul 2002, Dave wrote:

> Reply inline:
>
>  - Dave
>
> Dan Cross wrote:
> >
> > > The "offending" program in this case is the program that's so much simpler
> > > because it's using the new kernel interface rather than the old one.
> >
> > I don't think it would be simpler; I think it would be more
> > complicated.  You're replacing a simple, textual representation of an
> > object with a binary representation; you have to have some way to do
> > canonicalization in the common case, but even that path is thrwat with
> > danger.
> Manipulating text with all sorts of dynamic buffers is substantially
> more complicated than simply replacing a node in a linked list.
> The canonicalization is all being done by the kernel, or a library.
>
> >
> > > Nothing has to be rethought because all my proposed changes do is restore
> > > the strict seperation between nodes in a filename (something UNIX -
> > > and therefore Plan 9 - likes to rely on).
> >
> > But they change an already well-established interface.  Have you
> > thought through the implications of this, in all their macabre glory?
> > What you propose--changing the most basic interface for opening a file
> > in a system where everything looks more or less like a file--has huge
> > implications.  And all this just to support a strange edge-case, which
> > is adequately solved by substitutions in the filename.  Sure, it's not
> > perfect in some weird pathological case, but how often is this going to
> > come up in practice?  Remember: Optimize for the common case.
> Optimization for the common case is good, but creating a system where the
> uncommon case will cause major mayhem at the system level is evidence
> of a very unclean approach.  (When you consider the reasoning behind
> the problem (namely, spaces and slashes in filenames kill our ability
> to seperate nodes easily), it makes perfect sense that our solution
> isn't very clean.  The only clean solution is to restore the ancient
> UNIX ideal of being able to easily seperate nodes.  In other words,
> either kill spaces altogether and damn interoperability, or promote
> spaces to full citizenship.)
>
> >
> > > There's plenty of experience with other systems working on linked lists
> > > (including a huge amount of kernel code in my Linux box that I'm typing
> > > from, ATM).  Most of the problems with linked lists have been pretty
> > > well documented, by now.
> >
> > It's the huge amount of kernel code that Plan 9 is trying to avoid.
> String manipulation is more complex than linked list manipulation.
>
> >
> > Being forced to conform to a lot of external interfaces *will* kill the
> > system.
> I don't dispute that point, but the interface I propose is most unlike
> any other interface currently known to man (not trying to conform to any
> external interface).  I'm simply pointing out that failing to provide
> at least a 1-1 mapping with capabilities that are already widely used
> in external systems that must interoperate with ours *will* kill us.
>
> >
> > Besides, the point Nemo was trying to make umpteen posts ago was that,
> > yes, you can roll back changes using the dump filesystem, which gives
> > you temporal mobility.  He is right.
> You can do a lot of things if you're prepared to get involved in the
> functions that your OS should be doing automatically.  Try running an FTP
> mirror to a busy site that way, though, and you'll quickly discover why
> automation is a good thing.  The worst part about our system is that the
> "solution" you eventually find for an FTP mirror will be useless on an
> HTTP proxy.  When "solutions" need to be modified for each individual
> application, you know that the system isn't clean.
>
> >
> > 	- Dan C.
> >
>  - Dave C. :-)
>

Re: [9fans] blanks in file names

[Geoff Collyer]

dave@dave.tj,

Have you actually used Plan 9 or even read its manual?  I ask because
you've made a succession of odd statements:

> strlen() is an expensive operation.

No, it's not terribly expensive.  Even if it were, most strings are
short, so it's not much of an issue.  DES encryption is expensive.

> realloc() sucks in a multithreaded environment.

You don't define ``sucks'', but the whole malloc family use internal
locks on Plan 9 so that they *do* work well (or at least don't corrupt
memory) in multiprocess or multithread programs.

> Also, I'd like to mention again that I'm not asking the kernel to
> allocate memory.

The kernel already allocates memory, including string memory.  What's
the big deal?

Re: [9fans] blanks in file names

[Dave]

Reply inline:

 - Dave

anothy@cosym.net wrote:
> 
> on the "linked lists for paths instead of strings with /" topic:
> while i can see how this could be interesting to persue, from a
> systems architecture research point of view, i'm unclear on how
> exactly this helps us solve any of the problems people are
> talking about. maybe i'm just missing something; if so, please
> clue me in. but here's what i don't see:
This won't directly solve any userland problems, because it's a kernel
level change.  However, it _will_ provide the foundation necessary for
a real makeover for our shell, so it becomes a true translator between
the kernel and the user (what it was always meant to be).

> 
> first, it seems your suggestion is primarialy aimed at allowing
> "/" in file names (perhaps some argument about generality or
> interoperation with DOS-derived systems and "\" could be made,
> as well). okay, i can see the theoretical benefit there (but it
> strikes me as a much more dubious gain than spaces in file
> names). but what would this system _look_ like, from a user's
> point of view? what would the output of `pwd' be? a list of
> strings? but how can i store a representation of that? wouldn't
> that involve lots of interaction with the shell, or whatever
> else was doing the interpretation (like $path does, although
> that's used is very few places, and is very function-specific,
> so isn't really much of an issue)?
I remarked in a couple of my previous posts about the subject, but let
me show some examples of what I mean:
Let's say we have a shell that uses C-style encoding, and we're in a
directory called (using a C-syntax string) "/usr/\/\\\n\'blah":
crazysh> pwd
/usr//\
'blah
crazysh> pwd --unambiguous
/usr/\/\\\n\'blah
crazysh> ls
What's wrong with you?
You don't want to know what's in this directory ;-/
crazysh> ls --unambiguous
What\'s wrong with you\?
\0\0\0
You don\'t want to know what\'s in this directory ;-\/
crazysh> exit

You'll notice that the initial pwd command prints the directory in
easy-for-human-to-read format - what it was "meant" to look like.
The second command runs the same program as the first command, but the
_shell_ (not the pwd command!) is given the --unambiguous option so
it prints the resulting path in its native (C-style encoding) format.
(Programs don't produce "text" as output; they produce "objects," which
- as I remarked in a previous post - may be filenames, strings, etc.)
The third command prints a directory listing in easy-for-human-to-read
format, and the fourth tells the shell to print ls' output filenames in
C-style rather than raw form (where you'll notice a file consisting of
nothing but NULLs suddenly appears - a neat trick that can make users
of other OSs quite jealous).

Obviously, a user may opt for a shell using URL encoding as its default
encoding, or even replace the "raw" encoding with URL/C/etc. encoding.
It's all a matter of personal preference, and the kernel isn't involved
at all ... nor is any filesystem code.

> 
> second, i'm just not seeing at all how this helps us address
> spaces in file names (and maybe it wasn't meant to; this
> thread has wandered a bit). the dificult part of the problem
> (perhaps the entirety of the problem) seems to lie in getting
> the various tools that need to communicate to understand the
> edges of file names: things like "grep `{ls} foo". i fail to
> see how any of the kernel changes (or library additions) you
> propose even talk to this realm of problem.
Well, if you're using C-style syntax, filename edges are simply newlines.
If you're using url encoding, spaces seperate filenames.  If you're using
Base64 (which I believe any human would have to be crazy to do, but I
see no reason for the system to stop crazy people), you'd have plenty
of choices for filename edges.  You just have to make sure the both
sides of the pipe are working with the same encoding.  (e.g., "crazysh>
program-outputting-in-url | program-expecting-in-c" ain't gonna work,
but most of your userland programs would probably ship with your shell
or offer optionss to support your shell's encoding (or if your shell
uses an uncommon encoding, it might provide programs to convert to/from
more common encodings).)

> 
> third: want newline too?
A general solution should handle all cases.  I see no reason not to handle
newlines, too.  (In fact, my example above illustrated a lot more than
mere newlines being encoded.)

> 
> on another embedded thread, you wrote:
> // You can do a lot of things if you're prepared to get
> // involved in the functions that your OS should be doing
> // automatically.  Try running an FTP mirror to a busy site
> // that way, though, and you'll quickly discover...
> 
> that you've completely missed what dan was saying? he (and
> Nemo before him) was talking about the ability of Plan 9's
> dump to back out changes to kernel, library, and utilitys
> uniformly. i don't have a clue what FTP or HTTP mirrors or
> "things your OS should do automatically" have to do with
> any of this. are we just having unrelated conversations at
> each other?
I was pointing out that yesterday doesn't even attempt to address
the problem I was referring to (keeping an FTP mirror up-to-date).
I (thought I) understood that he was proposing to run a filter on files
he's mirroring.  My answer was quite simply that doing so is unfeasible
on a large scale.  I hope that clarifies a bit :-)

> ã‚¢
> 

Re: [9fans] blanks in file names

[Lucio De Re]

On Thu, Jul 11, 2002 at 02:14:59AM -0400, Dave wrote:
> 
> Reply inline:
> 
You make some valid points, but I think they are well in excess of
present requirements.  Not to say that a future OS/GUI will not (have
to) take them into consideration.

You do however confuse URI and HTML and you really shouldn't:


> The only way to have the urlifyfs concept providing a 100% complete
> solution is to use it as the default filesystem for your own stuff.
> The reason?  imagine downloading a file "blah%apos;" from an FTP server;
> now, you download a file "blah'" from an FTP server (which your urlifyfs
> faithfully translates into "blah%apos;" without realizing that it's
> destroying a different file).  Guess what?  You've just clobbered your
> original.  Now, if you're going to use urlifyfs for your own stuff

URI: blah%apos; -> blah%25pos%3b

++L

Re: [9fans] blanks in file names

[Dave]

Reply inline:

 - Dave

Dan Cross wrote:
> 
> > > I don't think it would be simpler; I think it would be more
> > > complicated.  You're replacing a simple, textual representation of an
> > > object with a binary representation; you have to have some way to do
> > > canonicalization in the common case, but even that path is thrwat with
> > > danger.
> > 
> > Manipulating text with all sorts of dynamic buffers is substantially
> > more complicated than simply replacing a node in a linked list.
> > The canonicalization is all being done by the kernel, or a library.
> 
> How could this possibly be in the kernel?  After all, you're talking
> about changing the interface to open a file; I pass a file name via
> some mechanism to a user level application that wants to call open on
> it.  What's it supposed to do?  Does the shell now pass a linked list
> as an argument to main somehow?  How does the system know that it's a
> file?  Do we have to replace the argument vector with some more complex
> representation that encapsulates type information (e.g., this argument
> is a file, this next one is a string, etc)?  Does the shell change to
> represent file names as lists?  Does the user suffer the indignation of
> having to specify a list of path components to represent a file?  Or do
> we provide a canonicalization library for shell arguments, in which
> case, you have the exact same problem as supporting spaces now, since
> most programs are going to expect to get file name arguments in the
> canonical representation?  If you do that, who calls it?  The shell or
> the library?
That's an interesting point I didn't quite consider ... we'll have to
change the exec interface a lot more than I suspected at first glance.
I wasn't planning that until much later, because it'll require very
fundamental changes to the shell.  (/me hates proposing incremental
changes, because they invariably depend on other fundamental changes in
order for people to see their utility.)

> 
> I for one am going to be *very* unhappy if I have to type:
> 
> 	cat ('' 'usr' 'cross' 'file')
> 
> Instead of:
> 
> 	cat /usr/cross/file
> 
> Or do you make every program that wants to open a file call a function
> to canonicalize a filename into the internal format before it calls
> open?
My image of a shell is a user interface.  It should translate
all output from programs into a format that's easy for a human to
understand, and should offer to translate data entered by the user
from an easy-for-a-human-to-input format into the machine format.
If you want to print /"usr"/"cross"/"file", you should be able to
type something like "cat /usr/cross/file" and have the shell translate
that into the collection of lists (/usr/bin/cat and /usr/cross/file)
required for the underlying system.  The shell should also translate
the output of the ls command, for instance, so it prints filenames
in an easy-for-humans-to-understand format.  The ls command, though,
should only print filenames in an easy-for-machine-to-understand
format.  Basically, the shell is the bidirectional translator between
computer-speak and human-speak.  That's it's raison d'étre (spelling?).
Getting the kernel away from plain text doesn't mean getting the shell
away from plain text.  The shell can choose to support any method(s)
it wants to represent filenames in an "easy-for-machine-to-understand"
format, since it'll be converting the filenames into linked lists for
the kernel.  Utilities like find or ls or whatever output filenames in
a format that your shell can read.  (I envision an rc file supplied by
the shell to let other programs know what formats it supports.)

> 
> > > But they change an already well-established interface.  Have you
> > > thought through the implications of this, in all their macabre glory?
> > > What you propose--changing the most basic interface for opening a file
> > > in a system where everything looks more or less like a file--has huge
> > > implications.  And all this just to support a strange edge-case, which
> > > is adequately solved by substitutions in the filename.  Sure, it's not
> > > perfect in some weird pathological case, but how often is this going to
> > > come up in practice?  Remember: Optimize for the common case.
> > 
> > Optimization for the common case is good, but creating a system where the
> > uncommon case will cause major mayhem at the system level is evidence
> > of a very unclean approach.  (When you consider the reasoning behind
> > the problem (namely, spaces and slashes in filenames kill our ability
> > to seperate nodes easily), it makes perfect sense that our solution
> > isn't very clean.  The only clean solution is to restore the ancient
> > UNIX ideal of being able to easily seperate nodes.  In other words,
> > either kill spaces altogether and damn interoperability, or promote
> > spaces to full citizenship.)
> 
> But Plan 9 can handle this.
> 
> One of the beautiful things about Plan 9 is that it provides a solution
> that's workable with little effort.  The various substitution file
> systems provide a workable solution without introducing any additional
> complexity.  If you want a total--100% complete--solution, then a
> `urlifyfs' can be written that uses URL escaping as a canonical
> representation, or something similar.  The system interface doesn't
> have to be changed, though.  *That* is the mark of a clean system
> design.
The only way to have the urlifyfs concept providing a 100% complete
solution is to use it as the default filesystem for your own stuff.
The reason?  imagine downloading a file "blah%apos;" from an FTP server;
now, you download a file "blah'" from an FTP server (which your urlifyfs
faithfully translates into "blah%apos;" without realizing that it's
destroying a different file).  Guess what?  You've just clobbered your
original.  Now, if you're going to use urlifyfs for your own stuff
on your Plan 9 system, you're going to have to deal with the same
shell-interaction issues that my system has to deal with.  The only
difference is that my system doesn't break if somebody forgets to use
urlifyfs on a new filesystem, because my system moves text representation
of filenames over to the shell, where it belongs, rather than dumping
that burden on a filesystem translation hack.

> 
> The Unix `ideal' was eliminated because it was overly complex, without
> a commensurate gain in functionality.  Besides, the inode system didn't
> really fit in well with the idea of 9p.
> 
> > > > There's plenty of experience with other systems working on linked lists
> > > > (including a huge amount of kernel code in my Linux box that I'm typing
> > > > from, ATM).  Most of the problems with linked lists have been pretty
> > > > well documented, by now.
> > > 
> > > It's the huge amount of kernel code that Plan 9 is trying to avoid.
> > 
> > String manipulation is more complex than linked list manipulation.
> 
> No, it's really not.  Consider passing a linked list as an argument to
> a function you're calling, versus passing an argument vector of
> strings.  How do you do that?  Do you muck with all the C startup code
> to make sure you get the linking and so right in such a way that the
> list is in a contiguous memory block so it doesn't get stomped by the
> image read by exec?  Do you pass each node in the list to main as a
> seperate string in the argument vector?  If so, how do you tell when
> a file name ends and another begins?  Do we introduce some convention
> for delineating the beginning and ends of a filename in a list
> representation, effectively creating a protocol that every program has
> to follow to take a filename as an argument?  Surely the former option
> is significantly easier....
This is only true with our current exec family, which has been essentially
carried over unchanged from UNIX.  It's based on strings, not on lists.
IMHO, arguments should be objects.  Those objects can be filenames,
options with or without arguments of their own, subcommands, just plain
strings, etc.  This makes arguments a lot more representitive of what
they actually are, and eliminates the need for complex argument-handling
libraries.  Obviously, this whole change can be totally transparent to
the user, because his shell is doing the necessary translations back
and forth.  However, you get an extremely powerful system as the payoff,
a system that makes it rather easy to reimplement all our current syscalls
as tiny library functions, possibly in an emulation library.

> 
> Consider a possible canonicalization routine that might be used in
> a substitution FS:
> 
> char *
> canonical(char *str)
> {
> 	char	*p, *s, *t;
>  
> 	if (str == nil || (p = malloc(2 * strlen(str) + 1)) == nil) {
> 		return(nil);
> 	}
> 	for (s = str, t = p; *s != '\0'; s++, t++) {
> 		if (isspace(*s)) {
> 			*t++ = '+';	/*  Or whatever.  */
> 			*t = '2';
> 			continue;
> 		}
> 		*t = *s;
> 	}
> 	if ((s = realloc(p, strlen(p) + 1)) == nil) {
> 		free(p);
> 	}
>  
> 	return(s);
> }
> 
> That's pretty straight-forward; just inserting into a linked list
> would be just as hard.  Doing so in a contiguous memory block would
> be, I think harder (you'd have to step over the list, keep a count
> of how much memory you needed, then allocate the list, copy each
> node and set the links.  That's a pain).
strlen() is an expensive operation.  realloc() sucks in a multithreaded
environment.  To top it all off, that algorithm doesn't take into account
the expansion which is ABSOLUTELY NECESSARY in order to achieve 100%
coverage.  (If you're not going to achieve a 1-1 mapping, it's silly to
even bother with this.)  Also, I'd like to mention again that I'm not
asking the kernel to allocate memory.  The userland program provides
a block of memory, and the kernel manipulates that block, returning an
error if the block is too small.

> 
> > > Being forced to conform to a lot of external interfaces *will* kill the
> > > system.
> > 
> > I don't dispute that point, but the interface I propose is most unlike
> > any other interface currently known to man (not trying to conform to any
> > external interface).  I'm simply pointing out that failing to provide
> > at least a 1-1 mapping with capabilities that are already widely used
> > in external systems that must interoperate with ours *will* kill us.
> 
> Well, if you *really* want 100% 1 to 1 mappings, use the URL encoding
> others have mentioned, or something similar.  As it is, it seems that
> this mostly works; about 80% of what's needed is there.
URL encoding _will_ work if it's implemented right (except for the
uncleanliness I mentioned above, and some more problems I mention below).
However, using URL encoding makes the resulting system just as ugly as
the one I'm proposing from the user's perspective, but much much uglier
from a system perspective.

> 
> > > Besides, the point Nemo was trying to make umpteen posts ago was that,
> > > yes, you can roll back changes using the dump filesystem, which gives
> > > you temporal mobility.  He is right.
> > 
> > You can do a lot of things if you're prepared to get involved in the
> > functions that your OS should be doing automatically.  Try running an FTP
> > mirror to a busy site that way, though, and you'll quickly discover why
> > automation is a good thing.  The worst part about our system is that the
> > "solution" you eventually find for an FTP mirror will be useless on an
> > HTTP proxy.  When "solutions" need to be modified for each individual
> > application, you know that the system isn't clean.
> 
> Yesterday is a wonderful tool, and can be scripted to do whatever you
> want.  Eg, copying all files that changed on June 14th back to the
> cache isn't very diffcult.
Yesterday can't be used to update the relative references in all the
README files in the FTP archives to the urlified versions.

> 
> I don't see what running a big FTP mirror has to do with it.  netlib is
> a big FTP site; it runs on Plan 9.  Maybe it's not a mirror, but so what?
Since it's not a mirror, it doesn't have to contend with all the spaceful
filenames you find in the non-Plan9 world.

> I also don't see how you can't leverage whatever you did for FTP with
> HTTP.  The substitution-style FS gives you a *lot* of flexibility in this
> area.
What you did in FTP was scanning the README files for references.
What you do in HTTP is updating all the href and src attributes in HTML
files (and hope none of the JAVA programs have embedded URLs that you
can't change at all), so you don't get broken links everywhere.

...unless you want to implement the transformation/detransformation
code in the FTP and HTTP servers, as well ... in which case your
system becomes one step worse than my system, because you have
transformation/detransformation code in two places on your system :-(

> 
> 	- Dan C.
> 

Re: [9fans] blanks in file names

[okamoto]

>Eh, I tire of the discussion

Yes, indeed.
We can refer to a statement from Rob times before.

Kenji

Re: [9fans] blanks in file names

[Dave]

Eh, I tire of the discussion ... let's just make a userland fix or
whatever ... if it bugs me too much, I'll just patch up the kernel,
or something. . .

 - Dave


arisawa@ar.aichi-u.ac.jp wrote:
> 
> Dave wrote:
> >If I had to vote a priori, blanks would certainly be a no-go for
> >filenames.  However, the pressure cooker has already burst, and our
> >...
> >keep our chicken from splattering all over the kitchen ;-/
> 
> Important or not? That is the problem.
> If we agree it is important, we must eat spaes.
> Then my chice is to cook spaces so that stomach is not broken.
> How to cook?
> I believe the first discussion should be forcussed
> to expression of path considering in future needs.
> I believe you and I share much opinion.
> 
> Kenji Arisawa
> 
> 

Re: [9fans] blanks in file names

[Dan Cross]

> > I don't think it would be simpler; I think it would be more
> > complicated.  You're replacing a simple, textual representation of an
> > object with a binary representation; you have to have some way to do
> > canonicalization in the common case, but even that path is thrwat with
> > danger.
> 
> Manipulating text with all sorts of dynamic buffers is substantially
> more complicated than simply replacing a node in a linked list.
> The canonicalization is all being done by the kernel, or a library.

How could this possibly be in the kernel?  After all, you're talking
about changing the interface to open a file; I pass a file name via
some mechanism to a user level application that wants to call open on
it.  What's it supposed to do?  Does the shell now pass a linked list
as an argument to main somehow?  How does the system know that it's a
file?  Do we have to replace the argument vector with some more complex
representation that encapsulates type information (e.g., this argument
is a file, this next one is a string, etc)?  Does the shell change to
represent file names as lists?  Does the user suffer the indignation of
having to specify a list of path components to represent a file?  Or do
we provide a canonicalization library for shell arguments, in which
case, you have the exact same problem as supporting spaces now, since
most programs are going to expect to get file name arguments in the
canonical representation?  If you do that, who calls it?  The shell or
the library?

I for one am going to be *very* unhappy if I have to type:

	cat ('' 'usr' 'cross' 'file')

Instead of:

	cat /usr/cross/file

Or do you make every program that wants to open a file call a function
to canonicalize a filename into the internal format before it calls
open?

> > But they change an already well-established interface.  Have you
> > thought through the implications of this, in all their macabre glory?
> > What you propose--changing the most basic interface for opening a file
> > in a system where everything looks more or less like a file--has huge
> > implications.  And all this just to support a strange edge-case, which
> > is adequately solved by substitutions in the filename.  Sure, it's not
> > perfect in some weird pathological case, but how often is this going to
> > come up in practice?  Remember: Optimize for the common case.
> 
> Optimization for the common case is good, but creating a system where the
> uncommon case will cause major mayhem at the system level is evidence
> of a very unclean approach.  (When you consider the reasoning behind
> the problem (namely, spaces and slashes in filenames kill our ability
> to seperate nodes easily), it makes perfect sense that our solution
> isn't very clean.  The only clean solution is to restore the ancient
> UNIX ideal of being able to easily seperate nodes.  In other words,
> either kill spaces altogether and damn interoperability, or promote
> spaces to full citizenship.)

But Plan 9 can handle this.

One of the beautiful things about Plan 9 is that it provides a solution
that's workable with little effort.  The various substitution file
systems provide a workable solution without introducing any additional
complexity.  If you want a total--100% complete--solution, then a
`urlifyfs' can be written that uses URL escaping as a canonical
representation, or something similar.  The system interface doesn't
have to be changed, though.  *That* is the mark of a clean system
design.

The Unix `ideal' was eliminated because it was overly complex, without
a commensurate gain in functionality.  Besides, the inode system didn't
really fit in well with the idea of 9p.

> > > There's plenty of experience with other systems working on linked lists
> > > (including a huge amount of kernel code in my Linux box that I'm typing
> > > from, ATM).  Most of the problems with linked lists have been pretty
> > > well documented, by now.
> > 
> > It's the huge amount of kernel code that Plan 9 is trying to avoid.
> 
> String manipulation is more complex than linked list manipulation.

No, it's really not.  Consider passing a linked list as an argument to
a function you're calling, versus passing an argument vector of
strings.  How do you do that?  Do you muck with all the C startup code
to make sure you get the linking and so right in such a way that the
list is in a contiguous memory block so it doesn't get stomped by the
image read by exec?  Do you pass each node in the list to main as a
seperate string in the argument vector?  If so, how do you tell when
a file name ends and another begins?  Do we introduce some convention
for delineating the beginning and ends of a filename in a list
representation, effectively creating a protocol that every program has
to follow to take a filename as an argument?  Surely the former option
is significantly easier....

Consider a possible canonicalization routine that might be used in
a substitution FS:

char *
canonical(char *str)
{
	char	*p, *s, *t;
 
	if (str == nil || (p = malloc(2 * strlen(str) + 1)) == nil) {
		return(nil);
	}
	for (s = str, t = p; *s != '\0'; s++, t++) {
		if (isspace(*s)) {
			*t++ = '+';	/*  Or whatever.  */
			*t = '2';
			continue;
		}
		*t = *s;
	}
	if ((s = realloc(p, strlen(p) + 1)) == nil) {
		free(p);
	}
 
	return(s);
}

That's pretty straight-forward; just inserting into a linked list
would be just as hard.  Doing so in a contiguous memory block would
be, I think harder (you'd have to step over the list, keep a count
of how much memory you needed, then allocate the list, copy each
node and set the links.  That's a pain).

> > Being forced to conform to a lot of external interfaces *will* kill the
> > system.
> 
> I don't dispute that point, but the interface I propose is most unlike
> any other interface currently known to man (not trying to conform to any
> external interface).  I'm simply pointing out that failing to provide
> at least a 1-1 mapping with capabilities that are already widely used
> in external systems that must interoperate with ours *will* kill us.

Well, if you *really* want 100% 1 to 1 mappings, use the URL encoding
others have mentioned, or something similar.  As it is, it seems that
this mostly works; about 80% of what's needed is there.

> > Besides, the point Nemo was trying to make umpteen posts ago was that,
> > yes, you can roll back changes using the dump filesystem, which gives
> > you temporal mobility.  He is right.
> 
> You can do a lot of things if you're prepared to get involved in the
> functions that your OS should be doing automatically.  Try running an FTP
> mirror to a busy site that way, though, and you'll quickly discover why
> automation is a good thing.  The worst part about our system is that the
> "solution" you eventually find for an FTP mirror will be useless on an
> HTTP proxy.  When "solutions" need to be modified for each individual
> application, you know that the system isn't clean.

Yesterday is a wonderful tool, and can be scripted to do whatever you
want.  Eg, copying all files that changed on June 14th back to the
cache isn't very diffcult.

I don't see what running a big FTP mirror has to do with it.  netlib is
a big FTP site; it runs on Plan 9.  Maybe it's not a mirror, but so what?
I also don't see how you can't leverage whatever you did for FTP with
HTTP.  The substitution-style FS gives you a *lot* of flexibility in this
area.

	- Dan C.

Re: [9fans] blanks in file names

[anothy]

on the "linked lists for paths instead of strings with /" topic:
while i can see how this could be interesting to persue, from a
systems architecture research point of view, i'm unclear on how
exactly this helps us solve any of the problems people are
talking about. maybe i'm just missing something; if so, please
clue me in. but here's what i don't see:

first, it seems your suggestion is primarialy aimed at allowing
"/" in file names (perhaps some argument about generality or
interoperation with DOS-derived systems and "\" could be made,
as well). okay, i can see the theoretical benefit there (but it
strikes me as a much more dubious gain than spaces in file
names). but what would this system _look_ like, from a user's
point of view? what would the output of `pwd' be? a list of
strings? but how can i store a representation of that? wouldn't
that involve lots of interaction with the shell, or whatever
else was doing the interpretation (like $path does, although
that's used is very few places, and is very function-specific,
so isn't really much of an issue)?

second, i'm just not seeing at all how this helps us address
spaces in file names (and maybe it wasn't meant to; this
thread has wandered a bit). the dificult part of the problem
(perhaps the entirety of the problem) seems to lie in getting
the various tools that need to communicate to understand the
edges of file names: things like "grep `{ls} foo". i fail to
see how any of the kernel changes (or library additions) you
propose even talk to this realm of problem.

third: want newline too?

on another embedded thread, you wrote:
// You can do a lot of things if you're prepared to get
// involved in the functions that your OS should be doing
// automatically.  Try running an FTP mirror to a busy site
// that way, though, and you'll quickly discover...

that you've completely missed what dan was saying? he (and
Nemo before him) was talking about the ability of Plan 9's
dump to back out changes to kernel, library, and utilitys
uniformly. i don't have a clue what FTP or HTTP mirrors or
"things your OS should do automatically" have to do with
any of this. are we just having unrelated conversations at
each other?
ア

Re: [9fans] blanks in file names

[Dave]

Reply inline:

 - Dave

Dan Cross wrote:
> 
> > The "offending" program in this case is the program that's so much simpler
> > because it's using the new kernel interface rather than the old one.
> 
> I don't think it would be simpler; I think it would be more
> complicated.  You're replacing a simple, textual representation of an
> object with a binary representation; you have to have some way to do
> canonicalization in the common case, but even that path is thrwat with
> danger.
Manipulating text with all sorts of dynamic buffers is substantially
more complicated than simply replacing a node in a linked list.
The canonicalization is all being done by the kernel, or a library.

> 
> > Nothing has to be rethought because all my proposed changes do is restore
> > the strict seperation between nodes in a filename (something UNIX -
> > and therefore Plan 9 - likes to rely on).
> 
> But they change an already well-established interface.  Have you
> thought through the implications of this, in all their macabre glory?
> What you propose--changing the most basic interface for opening a file
> in a system where everything looks more or less like a file--has huge
> implications.  And all this just to support a strange edge-case, which
> is adequately solved by substitutions in the filename.  Sure, it's not
> perfect in some weird pathological case, but how often is this going to
> come up in practice?  Remember: Optimize for the common case.
Optimization for the common case is good, but creating a system where the
uncommon case will cause major mayhem at the system level is evidence
of a very unclean approach.  (When you consider the reasoning behind
the problem (namely, spaces and slashes in filenames kill our ability
to seperate nodes easily), it makes perfect sense that our solution
isn't very clean.  The only clean solution is to restore the ancient
UNIX ideal of being able to easily seperate nodes.  In other words,
either kill spaces altogether and damn interoperability, or promote
spaces to full citizenship.)

> 
> > There's plenty of experience with other systems working on linked lists
> > (including a huge amount of kernel code in my Linux box that I'm typing
> > from, ATM).  Most of the problems with linked lists have been pretty
> > well documented, by now.
> 
> It's the huge amount of kernel code that Plan 9 is trying to avoid.
String manipulation is more complex than linked list manipulation.

> 
> Being forced to conform to a lot of external interfaces *will* kill the
> system.
I don't dispute that point, but the interface I propose is most unlike
any other interface currently known to man (not trying to conform to any
external interface).  I'm simply pointing out that failing to provide
at least a 1-1 mapping with capabilities that are already widely used
in external systems that must interoperate with ours *will* kill us.

> 
> Besides, the point Nemo was trying to make umpteen posts ago was that,
> yes, you can roll back changes using the dump filesystem, which gives
> you temporal mobility.  He is right.
You can do a lot of things if you're prepared to get involved in the
functions that your OS should be doing automatically.  Try running an FTP
mirror to a busy site that way, though, and you'll quickly discover why
automation is a good thing.  The worst part about our system is that the
"solution" you eventually find for an FTP mirror will be useless on an
HTTP proxy.  When "solutions" need to be modified for each individual
application, you know that the system isn't clean.

> 
> 	- Dan C.
> 
 - Dave C. :-)

Re: [9fans] blanks in file names

[arisawa]

Dave wrote:
>If I had to vote a priori, blanks would certainly be a no-go for
>filenames.  However, the pressure cooker has already burst, and our
>...
>keep our chicken from splattering all over the kitchen ;-/

Important or not? That is the problem.
If we agree it is important, we must eat spaes.
Then my chice is to cook spaces so that stomach is not broken.
How to cook?
I believe the first discussion should be forcussed
to expression of path considering in future needs.
I believe you and I share much opinion.

Kenji Arisawa

Re: [9fans] blanks in file names

[Dan Cross]

> The "offending" program in this case is the program that's so much simpler
> because it's using the new kernel interface rather than the old one.

I don't think it would be simpler; I think it would be more
complicated.  You're replacing a simple, textual representation of an
object with a binary representation; you have to have some way to do
canonicalization in the common case, but even that path is thrwat with
danger.

> Nothing has to be rethought because all my proposed changes do is restore
> the strict seperation between nodes in a filename (something UNIX -
> and therefore Plan 9 - likes to rely on).

But they change an already well-established interface.  Have you
thought through the implications of this, in all their macabre glory?
What you propose--changing the most basic interface for opening a file
in a system where everything looks more or less like a file--has huge
implications.  And all this just to support a strange edge-case, which
is adequately solved by substitutions in the filename.  Sure, it's not
perfect in some weird pathological case, but how often is this going to
come up in practice?  Remember: Optimize for the common case.

> There's plenty of experience with other systems working on linked lists
> (including a huge amount of kernel code in my Linux box that I'm typing
> from, ATM).  Most of the problems with linked lists have been pretty
> well documented, by now.

It's the huge amount of kernel code that Plan 9 is trying to avoid.

Being forced to conform to a lot of external interfaces *will* kill the
system.

Besides, the point Nemo was trying to make umpteen posts ago was that,
yes, you can roll back changes using the dump filesystem, which gives
you temporal mobility.  He is right.

	- Dan C.

Re: [9fans] blanks in file names

[Dave]

If I had to vote a priori, blanks would certainly be a no-go for
filenames.  However, the pressure cooker has already burst, and our
chicken is headed straight towards the ceiling.  Getting rid of spaces in
filenames is not much of an option, if we want to be able to get along
in the wide world of non-space-restricted systems.  (One simple way to
realize that s/ /anything/g isn't a very clean option is to note that
different domains are going to require humans to determine how to do the
conversion (and its inverse) and in what contexts to do each of the above.
Maintaining an FTP mirror, for instance, is not going to be easy at all.)
Rethinking our system to fully support blanks, IMHO, is the only way to
keep our chicken from splattering all over the kitchen ;-/

 - Dave


arisawa@ar.aichi-u.ac.jp wrote:
> 
> >Isn't this thread dead yet?
> It seems ...
> So I asked: Is it important?
> So I said: My preference is just s/ /_/g
> So I proposed: uq (unquote)
> Any way, I dislike ' ' in file name.
> 
> Kenji Arisawa
> 

Re: [9fans] blanks in file names

[Dave]

The "offending" program in this case is the program that's so much simpler
because it's using the new kernel interface rather than the old one.

Nothing has to be rethought because all my proposed changes do is restore
the strict seperation between nodes in a filename (something UNIX -
and therefore Plan 9 - likes to rely on).

There's plenty of experience with other systems working on linked lists
(including a huge amount of kernel code in my Linux box that I'm typing
from, ATM).  Most of the problems with linked lists have been pretty
well documented, by now.

 - Dave



FJ Ballesteros wrote:
> 
> 
> 
> Dave ha escrito:
> > 
> > You're missing the point: With the changes I propose to our kernel,
> > certain programs will be able to have much simpler logic.  In addition to
> > undoing the changes in the kernel, you'd have to rewrite the "offending"
> > program - only manual recoding can accomplish that.
> > 
> 
> No offending program if there's no native ' ' in file names. Just
> have to cope with those still seen from the outside world.
> I have a working trfs that deals with most cases here (still will miss
> names
> inside files). I'll send it tomorrow.
> 
> Changing open just because of this would mean to rethink many tools
> and nobody knows (because there's no experience with your proposed
> system)
> if it would introduce other problems. So I'd vote no.
> 

Re: [9fans] blanks in file names

[FJ Ballesteros]

Dave ha escrito:
> 
> You're missing the point: With the changes I propose to our kernel,
> certain programs will be able to have much simpler logic.  In addition to
> undoing the changes in the kernel, you'd have to rewrite the "offending"
> program - only manual recoding can accomplish that.
> 

No offending program if there's no native ' ' in file names. Just
have to cope with those still seen from the outside world.
I have a working trfs that deals with most cases here (still will miss
names
inside files). I'll send it tomorrow.

Changing open just because of this would mean to rethink many tools
and nobody knows (because there's no experience with your proposed
system)
if it would introduce other problems. So I'd vote no.

Re: [9fans] blanks in file names

[arisawa]

>Isn't this thread dead yet?
It seems ...
So I asked: Is it important?
So I said: My preference is just s/ /_/g
So I proposed: uq (unquote)
Any way, I dislike ' ' in file name.

Kenji Arisawa

Re: [9fans] blanks in file names

[David Gordon Hogan]

Isn't this thread dead yet?

Re: [9fans] blanks in file names

[Dave]

You're missing the point: With the changes I propose to our kernel,
certain programs will be able to have much simpler logic.  In addition to
undoing the changes in the kernel, you'd have to rewrite the "offending"
program - only manual recoding can accomplish that.

 - Dave


Fco.J.Ballesteros wrote:
> 
> This is a multi-part message in MIME format.
> --upas-xuojoahyqverhjcrtrjsvosjtu
> Content-Disposition: inline
> Content-Type: text/plain; charset="US-ASCII"
> Content-Transfer-Encoding: 7bit
> 
> Really, I make changes now and then; many times after
> trying the resulting binaries I change my mind and
> use yesterday + cp to restore the source back to its
> previous state. Some other times I bind temporary directories
> on top of the sources and make the changes there.
> 
> 
> --upas-xuojoahyqverhjcrtrjsvosjtu
> Content-Type: message/rfc822
> Content-Disposition: inline
> 
> Received: from mail.cse.psu.edu ([130.203.4.6]) by aquamar; Tue Jul  9 17:22:23 MDT 2002
> Received: from psuvax1.cse.psu.edu (psuvax1.cse.psu.edu [130.203.18.6])
> 	by mail.cse.psu.edu (CSE Mail Server) with ESMTP
> 	id E683A19A1C; Tue,  9 Jul 2002 11:22:09 -0400 (EDT)
> Delivered-To: 9fans@cse.psu.edu
> Received: from mta1.srv.hcvlny.cv.net (mta1.srv.hcvlny.cv.net [167.206.5.4])
> 	by mail.cse.psu.edu (CSE Mail Server) with ESMTP id 437E71998C
> 	for <9fans@cse.psu.edu>; Tue,  9 Jul 2002 11:21:03 -0400 (EDT)
> Received: from dave2.dave.tj (ool-4351482a.dyn.optonline.net [67.81.72.42])
>  by mta1.srv.hcvlny.cv.net
>  (iPlanet Messaging Server 5.2 HotFix 0.3 (built May 13 2002))
>  with ESMTP id <0GYZ00FXXMNYMH@mta1.srv.hcvlny.cv.net> for 9fans@cse.psu.edu;
>  Tue, 09 Jul 2002 11:21:34 -0400 (EDT)
> Received: (from dave@localhost)	by dave2.dave.tj (8.10.2/8.10.2)
>  id g69FNP431828	for 9fans@cse.psu.edu; Tue, 09 Jul 2002 15:23:25 +0000 (GMT)
> From: Dave <dave@dave.tj>
> Subject: Re: [9fans] blanks in file names
> In-reply-to: <c55e55e304beec4b477c71af7fa9c712@plan9.escet.urjc.es> from
>  <Fco.J.Ballesteros@Jul>
> To: 9fans@cse.psu.edu
> Message-id: <200207091523.g69FNP431828@dave2.dave.tj>
> MIME-version: 1.0
> X-Mailer: ELM [version 2.5 PL3]
> Content-type: text/plain; charset=us-ascii
> Content-transfer-encoding: 7BIT
> Sender: 9fans-admin@cse.psu.edu
> Errors-To: 9fans-admin@cse.psu.edu
> X-BeenThere: 9fans@cse.psu.edu
> X-Mailman-Version: 2.0.11
> Precedence: bulk
> Reply-To: 9fans@cse.psu.edu
> List-Id: Fans of the OS Plan 9 from Bell Labs <9fans.cse.psu.edu>
> List-Archive: <https://lists.cse.psu.edu/archives/9fans/>
> Date: Tue, 09 Jul 2002 11:23:25 -0400 (EDT)
> 
> You're not going to do that every time you make a change to the
> filesystem.  Besides, that won't undo all the "changes" to all the
> new programs that lack complexity because they don't have to reinvent
> the wheel.
> 
>  - Dave
> 
> 
> Fco.J.Ballesteros wrote:
> > 
> > >> But above all, I will undo the changes made in this respect to my
> > >> local system if you guys or the system designers choose a different way.
> > ...
> > > Undoing kernel-level changes won't be easy, especially when people start
> > 
> > 9fs dump
> > cp blah blah
> > 
> > Sorry, couldn't resist. I just love this system :-)
> > 
> 
> --upas-xuojoahyqverhjcrtrjsvosjtu--
> 

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

[-- Attachment #1: Type: text/plain, Size: 277 bytes --]

Really, I make changes now and then; many times after
trying the resulting binaries I change my mind and
use yesterday + cp to restore the source back to its
previous state. Some other times I bind temporary directories
on top of the sources and make the changes there.


[-- Attachment #2: Type: message/rfc822, Size: 2260 bytes --]

From: Dave <dave@dave.tj>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] blanks in file names
Date: Tue, 09 Jul 2002 11:23:25 -0400 (EDT)
Message-ID: <200207091523.g69FNP431828@dave2.dave.tj>

You're not going to do that every time you make a change to the
filesystem.  Besides, that won't undo all the "changes" to all the
new programs that lack complexity because they don't have to reinvent
the wheel.

 - Dave


Fco.J.Ballesteros wrote:
> 
> >> But above all, I will undo the changes made in this respect to my
> >> local system if you guys or the system designers choose a different way.
> ...
> > Undoing kernel-level changes won't be easy, especially when people start
> 
> 9fs dump
> cp blah blah
> 
> Sorry, couldn't resist. I just love this system :-)
> 

Re: [9fans] blanks in file names

[Dave]

Whoops ... I misspelled that one.  A more appropriate comment for you,
though, would have been to put it in libarms ;-P

 - Dave


Boyd Roberts wrote:
> 
> Dave wrote:
> 
> >It's possible that cannonization should still be done by library routines,
> >
> Yes, put it in libpope.
> 
> 

Re: [9fans] blanks in file names

[Dave]

You're not going to do that every time you make a change to the
filesystem.  Besides, that won't undo all the "changes" to all the
new programs that lack complexity because they don't have to reinvent
the wheel.

 - Dave


Fco.J.Ballesteros wrote:
> 
> >> But above all, I will undo the changes made in this respect to my
> >> local system if you guys or the system designers choose a different way.
> ...
> > Undoing kernel-level changes won't be easy, especially when people start
> 
> 9fs dump
> cp blah blah
> 
> Sorry, couldn't resist. I just love this system :-)
> 

Re: [9fans] blanks in file names

[Boyd Roberts]

Dave wrote:

>It's possible that cannonization should still be done by library routines,
>
Yes, put it in libpope.

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

>> But above all, I will undo the changes made in this respect to my
>> local system if you guys or the system designers choose a different way.
...
> Undoing kernel-level changes won't be easy, especially when people start

9fs dump
cp blah blah

Sorry, couldn't resist. I just love this system :-)

Re: [9fans] blanks in file names

[okamoto]

>if you are writing in english, not necessarily. yes if you are writing in
>japanese. in that case, it should have seventeen (five/seven/five) *onji*
>which is not a syllable, but a sound symbol. 

Aa!  Now, I understood the difference between 俳句 and Haiku.
I'll show you a 俳句example, which is one of the most famous one
written for this season by 芭蕉(Bashou) posted before.

古池や蛙とびこむ水の音

Abobe is written using Hiragana and Kanji, which is Japanese writing
style heritaged from Heian-Ara invented by Japanese women..., anyway
it should be read as

ふるいけや かわず とびこむ みずのおと

Now, you counts 5+7+5 hiraganas, which are the reading showtest units
(syllable?) in Japanese.  Here, I intentianlly put 'blank' between 5+7+5,
but it not used formally.

Another thing, in 俳句 we need some key word which implies a season
when the author want to express.   In the above example, it's 蛙(frog).
Frogs apear to us from early summer.  So, I felt that an English version
of Haiku which someone showed here is not 俳句, but 川柳(senryu).

Sorry making noise.

Kenji -- Enjoying UTF-8 capability of Plan 9

Re: [9fans] blanks in file names

[Dan Cross]

> IBM did that with their match-mode mainframe operating system.  Low and
> behold, it didn't change for 50 years.

Err, make that batch-mode.  Whoops.

	- Dan C.

Re: [9fans] blanks in file names

[Dave]

I believe those types of cannons are spelled with two Ns.

 - Dave


David Gordon Hogan wrote:
> 
> >> I thought the making of saints wasn't exactly in the kernel's job
> >> description.
> > 
> > Naw, making saints is "canonization".  Making artillery is a perfectly
> > good kernel task.
> 
> Such kernels may, however, be subject to export restrictions...
> 

Re: [9fans] blanks in file names

[Dan Cross]

> BTW - If we change the system in a fundamental way once and for all,
> we'll have a system (and a new set of conventions) that won't have to
> be changed for another 50 years, with any luck.  Somebody's going to
> want to encode full pathnames within nodes at some point in the future,
> and our decision to disallow that will force people to invent roundabout
> ways of doing so.

IBM did that with their match-mode mainframe operating system.  Low and
behold, it didn't change for 50 years.

	- Dan C.

Re: [9fans] blanks in file names

[David Gordon Hogan]

>> I thought the making of saints wasn't exactly in the kernel's job
>> description.
> 
> Naw, making saints is "canonization".  Making artillery is a perfectly
> good kernel task.

Such kernels may, however, be subject to export restrictions...

Re: [9fans] blanks in file names

[Berry Kercheval]

Boyd Roberts <boyd@strakt.com> writes:

> Dave wrote:
>  > The method of representation of a given node can be quite flexible
>  > (allowing node completion - or as I would term it, cannonization
>  > - to be done by the kernel),
> 
> I thought the making of saints wasn't exactly in the kernel's job
> description.


Naw, making saints is "canonization".  Making artillery is a perfectly
good kernel task.

Re: [9fans] blanks in file names

[Dave]

Okay, I see where you're going with this.  I fear I'm very badly
outnumbered here, and since I'm not about to implement the changes I
suggest, nobody else is likely to.  I guess we'll just have to keep
blanks out of our filenames. . .

 - Dave

BTW - If we change the system in a fundamental way once and for all,
we'll have a system (and a new set of conventions) that won't have to
be changed for another 50 years, with any luck.  Somebody's going to
want to encode full pathnames within nodes at some point in the future,
and our decision to disallow that will force people to invent roundabout
ways of doing so.


rob pike, esq. wrote:
> 
> > changing open(2), execve(2), etc. is
> > a requirement if we want a fundamental solution to the problem. 
> 
> Fundamental solutions should be applied only to fundamental problems.
> This is not a fundamental problem; far from it.  The means should be
> proportionate to the ends.
> 
> > > Don't you think that changing open, considering '/' in names, and
> > > similar stuff is just too much? 
> 
> Yes, it is, because to do so contradicts too many conventions that are
> integral to the system.
> 
> -rob
> 

Re: [9fans] blanks in file names

[Dave]

Plan 9 still has identifier numbers provided by the filesystem server.
We can toss or change those at some point in the future (increasing the
identifier's size, for instance, or adding a supplementary identifier)
if we identify files using linked structures.

 - Dave


forsyth@caldo.demon.co.uk wrote:
> 
> >>Finally, it gives us the capability of getting away from even the most
> >>elementary requirements in a filesystem (like inodes) at some point in
> >>the future without extensive code changes.  Basically, it's all about
> 
> Plan 9 hasn't really got assumptions about inodes.  9P doesn't use them.
> (makes it tricky to do NFS, as it happens, though that's not Plan 9's fault
> this time.)
> 

Re: [9fans] blanks in file names

[Dave]

It's possible that cannonization should still be done by library routines,
but I'd vote to let the kernel do it anyway for whatever optimizations
it can make (since it's not restricted to the standard syscalls for
file handling).

 - Dave


Boyd Roberts wrote:
> 
> Dave wrote:
>  > The method of representation of a given node can be quite flexible
>  > (allowing node completion - or as I would term it, cannonization
>  > - to be done by the kernel),
> 
> I thought the making of saints wasn't exactly in the kernel's job 
> description.
> 
> 

Re: [9fans] blanks in file names

[rob pike, esq.]

> changing open(2), execve(2), etc. is
> a requirement if we want a fundamental solution to the problem. 

Fundamental solutions should be applied only to fundamental problems.
This is not a fundamental problem; far from it.  The means should be
proportionate to the ends.

> > Don't you think that changing open, considering '/' in names, and
> > similar stuff is just too much? 

Yes, it is, because to do so contradicts too many conventions that are
integral to the system.

-rob

Re: [9fans] blanks in file names

[Dave]

Reply inline:

 - Dave

Fco.J.Ballesteros wrote:
> 
> I think we've lost focus. I'm scared.
> 
> The whole point was to handle characters which are allowed but may
> cause problems like ' ' and ''''.  And AFAIK, the problem was that we
> get them from outside systems, and that people may be so used to them
> that they may even start to use them on native Plan 9 files.
The reason that UNIX is so good and continues to be successful even
competing against the biggest software company in the world is that
UNIX has never taken this approach to solving problems; UNIX solves
problems in a fundamental way, and Plan 9 must, too, if it's to be a
worthy successor to UNIX.  AFAIK, changing open(2), execve(2), etc. is
a requirement if we want a fundamental solution to the problem.  IMHO,
the kernel should have an even more fundamental solution (seperation
of the path into nodes, each of which is identified by a structure),
while applications (which require a plaintext representation of a path)
will probably want to escape special characters (by whichever method
they want - since the kernel needs no escaping within itself, apps are
free to do whatever they want).

> 
> Don't you think that changing open, considering '/' in names, and
> similar stuff is just too much?  I'm scared.  That may be interesting,
> but it would lead to a very different system.
It won't lead to a very different system from the application programmer's
point of view.  It'll lead to a different system under the hood, which
is exactly what we want, because what we currently have under the hood
simply doesn't cut it.

> 
> Moreover, do you think that the system designers would ever consider
> '/' as a legitimate character within file names?  Although I don't
> know, I'd bet they'll never do that (at least I have to say I would
> never do it, sorry).
There's no reason not to allow the slash in node names, except that it
confuses utilities.  However, whitespace already confuses utilities in
the same way, so we lose nothing by allowing slashes in node names.

> 
> I think I'm just going to try option bⁱ myself and then send a diff
> in case it works out.
Option b is a userland option, not a kernel option.  All you're doing
is deferring the decision to change our kernel until a later time (by
which point we'll hopefully have a clearer idea about how we want to
change the kernel).

> 
> But above all, I will undo the changes made in this respect to my
> local system if you guys or the system designers choose a different way.
> It's a very nice system and I wouldn't like to get N different ones
> nor to break it.
Undoing kernel-level changes won't be easy, especially when people start
coding programs using the kernel's new path representation; rewriting
them using standard strings would (a) make the programs' logic a lot
more complicated; and consequentally (b) make the programs a lot less
efficient.

> 
> 
> -- 
> ⁱ Option b was: remove quoting from ls et al, add it to those that
> print commands, and fix those that don't cosider ' ' in file names.
> 

Re: [9fans] blanks in file names

[forsyth]

>>Finally, it gives us the capability of getting away from even the most
>>elementary requirements in a filesystem (like inodes) at some point in
>>the future without extensive code changes.  Basically, it's all about

Plan 9 hasn't really got assumptions about inodes.  9P doesn't use them.
(makes it tricky to do NFS, as it happens, though that's not Plan 9's fault
this time.)

Re: [9fans] blanks in file names

[Boyd Roberts]

Dave wrote:
 > The method of representation of a given node can be quite flexible
 > (allowing node completion - or as I would term it, cannonization
 > - to be done by the kernel),

I thought the making of saints wasn't exactly in the kernel's job 
description.

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

I think we've lost focus. I'm scared.

The whole point was to handle characters which are allowed but may
cause problems like ' ' and ''''.  And AFAIK, the problem was that we
get them from outside systems, and that people may be so used to them
that they may even start to use them on native Plan 9 files.

Don't you think that changing open, considering '/' in names, and
similar stuff is just too much?  I'm scared.  That may be interesting,
but it would lead to a very different system.

Moreover, do you think that the system designers would ever consider
'/' as a legitimate character within file names?  Although I don't
know, I'd bet they'll never do that (at least I have to say I would
never do it, sorry).

I think I'm just going to try option bⁱ myself and then send a diff
in case it works out.

But above all, I will undo the changes made in this respect to my
local system if you guys or the system designers choose a different way.
It's a very nice system and I wouldn't like to get N different ones
nor to break it.


-- 
ⁱ Option b was: remove quoting from ls et al, add it to those that
print commands, and fix those that don't cosider ' ' in file names.

Re: [9fans] blanks in file names

[Scott Schwartz]

| If '/' is prohibitted as an element of file name and directory name,
| then no change to open is required.
| 
| Let's assume we accept '/' as an element of names,
| then how do you express path in rc?
 
cd (stl list/vector foo)

I think this discussion is verging on reinventing lisp, where you have a
well specified format for textually representing data, standard primatives
for reading and writing it, and a set of datatypes to store it in.

Re: [9fans] blanks in file names

[Dave]

Oops ... I guess I replied a little too early ;-/

 - Dave


arisawa@ar.aichi-u.ac.jp wrote:
> 
> >new_open is not reqired.
> I should have added:
> But I prefer new_open for kernel interface.
> 
> Kenji Arisawa
> 

Re: [9fans] blanks in file names

[Dave]

The idea behind our new open(2) is to create a cleaner representation
for paths.  If we want, we can patch the old open(2) - which could
become a library function open2(3) - to support one or more of the
userland naming conventions we were discussing.  However, I don't think
the kernel should meddle with userland naming conventions; having the
kernel deal with binary data in linked lists is by far the cleanest
approach proposed thus far, IMHO.  It also opens the door to all sorts
of neat possibilities for path manipulation without string manipulation.
Finally, it gives us the capability of getting away from even the most
elementary requirements in a filesystem (like inodes) at some point in
the future without extensive code changes.  Basically, it's all about
flexibility, a long-time strength of the UNIX system which has only been
improved upon in Plan 9.

 - Dave


arisawa@ar.aichi-u.ac.jp wrote:
> 
> Another escape may be a % of that is used in URI.
> 
> %	->	%25
> /	->	%2f
> space	->	%20
> 
> and if open can accept these, new_open is not reqired.
> 
> Kenji Arisawa
> 

Re: [9fans] blanks in file names

[arisawa]

>new_open is not reqired.
I should have added:
But I prefer new_open for kernel interface.

Kenji Arisawa

Re: [9fans] blanks in file names

[arisawa]

Another escape may be a % of that is used in URI.

%	->	%25
/	->	%2f
space	->	%20

and if open can accept these, new_open is not reqired.

Kenji Arisawa

Re: [9fans] blanks in file names

[Geoff Collyer]

I had thought that one of my sicker ideas was too ridiculous to
suggest, but perhaps not, given the precedent:

A pair of kernel devices (or a single device with an encode or decode
indicator) to translate hURL-encoded file names into plan 9 file
names, and vice versa.  After translation, they are passed to namec.
This lets you use all the glorious botches that have been invented for
hURL (or is it URI?) encoding.

	#:dfile://localhost/my%20file
and
	#:dmy%20file

would map to "my file".  In the presence of a file server that
understands the encoded form, one could do the reverse mapping:
"#:emy file" should probably map to

	my%20file

A further benefit of using the encoded form of names in files is that
programs can then guess fairly reliably which are file names: a field
starting with "file://localhost/" is likely to be a file name.


I'm not seriously suggesting using awful web syntax, but perhaps the
general idea suggests a way forward.

Re: [9fans] blanks in file names

[Dave]

For userland programs, I'd probably vote to simply escape "interesting"
characters that occur within filenames:

1) /bin/awk
2) \/bin\/awk
3) /bin\/awk
4) \/bin/awk

We'd still have to annoy sed, awk, sort, etc. with this approach, but I
don't think that's too bad.  (If we really want, we can establish escape
sequences that don't have these chars in them: e.g., \s for slash, \_
for space, etc.)

Just my penny pair,
Dave Cohen <dave@dave.tj>


arisawa@ar.aichi-u.ac.jp wrote:
> 
> If '/' is prohibitted as an element of file name and directory name,
> then no change to open is required.
> 
> Let's assume we accept '/' as an element of names,
> then how do you express path in rc?
> 
> /bin/awk may be :
> 1. /bin/awk as it has been.
> 2. a file named "/bin/awk" in current directory.
> 3. a file named "bin/awk" in "/"
> 4. a file named "awk" in "/bin" and "/bin" is in current directory.
> 
> Kenji Arisawa
> 

Re: [9fans] blanks in file names

[arisawa]

If '/' is prohibitted as an element of file name and directory name,
then no change to open is required.

Let's assume we accept '/' as an element of names,
then how do you express path in rc?

/bin/awk may be :
1. /bin/awk as it has been.
2. a file named "/bin/awk" in current directory.
3. a file named "bin/awk" in "/"
4. a file named "awk" in "/bin" and "/bin" is in current directory.

Kenji Arisawa

Re: [9fans] blanks in file names

[Dave]

How about representing paths internally as connected structures (linked lists, if you will), each of which identifies a particular node.  The method of representation of a given node can be quite flexible (allowing node completion - or as I would term it, cannonization - to be done by the kernel), and user programs could establish their own conventions for dealing with user input and user output (which seems logical to me, since the kernel really isn't intended to be a human user interface, only a program "user" interface).

Here's an off-the-top-of-my-head example to illustrate what I mean.  I'm sure we can come up with a much better system if we all think about this a bit, and figure out how it can be improved (or how it can be replaced with something even better - the purpose of this example is simply to get us thinking in a particular direction that I believe is quite promising):

struct node_t {
 char* name; size_t name_l; /* the node name, and the length of the name */
 /* An error should probably be returned by the kernel if hint is ambiguous. */
 char* hint; /* a regexp that can be fed to attempt to "autocomplete" */
 size_t inode; /* another unique identifier for a node - should we allow it? */
 node_t* next_node;
};

/* canonize(2) converts a node_t chain into its canonical form, expanding all
 *  hints into the corresponding names.  If there are any possible
 *  canonizations and dst is not NULL, dst is filled with node_t chains
 *  representing all the possible canonizations of the node_t chain given.
 * PARAMETERS
 *  src is the node_t chain that you're attempting to canonize.  It's probably
 *   obtained by parsing the input to our shell, and deciding how to interpret
 *   it.
 *  On calling canonize(2), dst should point to a memory block large enough to
 *   accomodate *dst_l node_t structs.  canonize(2) will use the memory block to
 *   store all the nodes of all the node_t chains it creates.
 *  On return from canonize(2), dst_l will be changed to the number of node_t
 *   chains representing possible canonizations (i.e., it's a value-result
 *   parameter), and the first dst_l locations in dst will contain the starting
 *   nodes of the node_t chains representing the possible canonizations.
 * RETURN VALUE
 *  0 on success
 *  -1 on failure, with errno set appropriately
 *  Multiple matches is counted as an error condition, since canonize(2) failed
 *   to canonize the node_t chain.
 *  If dst is too small, dst will be filled as much as possible, but an error
 *   will still be returned.
 * NOTES
 *  Having the self-referencing pointer in node_t refer to the previous node
 *   instead of the next node would allow canonize(2) to save lots of buffer
 *   space in dst for ambiguities that occur deep inside a long node_t chain.
 *   However, I don't believe we should do that, since (a) ambiguities that
 *   occur relatively early in a long node_t chain allow canonize(2) to save
 *   lots of buffer space in dst in the current implementation; and more
 *   importantly, (b) any other routine in the kernel that deals with node_t
 *   chains will have to walk the entire linked list before processing anything
 *   if we implement that change, since the pointer it'll see won't be to the
 *   start of the "pathname," but rather to the end.
 */
int canonize (const node_t* src, node_t dst[], size_t* dst_l);

int new_open (const node_t* path, int flags, int mode);
/* The old open(2) can instead become open(3), with a rather trivial
 *  implementation:
 * int open (const char* path, int flags, int mode) {
 *  split path on unescaped '/' chars;
 *  create a node_t for each of the components obtained above;
 *  link the nodes together into a chain;
 *  return openv(our_node_t_chain,flags,mode);
 * };
 */

/* Here's one final example - the reason I resist the name openv(2): */
int new_execve (const node_t* path, char* const argv[], char* const envp[]);
/* The "v" here refers to a different aspect of the function.  I don't like
 *  overloading the meaning, because we'd have to call our new execve(2)
 *  execvev(2) or something, and that'd be a little insane, IMHO.  That's why I
 *  chose the "new_" prefix.
 * In reality, I'd much rather see the old open(2) be renamed to old_open(2),
 *  and have the new_open(2) be renamed to just plain open(2), damning backward
 *  compatibility (and likewise with the exec(2) family, as well as the stat(2)
 *  call, etc.).  Programs that expect open(2) to be the old open and don't feel
 *  like asking their authors for a rewrite can simply use a runtime library to
 *  rename old_open(2) to open(2) and hide our new open(2) entirely (since
 *  they're obviously not going to use it, anyway).  However, I'm afraid this
 *  type of drastic change may catch some people off-guard.
 */

Dave Cohen <dave@dave.tj>


rob pike, esq. wrote:
> 
> You guys are all arguing about system stuff but it's the
> *user interface* that you're really arguing about, and
> breaking.  You are opening a can of worms you will never
> get closed again.  Change space! Change the file delimiter!
> The shell will never recover.  The system will break.
> I will mourn.
> 
> -rob
> 

Re: [9fans] blanks in file names

[Mark Bitting]

On Friday 05 July 2002 15:21, you wrote:
> >>>> Change space! Change the file delimiter!
> >>>> The shell will never recover.  The system will break.
> >>>> I will mourn.
> >>>
> >>> Isn't a haiku supposed to have 17 syllables?
> >>
> >> Some would dispute the imposition of such an absolute rule.
> >
> > Is that why your message has only 16 syllables?
>
> Word.

To write a Haiku 
of exactly seventeen syllables
is diff.

Re: [9fans] blanks in file names

[Jim Choate]

On Fri, 5 Jul 2002, David Gordon Hogan wrote:

> >>>> Change space! Change the file delimiter!
> >>>> The shell will never recover.  The system will break.
> >>>> I will mourn.
> >>> 
> >>> Isn't a haiku supposed to have 17 syllables?
> >> 
> >> Some would dispute the imposition of such an absolute rule.
> > 
> > Is that why your message has only 16 syllables?
> 
> Word.

Actually only one style of Hiaku uses 17 'syllables' in Japanese. Not the
same thing as English 'syllables'. For an excellent discussion on this
point, and how to compensate for it...

The Haiku Handbook: How to write, share, and teach Haiku
W.J. Higginson, P. Harter
ISBN 4-7700-1430-9
$13 US

Enjoy!


 --
    ____________________________________________________________________

              When I die, I would like to be born again as me.

                                            Hugh Hefner
     ravage@ssz.com                                         www.ssz.com
     jchoate@open-forge.org                          www.open-forge.org

    --------------------------------------------------------------------

Re: [9fans] blanks in file names

[David Gordon Hogan]

>>>> Change space! Change the file delimiter!
>>>> The shell will never recover.  The system will break.
>>>> I will mourn.
>>> 
>>> Isn't a haiku supposed to have 17 syllables?
>> 
>> Some would dispute the imposition of such an absolute rule.
> 
> Is that why your message has only 16 syllables?

Word.

Re: [9fans] blanks in file names

[Sape Mullender]

>>> Change space! Change the file delimiter!
>>> The shell will never recover.  The system will break.
>>> I will mourn.
>> 
>> Isn't a haiku supposed to have 17 syllables?
> 
> Some would dispute the imposition of such an absolute rule.

Is that why your message has only 16 syllables?

Re: [9fans] blanks in file names

[David Gordon Hogan]

>> Change space! Change the file delimiter!
>> The shell will never recover.  The system will break.
>> I will mourn.
> 
> Isn't a haiku supposed to have 17 syllables?

Some would dispute the imposition of such an absolute rule.

Re: [9fans] blanks in file names

[okamoto]

古池や蛙飛び込む水の音

furu ike ya kawazu tobikomu mizuno oto

Hmmm, it seems to be too long than 17 characters. :-)
Sorry ;_;

Kenji

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

[-- Attachment #1: Type: text/plain, Size: 702 bytes --]

I'm sorry if in any of my previous mails I did sound
like `Rob did it wrong' or something like that.
I didn't mean that at all, and apologize if that
was what I actually wrote.

I think we all now agree that
-> translating in the mount driver is a bad idea.
-> translating in the servers might work, but we're not sure.
-> %q is a good thing, but,
-> ls et al shouldn't use %q (unless other programs change their conventions
  to read file names and digest the quotes properly).

I'm sorry my diff triggered this, but in any case I think we
all are trying to find a way, and more discussions like this will
probably follow.

thanks you all; this has been instructive, at least for me.

[-- Attachment #2: Type: message/rfc822, Size: 1393 bytes --]

From: forsyth@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] blanks in file names
Date: Thu, 4 Jul 2002 13:20:54 +0100
Message-ID: <20020704122031.530FF19981@mail.cse.psu.edu>

>>So we need an uniform set of conventions, and make all the programs
>>obey them. And that's not the case right now. That's all I was saying
>>regarding %q.

yes, but to be fair, i believe that is the `making the effort' (if i remember the phrase correctly)
that rob mentioned in a previous message!

Re: [9fans] blanks in file names

[forsyth]

>>So we need an uniform set of conventions, and make all the programs
>>obey them. And that's not the case right now. That's all I was saying
>>regarding %q.

yes, but to be fair, i believe that is the `making the effort' (if i remember the phrase correctly)
that rob mentioned in a previous message!

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

:  i don't think %q per se is a bad thing, but i don't think filenames
:  should require it and hence i don't think tools like ls, pwd, etc
:  should output filenames using it.

agree

Re: [9fans] blanks in file names

[rog]

> Perhaps all this discussion is a symptom that a different convention to
> name files is needed. For example, if file names were always quoted
> (need they to be or not), all programs could rely on a simple set of
> rules to handle file names.
>
> Is there agreement on this? Or is something else I'm also missing?

i don't think %q per se is a bad thing, but i don't think filenames
should require it and hence i don't think tools like ls, pwd, etc
should output filenames using it.

then we're back to where we started (a comfortable place to be IMHO)
except that we have a nice, easily available convention for
serialising arguments containing spaces, and we can talk to alien
filesystems that expect spaces in their names.

  rog.

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

[-- Attachment #1: Type: text/plain, Size: 169 bytes --]

So we need an uniform set of conventions, and make all the programs
obey them. And that's not the case right now. That's all I was saying
regarding %q.

thanks



[-- Attachment #2: Type: message/rfc822, Size: 2227 bytes --]

From: forsyth@caldo.demon.co.uk
To: 9fans@cse.psu.edu
Subject: Re: [9fans] blanks in file names
Date: Thu, 4 Jul 2002 10:41:48 +0100
Message-ID: <30a861a9c8a7a63c0e69d46049d70908@caldo.demon.co.uk>

>>Is there agreement on this? Or is something else I'm also missing?

not exactly.  i think there's an implication though about the uniform
implementation of quoting rules throughout the system so that
	read(filename)
	open(dirname + "/" + filename)
would work because `filename' and `dirname' etc haven't any quotes,
because lines read from standard input (say) have been parsed.
some programs do so already but the conventions vary from place to place.
essentially, strings inside programs are in their parsed form and
strings read and written by a program are expected to be suitably quoted.
thus
	ls | mumble
would work because mumble will apply the standard rules (might be as simple
as calling tokenize consistently) to each line of its input, and thus unquote it
to reveal the original file names.

i say it's not just file names particularly; for instance, input to a program might
be fields
	when where howmuch why who
allowing
	15/9/1660 London 4d 'First Cup of Tea' 'Saml Pepys'
and with suitable changes comm, sort, etc. could be applied in obvious ways.

Re: [9fans] blanks in file names

[forsyth]

>>Is there agreement on this? Or is something else I'm also missing?

not exactly.  i think there's an implication though about the uniform
implementation of quoting rules throughout the system so that
	read(filename)
	open(dirname + "/" + filename)
would work because `filename' and `dirname' etc haven't any quotes,
because lines read from standard input (say) have been parsed.
some programs do so already but the conventions vary from place to place.
essentially, strings inside programs are in their parsed form and
strings read and written by a program are expected to be suitably quoted.
thus
	ls | mumble
would work because mumble will apply the standard rules (might be as simple
as calling tokenize consistently) to each line of its input, and thus unquote it
to reveal the original file names.

i say it's not just file names particularly; for instance, input to a program might
be fields
	when where howmuch why who
allowing
	15/9/1660 London 4d 'First Cup of Tea' 'Saml Pepys'
and with suitable changes comm, sort, etc. could be applied in obvious ways.

Re: [9fans] blanks in file names

[Nikolai SAOUKH]

On Thu, Jul 04, 2002 at 09:39:56AM +0200, Lucio De Re wrote:

| Mapping filenames
| to unique 27-character UTF8 strings with certain restrictions should
| be possible through such a shim.

27-octet utf8 strings, to be precise. In characters it is much less :-)

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

> it would be good to have some simple but consistent conventions for quoting data,
> not particularly file names.  it's true that it would probably affect many commands
> (sort, sed, awk etc) but i think it might be worthwhile.

I'd like the file names to be always the same. What I don't like about
%q is that some times you get /a/b and other times you get '/a/b c'.

This is a serious problem IMHO, since the "'" would make open fail.

I'd like

	read(filename)
	open(filename)

and
	read(filename)
	open(dirname + "/" + filename)


to keep on working, which is no longer the case after %q. (Didn't check
naming in the kernel recently, but I think these calls would fail).

Perhaps all this discussion is a symptom that a different convention to
name files is needed. For example, if file names were always quoted
(need they to be or not), all programs could rely on a simple set of
rules to handle file names.

Is there agreement on this? Or is something else I'm also missing?

Re: [9fans] blanks in file names

[Ralph Corderoy]

> term% rm 'some files'
> removes some_files

Hark!  The scent of DWIM.

    http://www.tuxedo.org/~esr/jargon/html/entry/DWIM.html


Ralph.

Re: [9fans] blanks in file names

[forsyth]

>>:  should be the character in the name, the quoting approach seems
>>:  the only one that works properly, and we have some work to do
>>:  to work out all the interactions with shell scanning (oh dear!), and changing

>>But it's not just the shell, almost *any* program reading file names would
>>have to deal with quoting. What has been a file name is changing after %q,

you stopped the quote just a little early:
	to work out all the interactions with shell scanning (oh dear!), and changing
 ->	(i suspect) the input conventions of quite a few commands, but still, it's a finite task.
but i ought to have said `the input [and output] conventions ...'

it would be good to have some simple but consistent conventions for quoting data,
not particularly file names.  it's true that it would probably affect many commands
(sort, sed, awk etc) but i think it might be worthwhile.

it's something that XML can't do properly for instance.
just amazing.  all that effort for a revolution, and botched at the start.

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

That's important since I may have names like
'a b' and 'a_b', both legal, and your translation
would get into throuble here.

> Hello,
> 
>>You can't do that becaause if a file contains a genuine
>>underscore, the outgoing software will think it's a space.
> the outgoing software will also think it's a underscaore.
> Is this important?
> Some OSs already map 'Windows' to 'WINDOWS'.
> 
> Kenji Arisawa

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

forsyth:
:  the problem rob alluded to, by analogy with NAT, should not
:  arise within the Plan 9 system.  for instance, if i have a file
:  of file names, can i read it in and be sure to access those names?
:  if space is _ and _ is `boo!' it's anyone's guess.

But space is not _, space is space.
As I said in a previous mail, if you have a file of file names,
you can still read and use it. AFAIK, you can be sure to access those
names.

Nevertheless, I can understand your arguments for doing it at
boundaries and not at the kernel.

:  should be the character in the name, the quoting approach seems
:  the only one that works properly, and we have some work to do
:  to work out all the interactions with shell scanning (oh dear!), and changing

But it's not just the shell, almost *any* program reading file names would
have to deal with quoting. What has been a file name is changing after %q,
before that, a program would expect something like [/]a/b/..., that's no longer
the case.

Re: [9fans] blanks in file names

[Lucio De Re]

On Thu, Jul 04, 2002 at 07:34:25AM +0100, forsyth@caldo.demon.co.uk wrote:
> 
> the problem rob alluded to, by analogy with NAT, should not
> arise within the Plan 9 system.  for instance, if i have a file
> of file names, can i read it in and be sure to access those names?
> if space is _ and _ is `boo!' it's anyone's guess.
> 
I think we're going somewhere <grin>

We already have shims between the conventional Plan 9 filesystem
and foreign ones.  These are our declared boundaries and it is
fairly clear what translations are required at each of these
boundaries.

It seems to me that Nemo is - perhaps unintentionally, certainly
not in an explicit fashion - creating an inner boundary that declares
the core of Plan 9 much more liberal and a shim is therefore
necessary so that the existing legacy applications can interact
with it.

I would vote for such a feature.  The kernel and 9P would be freed
of filename shackles, the shim would apply the existing rules.
Imagine, for example, a shim that would enable a 2nd Edition
application to access the 4th Edition filesystem.  Mapping filenames
to unique 27-character UTF8 strings with certain restrictions should
be possible through such a shim.

Even the choice of delimiter between path components would be
available to those willing to produce the necessary shim.

++L

PS: while checking the above, I was wondering where %q belongs in
that model.  I would think the shim must somehow provide the actual
quote and dequote operations.  Hm, maybe such a shim can also add
descriptive names and, shudder, access control lists in a Plan 9
friendly form.

Re: [9fans] blanks in file names

[forsyth]

[-- Attachment #1: Type: text/plain, Size: 3315 bytes --]

the advantage of pushing stuff like this back to the boundaries,
the points at which Plan 9 interacts with other systems and formats
(often using 9P), rather than building it into the heart of Plan 9, is that
at least the Plan 9 system itself can have straightforward internal rules.
at the boundaries you're bound to be aware that there
are differences (in practice).  for instance, you can create
names via dossrv that Windows itself cannot access properly
(left as an exercise for the reader), but aren't prohibited by
the FAT32 specification, or indeed, documented accurately
anywhere.  it's only a little better with 9660srv.

the problem rob alluded to, by analogy with NAT, should not
arise within the Plan 9 system.  for instance, if i have a file
of file names, can i read it in and be sure to access those names?
if space is _ and _ is `boo!' it's anyone's guess.

it's more understandable if, when problems do arise, they appear only at
the boundaries with other systems, where there are necessarily
some differences (remembering of course that any given difference might or
might not be strictly `necessary').

so far, it seems to me:

if Plan 9 allows space directly in the storage names, then space
should be the character in the name, the quoting approach seems
the only one that works properly, and we have some work to do
to work out all the interactions with shell scanning (oh dear!), and changing
(i suspect) the input conventions of quite a few commands, but still, it's a finite task.
the possible scale of that task--even understanding all the implications--
can i think be underestimated, which is why i snarl a bit at
people who are incredibly glib about it [which rob is not].

otherwise, Plan 9 should prohibit space in its file names, as
previously, and the user interface can use something like U+00A0 to
provide the functionality that at least some humans seem to think is
essential.  (computers don't give a dam whether names have spaces or
not.) at the boundaries with Windows or Mac or Unix that character can
be mapped to and fro, but there is already mapping of various sorts
taking place there.  (Windows itself has trouble enough with its
conventions; another little exercise: what is a file name separator
under Windows and how do you use it?) to return to the file name list
example, it's less surprising that Plan 9 can't take a list of file in
Windows style and access them directly.  that wouldn't work whether
space were allowed or not.   Plan 9 already does translate at
the boundaries between UTF-8 and whatever the other system uses.

the latter approach handles not just space but also some other
characters such as `/' that turn up at the boundaries, but cannot be
handled by Plan 9.  again, some are glib about the need for it,
and i snarl again.  even so, pragmatically there's no need to use one technique
for everything if space is regarded as utterly different from all
other such characters.  space could be used as itself within the system
and we can map `/' as and when we find it.

it's also worth noting that having consistent quoting conventions to allow spaces
in space-separated input is probably worthwhile on its own, so not all
the work in Plan A will vanish.  don't quit that editor just yet.

[-- Attachment #2: Type: message/rfc822, Size: 1658 bytes --]

To: 9fans@cse.psu.edu
Subject: Re: [9fans] blanks in file names
Date: Thu,  4 Jul 2002 07:10:55 +0900
Message-ID: <20020703221118.15154199B7@mail.cse.psu.edu>

Hello,

>You can't do that becaause if a file contains a genuine
>underscore, the outgoing software will think it's a space.
the outgoing software will also think it's a underscaore.
Is this important?
Some OSs already map 'Windows' to 'WINDOWS'.

Kenji Arisawa

Re: [9fans] blanks in file names

[arisawa]

Hello,

>You can't do that becaause if a file contains a genuine
>underscore, the outgoing software will think it's a space.
the outgoing software will also think it's a underscaore.
Is this important?
Some OSs already map 'Windows' to 'WINDOWS'.

Kenji Arisawa

Re: [9fans] blanks in file names

[rob pike, esq.]

You can't do that becaause if a file contains a genuine
underscore, the outgoing software will think it's a space.

-rob

--On Wednesday, July 03, 2002 10:54 PM +0900 arisawa@ar.aichi-u.ac.jp wrote:

> Hello,
>
> I prefer spaces are mapped simplly to underscores.
> Then
> term% touch 'some files'
> create some_files
> and
> term% rm 'some files'
> removes some_files
> However, I don't think these defects are practically important.
> I am rather afraid Plan9 loses simplicity.
>
> Kenji Arisawa
> E-mail: arisawa@aichi-u.ac.jp

Re: [9fans] blanks in file names

[rob pike, esq.]

You guys are all arguing about system stuff but it's the
*user interface* that you're really arguing about, and
breaking.  You are opening a can of worms you will never
get closed again.  Change space! Change the file delimiter!
The shell will never recover.  The system will break.
I will mourn.

-rob

Re: [9fans] blanks in file names

[rob pike, esq.]

Because file names come in places other than directories. You
can't always get this right.  If you translate them going into
open(), you will be inconsistent with file names used other ways.
It's similar to the problem with NAT boxes and hidden IP addresses.

I'd say do nothing yet. There is no clear right answer.

-rob

--On Tuesday, July 02, 2002 4:57 PM +0200 FJ Ballesteros 
<nemo@gsyc.escet.urjc.es> wrote:

>>
>> Some characters were just not meant to be in filenames, which would
>> avoid the above.
>
> But they are, even in my system there're many files with blanks on their
> names
> (and I never choose such names).
>
> Since IFAIK none of the authors have anything against the proposal
> that came out of 9fans (see my previous post, where I tried to summarize
> it),
> what if I change the source just to do it?
>
> 	- let Alt-spc generate our unicode blank character
> 	- change the system to translate blanks in file names to the
> 	  same character.
> 	- change the quote library so it does not quote that character.
>
> I don't have much time right know and that's why I ask before changing
> the
> code. I'm willing to implement it if there's agreement, so please, don't
> ignore the proposal just becase there's no code attached by now,
> just let me know.

Re: [9fans] blanks in file names

[FJ Ballesteros]

Well, I agree, but since I got your mail after a tiny spare-time bucket, 
I already
tried it and sent the diff. 

And now that I have been playing with it for a while, I have
started to love it, sic. 

Regarding the inconsistence, I think that most of the cases
are still handled by the side-effect that 'a x' is still
understood as 'a x'. Although not fully consistent, I don't know
a better way.


"rob pike, esq." ha escrito:
> 
> Because file names come in places other than directories. You
> can't always get this right.  If you translate them going into
> open(), you will be inconsistent with file names used other ways.
> It's similar to the problem with NAT boxes and hidden IP addresses.
> 
> I'd say do nothing yet. There is no clear right answer.
> 
> -rob
> 
> --On Tuesday, July 02, 2002 4:57 PM +0200 FJ Ballesteros
> <nemo@gsyc.escet.urjc.es> wrote:
> 
> >>
> >> Some characters were just not meant to be in filenames, which would
> >> avoid the above.
> >
> > But they are, even in my system there're many files with blanks on their
> > names
> > (and I never choose such names).
> >
> > Since IFAIK none of the authors have anything against the proposal
> > that came out of 9fans (see my previous post, where I tried to summarize
> > it),
> > what if I change the source just to do it?
> >
> >       - let Alt-spc generate our unicode blank character
> >       - change the system to translate blanks in file names to the
> >         same character.
> >       - change the quote library so it does not quote that character.
> >
> > I don't have much time right know and that's why I ask before changing
> > the
> > code. I'm willing to implement it if there's agreement, so please, don't
> > ignore the proposal just becase there's no code attached by now,
> > just let me know.

Re: [9fans] blanks in file names

[FJ Ballesteros]

I wouldn't say  this change makes the system more complex,
one thing I found after playing with this 
is that even though I made no change
to the quote stuff in the C library, programs started to
handle funny file names without relying on %q.

This is the semantics for the thing I sent before:
both touch 'a x' and touch 'a\0xa0x'
would create a file named 'a x'

Thus in both cases, ls would get
'a\0x00a0x'

Besides, foreign systems would still see 'a x'

(Im sorry, but don't know how to type Alt X 00A0 under windows,
 \x00a0 above should be read like our non-break space character).

Regarding what you say about "_", I'd still like to be able
to type "touch a_x.c". IMHO, the
 character of choice doesn't matter to much, but I think
it would be better if it's not an already used character.


arisawa@ar.aichi-u.ac.jp ha escrito:
> 
> Hello,
> 
> I prefer spaces are mapped simplly to underscores.
> Then
> term% touch 'some files'
> create some_files
> and
> term% rm 'some files'
> removes some_files
> However, I don't think these defects are practically important.
> I am rather afraid Plan9 loses simplicity.
> 
> Kenji Arisawa
> E-mail: arisawa@aichi-u.ac.jp

Re: [9fans] blanks in file names

[arisawa]

Hello,

I prefer spaces are mapped simplly to underscores.
Then
term% touch 'some files'
create some_files
and
term% rm 'some files'
removes some_files
However, I don't think these defects are practically important.
I am rather afraid Plan9 loses simplicity.

Kenji Arisawa
E-mail: arisawa@aichi-u.ac.jp

Re: [9fans] blanks in file names

[Lucio De Re]

*** ramble alert! ***

On Wed, Jul 03, 2002 at 10:00:55AM +0200, Fco.J.Ballesteros wrote:
> 
> IMHO, the problem is mostly the user programs and not the kernel.
> AFAIK, the kernel is fine if you don't use / and \0 as delimiters
> (which seems reasonable to me, although some guys might want to use it too).
> 
Yes, it would be convenient for the delimiter to be soft, for example.

> But the tradition that blanks separate arguments is deeply embedded in
> user programs, perhaps most notably the shell.
> 
I certainly will not argue with this.

> Assume the kernel has changed to use openv[], what would the shell
> do to deal with spaces vs 00A0s ?
> 
Well, it's not as if I'm the genius here :-)
  
The command line (ignore the shell for a moment as its interpreter)
is a very clear expression of the user's intent, in that there is
a long tradition behind it and, eqaully important(ly), it is visually
unambiguous (within human limits).    The feature one often overlooks
of spaces is that they can be repeated with no change of semantics
or replaced by other whitespace with similar results (cf. sendmail's
configuration file and its tab/space madness).

So it would seem reasonable to retain the convention unchanged in
the familiar contexts and seek alternatives in the new, graphical
environments, for example.  Is there perhaps an undiscovered
technique to express a shell pipeline in a graphical format?  If
so, can new separator rules be brought to apply, escaping some of
the limitations of the traditional approach?

Maybe facetiously, I'd like to point out that the space is the
absence of a character to the eye, if we used a typescript interface
with pixel-dots in the character positions, drag-and-dropping
command line parameters would be automatically quoted by the
background pixel-dots.  That is, until you want a pixel-dot character
in an argument. Sigh.

Still, a background of single quotes would work and be horrendously
ugly, argument completion may become mandatory as the shell now no
longer can provide that function as everything is quoted.

In short, no, I do not have an answer for Nemo, but it seems to me
that bending the conventional interface is not going to work well,
we ought to be looking for new answers, not unlike the ingeniousness
that brought us Plan 9 in the first place and acme as the most
extreme example.

++L

Re: [9fans] blanks in file names

[Fco.J.Ballesteros]

rog@vitanuova.com :
:  /sys/src/cmd/dossrv
:  /sys/src/cmd/9660srv
:  /sys/src/cmd/tapefs
:  /sys/src/cmd/unix/u9fs
:  /sys/src/cmd/ftpfs
:  	convert "space char" to/from external actual space on create,
:  	walk, wstat, stat and directory reads.

One crazy idea I had was to do that translation in the mount driver.
That way the server would be happy to think that it uses space, and
the client plan 9 program would be happy to see 00A0 or whatever
without confussion with the space character. 

lucio@proxima.alt.za : 
:  What I'm saying, is that I'd like to target a kernel that is entirely
:  delimiter agnostic and promote each user application in the same
:  direction as a long-term project.  In the interim, constructs that
:  cast delimiters in stone should be removed wherever possible.

IMHO, the problem is mostly the user programs and not the kernel.
AFAIK, the kernel is fine if you don't use / and \0 as delimiters
(which seems reasonable to me, although some guys might want to use it too). 

But the tradition that blanks separate arguments is deeply embedded in
user programs, perhaps most notably the shell.

Assume the kernel has changed to use openv[], what would the shell
do to deal with spaces vs 00A0s ?

Re: [9fans] blanks in file names

[Dan Cross]

> /sys/src/9/port/latin1.c

Speaking of this....  Is there a good reason to leave latin1 in the
kernel, and not pull it up into libc?  I suggested this once before,
and Russ didn't want to move it since it's most frequently used in the
kernel.  However, it seems silly to me that it's not in bitsy/keyboard,
for one, and anyone who wants to write a keyboard-like application
would want to avail themselves of it.  For instance, I was working on a
Targus keyboard driver, and had to copy it out of the kernel to link
against.

	- Dan C.

Re: [9fans] blanks in file names

[plan9]

On Tue, Jul 02, 2002 at 12:53:16PM +0100, matt wrote:
> 
> I often use spaces in my filenames, they are human

Do they all go down to the Space Bar and get pounded after work?

Re: [9fans] blanks in file names

[rog]

> 	- change the system to translate blanks in file names to the
> 	  same character.

it might be worth identifying the exact points that would need changing
should such a scheme be chosen.

/sys/src/cmd/dossrv
/sys/src/cmd/9660srv
/sys/src/cmd/tapefs
/sys/src/cmd/unix/u9fs
/sys/src/cmd/ftpfs
	convert "space char" to/from external actual space on create,
	walk, wstat, stat and directory reads.

/sys/src/9/port/latin1.c
	make it possible to have a single character ALT sequence
	(if ALT-space is what's desired).

>	- change the quote library so it does not quote that character.

it doesn't matter if it does...
however various places would need to change back so they didn't quote
chars (e.g.  ls, pwd?)

a suitable character would need to be chosen (e.g.  00A0), and it
would be nice if most fonts displayed it in a reasonably consistent
manner.

perhaps a new function could be added to libc that does the
conventional conversion from ' ' to the new space char so that
programs with GUI entry boxes that know that a string is being typed
in a filename context can trivially do the conversion.

what have i missed?

Re: [9fans] blanks in file names

[Lucio De Re]

On Tue, Jul 02, 2002 at 04:57:18PM +0200, FJ Ballesteros wrote:
> > 
> > Some characters were just not meant to be in filenames, which would avoid the
> > above.
> 
> But they are, even in my system there're many files with blanks on their
> names
> (and I never choose such names).
> 
I prefer the suggested option of an open(2) (openv?) that takes a
strings vector instead of a slash-delimited path.  It seems to me
that such an approach is still viable as an objective and it would
be expedient to keep it in mind and avoid developments that might
defeat it.

What I'm saying, is that I'd like to target a kernel that is entirely
delimiter agnostic and promote each user application in the same
direction as a long-term project.  In the interim, constructs that
cast delimiters in stone should be removed wherever possible.

Of course, I may be talking about a completely different operating
system, but it strikes me that Plan 9 is the most suitable foundation
for such a thing and its developers' abilities certainly (in my
opinion) unmatched, so why not start there?

++L

Re: [9fans] blanks in file names

[FJ Ballesteros]

> 
> Some characters were just not meant to be in filenames, which would avoid the
> above.

But they are, even in my system there're many files with blanks on their
names
(and I never choose such names).

Since IFAIK none of the authors have anything against the proposal
that came out of 9fans (see my previous post, where I tried to summarize
it),
what if I change the source just to do it? 

	- let Alt-spc generate our unicode blank character
	- change the system to translate blanks in file names to the
	  same character.
	- change the quote library so it does not quote that character.

I don't have much time right know and that's why I ask before changing
the
code. I'm willing to implement it if there's agreement, so please, don't
ignore the proposal just becase there's no code attached by now,
just let me know.

Re: [9fans] blanks in file names

[Boyd Roberts]

Seems like GNU have a 'solution':

NAME
       xargs - build and execute command lines from standard input

SYNOPSIS
       xargs  [-0prtx]  [-e[eof-str]] [-i[replace-str]] [-l[max-lines]] [-n
       max-args] [-s max-chars] [-P max-procs]  [--null]  [--eof[=eof-str]]
       [--replace[=replace-str]]  [--max-lines[=max-lines]] [--interactive]
       [--max-chars=max-chars] [--verbose] [--exit] [--max-procs=max-procs]
       [--max-args=max-args] [--no-run-if-empty] [--version] [--help] [com­
       mand [initial-arguments]]

DESCRIPTION
       This manual page documents the GNU version of  xargs.   xargs  reads
       arguments from the standard input, delimited by blanks (which can be
       protected with double or single quotes or a backslash) or  newlines,
       and  executes  the  command (default is /bin/echo) one or more times
       with any initial-arguments followed by arguments read from  standard
       input.  Blank lines on the standard input are ignored.

       xargs exits with the following status:
       0 if it succeeds
       123 if any invocation of the command exited with status 1-125
       124 if the command exited with status 255
       125 if the command is killed by a signal
       126 if the command cannot be run
       127 if the command is not found
       1 if some other error occurred.

Some characters were just not meant to be in filenames, which would avoid the
above.

Re: [9fans] blanks in file names

[matt]

> it's not just spaces.  i have had to handle / as well, for instance.
> that might not be of interest to some, but it has occurred.

Reserved characters in filename conventions seem to me to be archaic.

I often use spaces in my filenames, they are human

The directory separator for instance is in band signalling.

If I can quote Douglas A Gwyn here * :

"For example, I take it as one more example of the evil of stealing
perfectly legitimate code values for in-band control purposes."

Being unable to use / as part of a filename is burdensome when it comes to
"interesting" file servers.

For instance one of my daydreams is the IRC fileserver with the channel name
as a directory but many many channels use / in their channel names. (In fact
many people go out of their way to cram as many special characters into IRC
channel names as possible).

I've wondered if it was possible to define the path separator on a per
process basis or something.

Perhaps you can already?

matt




*
http://groups.google.com/groups?q=in+band+group:comp.os.plan9&hl=en&lr=&ie=U
TF-8&selm=3CCE1366.EAB27A38%40null.net&rnum=2

Re: [9fans] blanks in file names

[forsyth]

>>rog@vitanuova.com wrote:
>> the benefits of converting to utf-8 throughout were obvious.  does the
>> ability to type ' ' rather than (say) ALT-' ' really confer comparable
>> advantages?

>The benefits of any feature depend on whether you make use of them.
>People living in an ASCII world get no particular benefit from UTF8,
>and people with spaces in file names (e.g. on a Windows filesystem)
>get substantial benefit from having their files handled properly.

no one is suggesting, least of all roger, that nothing be done to
allow access to files between Plan 9 and systems that have spaces in names.
the disagreement is about the scope, and the means.

>>In any case, I agree that blanks are here to stay and I'd like Plan 9
>>to handle then as nicely as it handles .

it's not just spaces.  i have had to handle / as well, for instance.
that might not be of interest to some, but it has occurred.
from an end user's point of view, it seems perfectly reasonable to me.

i'd also pick out something from a previous comment:

>>Plan 9 to Unicode and UTF: too hard, too much code to change, too many
>>symmetries broken.  But there's no way this problem is as hard as that
>>conversion, and we handled that one just fine.  All that's missing is

surely it was much easier to do once the problems of Unicode's
original 8-bit representation pre-UTF had been dealt with:

	In August 1992, X-Open circulated a proposal for another UTF-like byte
	encoding of Unicode characters.  Their major concern was that an
	embedded character in a file name (in particular a slash) could be
	part of an escape sequence in UTF and therefore confuse a traditional
	file system.

that single change to UTF made it more straightforward to work out where
the potential problems were, not least some older code could no longer fail (as it
would have done with the earlier proposal):
	p = strrchr(filename, '/');
for instance.  prior to that, each such instance needed to be tracked down
and examined, assuming (on non-Plan9 systems) that source was available.
of course, larger changes were required to tools that needed to support Unicode
well (regexp, tr, wc, and so on).   still, with many potential possibilities
for mechanical confusion eliminated at a stroke, i'd say it instantly made the
idea attractive.  there were not as many problems to worry about.

i do think the support for quoting is useful for many things (roger added support for quoting to
Inferno's String module several years ago for just that reason), but i'm not sure myself
it's the right or sufficient solution for file names.

[9fans] blanks in file names

[Fco.J.Ballesteros]

After reading all the post about this, and thinking about it
I think it's more clean to use different characters for space and
blank within a file name — as someone said before.

The reason why I think so is that it would simplify the library
and at the same time avoid problems with quoting. For example,
although replica uses %q, I ended up deleting a file name 'chk, because
somehow replica got confussed. It's true that I could have tried
to avoid the confussion and fix the bug; but I think this suggests that
it's not so easy to quote things.

On the other side, there're two places where file names get blanks in:
- from foreign systems
- locally created files

On both places it'd be easy for the user to type Alt-spc, perhaps more
simpler than it'd be to write 'blah blah'.

Now, does anyone from the Labs think otherwise, and if so,
what's the reason. I'd like to learn from this mistake, if my current
view of the problem can be considered as so.

In any case, I agree that blanks are here to stay and I'd like Plan 9
to handle then as nicely as it handles ☺ⁱ⁲.
thanks



ⁱ Why did ☺ get into unicode?
⁲ Why didn't :-( get in then? (Did they read TPOP?).