[9fans] Tvx update

[9fans] Tvx update

[EBo]

I am please to announce the 05/22/2010 update of Tvx has been accepted
into TinyCoreLinux's standard distribution package list.

This update includes all updates to 9vx and sysfromiso including more
strongarm support, a 9vx wrapper which assists new users in either copying
or linking to the base Plan 9 root tree.  All of the packages are currently
set up in Tvx's dependency file so installing Tvx also installs the root,
docs, and source code.

Please feel free to play with it and let me know if you find any bugs or
suggested improvements.

  Best regards,

  John (EBo) David --

Re: [9fans] Tvx update

[Nick LaForge]

Amazing.  I have no excuse for not using Linux now.

Re: [9fans] Tvx update

[EBo]

On Tue, 25 May 2010 11:34:13 -0800, Nick LaForge <nicklaforge@gmail.com>
wrote:
> Amazing.  I have no excuse for not using Linux now.

Let me know about your experiences.


  EBo --

Re: [9fans] Tvx update

[Nick LaForge]

I tried it and it is very fast to boot and run.  However, the wrapper
did not copy Tvx-root to my home dir.  Option 2 links to the package
install in /usr/local, but option 1 also links, but to the package
loopback mount in /tmp.  The only files in my home are in the top
directory (the license, etc.).  Writing to the symlinks fails.  And so
does reading utf8 filenames.

Choosing 1 gives this error:

cp: can't preserve ownership of
'/home/tc/Tvx-root/sys/lib/ghostscript' :Operation not permitted

and so on for many more files in /sys, /tmp, and /usr.

Nick

Re: [9fans] Tvx update

[erik quanstrom]

> '/home/tc/Tvx-root/sys/lib/ghostscript' :Operation not permitted

would it be possible given external constraints, to pick
a name more in keeping with plan 9 style like tvxroot
rather than Tvx-root?

- erik

Re: [9fans] Tvx update

[EBo]

>> '/home/tc/Tvx-root/sys/lib/ghostscript' :Operation not permitted
>
> would it be possible given external constraints, to pick
> a name more in keeping with plan 9 style like tvxroot
> rather than Tvx-root?

I would say it is possible/probable but there are a couple of details we
would need to work out amongst ourselves so I only bother the TCL admins
once on this issue.

TCL *require* that the source and documentation be included as a separate
extension.  Because of the root requirement we can rename it to tvxroot,
but should they be tvxrootsrc or tvxroot-src?  Same for the docs.

  EBo --

Re: [9fans] Tvx update

[EBo]

> I tried it and it is very fast to boot and run.  However, the wrapper
> did not copy Tvx-root to my home dir.  Option 2 links to the package
> install in /usr/local, but option 1 also links, but to the package
> loopback mount in /tmp.  The only files in my home are in the top
> directory (the license, etc.).  Writing to the symlinks fails.  And so
> does reading utf8 filenames.

My first guess is that the USB was not set up with persistent home.  So,
please double check TCL's setup and installation instructions and verify
that you in fact have persistent home working.  If you do, and you still
have this problem please email me off-list and we can sort it out and post
a solution/report.

> Choosing 1 gives this error:
>
> cp: can't preserve ownership of
> '/home/tc/Tvx-root/sys/lib/ghostscript' :Operation not permitted
>
> and so on for many more files in /sys, /tmp, and /usr.

That is not an error, but a warning.  It is caused by the operation being
run as user tc and not root.  I have thought about several sollutions to
this problem, but the only two realistic ones is to redirect warning/error
messages to /dev/nul or run the copy operation as sodo.  At first I did not
like either operation but just realized that the system already accepts
sudo commands from user tc so it does not pose a security risk.  I'll add
that to the list of thigs to fix for next revision.

  EBo --

Re: [9fans] Tvx update

[Nick LaForge]

sed '43 s/H/L/' Tvx

Why are there TWO ways for cp to follow symlinks?  Why are there
symlinks it all?  (That should do it.)

And, still:

cd /sys/doc
ls | grep 8
8
cd 8
Can't cd 8: '8' file does not exist
cd 8½
Can't cd 8½: '8½' file does not exist
cd 8?
ls 8?.ms
8�.ms
touch 9½
ls 9½
9½

Nick

Re: [9fans] Tvx update

[erik quanstrom]

> > cp: can't preserve ownership of
> > '/home/tc/Tvx-root/sys/lib/ghostscript' :Operation not permitted

cp doesn't try to preserve ownership by default.
(unless there's something new and wierd that i
missed.)  perhaps you could track down the source
of the cp flags and change them.

> like either operation but just realized that the system already accepts
> sudo commands from user tc so it does not pose a security risk.  I'll add

s/a (security risk)/an additional \1/

would seem much easier to fix cp's flags.

- erik

Re: [9fans] Tvx update

[ron minnich]

On Wed, May 26, 2010 at 11:02 AM, Nick LaForge <nicklaforge@gmail.com> wrote:
> I tried it and it is very fast to boot and run.  However, the wrapper
> did not copy Tvx-root to my home dir.  Option 2 links to the package
> install in /usr/local, but option 1 also links, but to the package
> loopback mount in /tmp.

Is this because option 2 is the persistent install and option 1 is
not? Note the loopback mounts are readonly squashfs mounts.

> The only files in my home are in the top
> directory (the license, etc.).  Writing to the symlinks fails.  And so
> does reading utf8 filenames.

Be careful .. did you set up a persistent /home? I'm not sure what you
have done. There's no substitute at some point for seeing how things
work at tinycore.org.


cd /sys/doc
ls | grep 8
8
cd 8
Can't cd 8: '8' file does not exist
cd 8½
Can't cd 8½: '8½' file does not exist
cd 8?
ls 8?.ms
8�.ms
touch 9½
ls 9½
9½

That's an interesting problem. I wonder if it is some artifact of the
use of squashfs. The sam and venti directories are fine.

This may be a flaw in tinycore ... we'll talk to them.

Thanks for the testing. I have found one thing out ... I don't think
9vx is SMP-safe. I've had some real problems building in /sys/src,
where 9vx gets a segv on "cpu 5" or whatever. If I boot with only one
cpu, there are no problems. Worth keeping in mind ... I may modify my
version of 9vx to wire itself to cpu 0, as well as its children, and
see if the failures I'm seeing go away. That's a pointer at least to
where the issues may be.

Also ...

>sed '43 s/H/L/' Tvx

Sorry, you lost me.

>Why are there TWO ways for cp to follow symlinks?  Why are there
>symlinks it all?  (That should do it.)

They're not going away I bet, but what are you talking about here?


ron

Re: [9fans] Tvx update

[Nick LaForge]

On Wed, May 26, 2010 at 3:58 PM, ron minnich <rminnich@gmail.com> wrote:
> Be careful .. did you set up a persistent /home? I'm not sure what you
> have done. There's no substitute at some point for seeing how things
> work at tinycore.org.

Not necessary -- once the install script is fixed to actually follow
the symlink it works on tmpfs home too.

> cd /sys/doc
> ls | grep 8
> 8
> cd 8
> Can't cd 8: '8' file does not exist
> cd 8½
> Can't cd 8½: '8½' file does not exist
> cd 8?
> ls 8?.ms
> 8�.ms
> touch 9½
> ls 9½
> 9½
>
> That's an interesting problem. I wonder if it is some artifact of the
> use of squashfs. The sam and venti directories are fine.

The sam and venti directories don't have any non-ascii utf-8.  Also,
from the host:
ls sys/doc/?½.ms
8½.ms
ls sys/doc/??½.ms
9ý.ms (The file I touched from 9vx)

So 9vx is misreading utf-8 from the host, and unsurprisingly, it tacks
extra bits on when writing.  This happens on a tmpfs home and on a
persistent ext2 home (I actually didn't test how the HOST interprets
it on ext2 because I got rid of my persistent home.)

>>sed '43 s/H/L/' Tvx
>
> Sorry, you lost me.

change /usr/local/bin/Tvx:43 to
cp -LRp /usr/local/Tvx-root/* ${P9}

On Wed, May 26, 2010 at 3:53 PM, erik quanstrom <quanstro@quanstro.net> wrote:
>perhaps you could track down the source of the cp flags and change them.

The 'p' flag preserves the dates, although the directories that were
separated (/sys/src and /sys/doc) were touched anyway by the
installer.

> They're not going away I bet, but what are you talking about here?

No, I want symlinks to go away from the universe.  (Bind them in a bag
and drown them in the ocean)

Re: [9fans] Tvx update

[ron minnich]

On Thu, May 27, 2010 at 1:50 AM, Nick LaForge <nicklaforge@gmail.com> wrote:
> On Wed, May 26, 2010 at 3:58 PM, ron minnich <rminnich@gmail.com> wrote:

>> They're not going away I bet, but what are you talking about here?
>
> No, I want symlinks to go away from the universe.  (Bind them in a bag
> and drown them in the ocean)

you'll get no argument from me on that score.

ron

Re: [9fans] Tvx update

[EBo]

On Thu, 27 May 2010 02:51:44 +0000, ron minnich <rminnich@gmail.com>
wrote:
> On Thu, May 27, 2010 at 1:50 AM, Nick LaForge <nicklaforge@gmail.com>
> wrote:
>> On Wed, May 26, 2010 at 3:58 PM, ron minnich <rminnich@gmail.com>
wrote:
> 
>>> They're not going away I bet, but what are you talking about here?
>>
>> No, I want symlinks to go away from the universe.  (Bind them in a bag
>> and drown them in the ocean)

Nick, just tell us how you feel... don't hold anything back now ;-)

> you'll get no argument from me on that score.

Ok.  I'll remove it in the next version.  It was only added as an attempt
to deal with people who do not want to make a second copy of the root.  As
a note, the first version actually installed into /home/tc, but there were
some issues with how TCL is set up and no realistic way to install into
${home}  . The stuff in installed readonly (via squashfs) into /usr/local,
and I have been told that there is no mechanism to install directly into
${home}.

I'll play with this more when I have had some sleep,,,

   EBo --

Re: [9fans] Tvx update

[Nick LaForge]

>> you'll get no argument from me on that score.

> Ok.  I'll remove it in the next version.

I think he was responding to my hyperbolic statement there.  There's
no reason not to use symlinks if we can't bind things anyway.  Your
option 2 just links to the root dir in /usr/local which is just fine
provided the permissions are in place and the /usr/local dir is not
just a bunch of symlinks to individual files in /tmp/tcloop  (actually
the case in my tinycore install, not that I care).  The only problem
with symlinks here wasith your option 1 where cp was making copies of
symlinks instead of original files beacuse you used 'cp -HRp' instead
of 'cp -LRp'.

Nick

Re: [9fans] Tvx update

[EBo]

On Wed, 26 May 2010 21:03:18 -0800, Nick LaForge <nicklaforge@gmail.com>
wrote:
>>> you'll get no argument from me on that score.
>
>> Ok.  I'll remove it in the next version.
>
> I think he was responding to my hyperbolic statement there.  There's
> no reason not to use symlinks if we can't bind things anyway.  Your
> option 2 just links to the root dir in /usr/local which is just fine
> provided the permissions are in place and the /usr/local dir is not
> just a bunch of symlinks to individual files in /tmp/tcloop  (actually
> the case in my tinycore install, not that I care).  The only problem
> with symlinks here wasith your option 1 where cp was making copies of
> symlinks instead of original files beacuse you used 'cp -HRp' instead
> of 'cp -LRp'.

ok.  Thanks.  I think there is a bit more going on than the s/H/L/ issues.
Need to look into it though.

I also started a FAQ a couple of days ago, but there is not enough in it
to really post yet.  Documenting the issues with persistent home/local/etc.
is one of the things on my todo list.

Thanks for testing all this BTW.

  EBo --

Re: [9fans] Tvx update

[EBo]

On Wed, 26 May 2010 21:03:18 -0800, Nick LaForge <nicklaforge@gmail.com>
wrote:
>>> you'll get no argument from me on that score.
>
>> Ok.  I'll remove it in the next version.
>
> I think he was responding to my hyperbolic statement there.  There's
> no reason not to use symlinks if we can't bind things anyway.  Your
> option 2 just links to the root dir in /usr/local which is just fine
> provided the permissions are in place and the /usr/local dir is not
> just a bunch of symlinks to individual files in /tmp/tcloop  (actually
> the case in my tinycore install, not that I care).  The only problem
> with symlinks here wasith your option 1 where cp was making copies of
> symlinks instead of original files beacuse you used 'cp -HRp' instead
> of 'cp -LRp'.

sorry for getting thing out of order.  I simply took the hyperbolic
statement as a request to remove it.  I could be OK with that, but would
want to think about the consequences of taking either path.  Also, I am not
sure that the L/H solution will solve all of the issues -- there is a
ownership problem, but I am not sure if 9vx has to keep the system level
applications owned by root/admin/sys.  Actually I think that would be a
very good idea, but the only way to keep the permissions straight (that I
know of anyway) is to run the cp under sudo.  Anyway, I have not completely
thought through the issues yet, but ...

Laters,

    EBo --

ps: I have a big stick, and if I ever find the sandman I think I will give
him a resounding thwack up the back of his head...

Re: [9fans] Tvx update

[hiro]

I don't think the tinycore guys would refuse tvx because of included
man pages or source.

Re: [9fans] Tvx update

[Jorden M]

On Thu, May 27, 2010 at 10:30 AM, hiro <23hiro@googlemail.com> wrote:
> I don't think the tinycore guys would refuse tvx because of included
> man pages or source.
>
>

Unless they like emacs.

Re: [9fans] Tvx update

[EBo]

> I don't think the tinycore guys would refuse tvx because of included
> man pages or source.

maybe, but I know that the first time submitted a package for review they
specifically pointed it out and asked me to break them up.  I do not
remember if it was rejected for that reason, or stated as a strong
preference.  Either way, this is their requirement and I have no reason to
deny them that.  What seemed to pass in the last review was to set up a
dependency for all root/src/doc packages be deps for tvx.  As a note, a
couple of the stated reasons for their preferences are 1) smaller size if
you do not need them, and 2) smaller updates when changing documentation
(since it does not require also updating the base programs, etc.).  This is
somewhat meaningless for 9vx/plan9, but is their general policy.  So I
comply.

  EBo --

Re: [9fans] Tvx update

[ron minnich]

On Thu, May 27, 2010 at 7:26 PM, EBo <ebo@sandien.com> wrote:
>
>> I don't think the tinycore guys would refuse tvx because of included
>> man pages or source.
>
> maybe, but I know that the first time submitted a package for review they
> specifically pointed it out and asked me to break them up.

Hmm. Seems that no matter what we may think, EBo *knows* what they'll
take, since he talked to them directly.

I also think from my work with tvx is right, but I don't know for sure :-)

ron

Re: [9fans] Tvx update

[ron minnich]

On Thu, May 27, 2010 at 4:19 AM, EBo <ebo@sandien.com> wrote:

> Ok.  I'll remove it in the next version.  It was only added as an attempt
> to deal with people who do not want to make a second copy of the root.

Ebo, I'm sorry if I added to the confusion.

Don't change your use of symlinks. I meant more as a global thing: see
Korn's paper "Symlinks are a botch".

I don't see a problem with your use of them.

ron

Re: [9fans] Tvx update

[EBo]

>> Ok.  I'll remove it in the next version.  It was only added as an
>> attempt
>> to deal with people who do not want to make a second copy of the root.
> 
> Ebo, I'm sorry if I added to the confusion.
> 
> Don't change your use of symlinks. I meant more as a global thing: see
> Korn's paper "Symlinks are a botch".
> 
> I don't see a problem with your use of them.

no real confusion.  Using symlinks was simply a judgment call.  I consider
it a hack, but it addresses the issue allowing user modifiable roots while
also allowing a mode which ensures a pristine environment on boot. As far
as I can tell the later is a fundamental design philosophy and goal within
TinyCore.  So, this was the best I could come up with at the moment as a
workable compromise for both camps.  So, I am willing to go either way.

Thanks for the additional feedback though, it helps to have the clarity.

  EBo --

Re: [9fans] Tvx update

[ron minnich]

On Thu, May 27, 2010 at 7:02 AM, EBo <ebo@sandien.com> wrote:
>

> no real confusion.  Using symlinks was simply a judgment call.  I consider
> it a hack, but it addresses the issue allowing user modifiable roots while
> also allowing a mode which ensures a pristine environment on boot.

it was the right call.

ron

Re: [9fans] Tvx update

[Chad Brown]

On May 26, 2010, at 10:48 PM, ron minnich wrote:

> Don't change your use of symlinks. I meant more as a global thing: see
> Korn's paper "Symlinks are a botch".

Can I beg a specific title or reference?  My efforts with google turned up primarily references to your original post (and a former US Ambassador to Togo).

Thank you,
*Chad

Re: [9fans] Tvx update

[Bakul Shah]

On Thu, 27 May 2010 12:59:33 PDT Chad Brown <yandros@MIT.EDU>  wrote:
> On May 26, 2010, at 10:48 PM, ron minnich wrote:
>
> > Don't change your use of symlinks. I meant more as a global thing: see
> > Korn's paper "Symlinks are a botch".
>
> Can I beg a specific title or reference?  My efforts with google turned =
> up primarily references to your original post (and a former US =
> Ambassador to Togo).
>
> Thank you,
> *Chad

I think Ron was referring to Korn's 1987 usenet posting,
where Korn said "the implementation of symbolic links on BSD
Unix is a botch".  Not exactly the same thing.  If BSD had
implemented ".." correctly (i.e. walk back up one level in
the given path), symlinks would have been more useful and
less surprising. But for that they would have had to store
current dir path in the kernel; which was not a popular idea
then (or even now as it seems /bin/pwd has to grovel around
reconstructing $PWD path painfully).  And a limit of 8
indirect symlink was hokey. Detecting loops isn't hard.

Re: [9fans] Tvx update

[ron minnich]

On Thu, May 27, 2010 at 8:16 PM, Bakul Shah <bakul+plan9@bitblocks.com> wrote:

> I think Ron was referring to Korn's 1987 usenet posting,
> where Korn said "the implementation of symbolic links on BSD
> Unix is a botch".

Uh oh. Is my memory really that bad? Cart me off to the Rock of Ages
Home for Retired Hackers! Do I get the room next to Eugene Miya?

Anyway, what really stuck in my mind from Korn's post was his
characterization of symbolic links as a "non local goto", and I got
the impression from the "paper" that he was not enthusiastic about the
idea in general, much less the implementations.

Thanks

ron

Re: [9fans] Tvx update

[erik quanstrom]

> Anyway, what really stuck in my mind from Korn's post was his
> characterization of symbolic links as a "non local goto", and I got
> the impression from the "paper" that he was not enthusiastic about the
> idea in general, much less the implementations.

and, the whole idea breaks down unless there is a single,
global namepace.

- erik

Re: [9fans] Tvx update

[Ethan Grammatikidis]

On 28 May 2010, at 11:34, erik quanstrom wrote:

>> Anyway, what really stuck in my mind from Korn's post was his
>> characterization of symbolic links as a "non local goto", and I got
>> the impression from the "paper" that he was not enthusiastic about
>> the
>> idea in general, much less the implementations.
>
> and, the whole idea breaks down unless there is a single,
> global namepace.

Oh, good point. I remember having trouble with symlinks & chroots. I
had several chroots in linux with scripts to set them up, & even
symlinks from outside into the chroot (e.g. to make 32-bit home easily
accessible from 64-bit home) were enough of a nuisance that I ended up
substituting binds.

OTOH I also have a vague memory of doing something clever with a
symlink & a chroot, but I can't remember what now.

>
> - erik
>

--
Simplicity does not precede complexity, but follows it. -- Alan Perlis

Re: [9fans] Tvx update

[Ethan Grammatikidis]

On 27 May 2010, at 21:16, Bakul Shah wrote:

> If BSD had
> implemented ".." correctly (i.e. walk back up one level in
> the given path), symlinks would have been more useful and
> less surprising.

This "correct" implementation of symlinks has never seemed right to
me. Linux / Bash used to do it the "wrong" way as recently as 2001 if
not more recently. I was much more comfortable with the "wrong" way,
with symlinks just being a portal to another place entirely, and I
wish I could pinpoint why. Having the path "faked" after following a
symlink just feels badly wrong, and I wish I could put a semantic
reason on it.

--
Simplicity does not precede complexity, but follows it. -- Alan Perlis

Re: [9fans] Tvx update

[erik quanstrom]

> On 27 May 2010, at 21:16, Bakul Shah wrote:
>
> > If BSD had
> > implemented ".." correctly (i.e. walk back up one level in
> > the given path), symlinks would have been more useful and
> > less surprising.
>
> This "correct" implementation of symlinks has never seemed right to
> me. Linux / Bash used to do it the "wrong" way as recently as 2001 if
> not more recently. I was much more comfortable with the "wrong" way,
> with symlinks just being a portal to another place entirely, and I
> wish I could pinpoint why. Having the path "faked" after following a
> symlink just feels badly wrong, and I wish I could put a semantic
> reason on it.

would you deconstruct bind/mount points as well?
recursively? that way lies vms/windows.

- erik

Re: [9fans] Tvx update

[Ethan Grammatikidis]

On 28 May 2010, at 12:59, erik quanstrom wrote:

>> On 27 May 2010, at 21:16, Bakul Shah wrote:
>>
>>> If BSD had
>>> implemented ".." correctly (i.e. walk back up one level in
>>> the given path), symlinks would have been more useful and
>>> less surprising.
>>
>> This "correct" implementation of symlinks has never seemed right to
>> me. Linux / Bash used to do it the "wrong" way as recently as 2001 if
>> not more recently. I was much more comfortable with the "wrong" way,
>> with symlinks just being a portal to another place entirely, and I
>> wish I could pinpoint why. Having the path "faked" after following a
>> symlink just feels badly wrong, and I wish I could put a semantic
>> reason on it.
>
> would you deconstruct bind/mount points as well?
> recursively? that way lies vms/windows.

No, a bind's different... Maybe I'm being an idiot, but I'd like to
have a neat & tidy argument against symlinks, saying symlinks
introduce a "damned if you do, damned if you don't" kind of issue.
Maybe the fact that no-one seems to like to keep the pwd in the kernel
is argument enough.

>
> - erik
>

--
Simplicity does not precede complexity, but follows it. -- Alan Perlis

Re: [9fans] Tvx update

[erik quanstrom]

> > would you deconstruct bind/mount points as well?
> > recursively? that way lies vms/windows.
>
> No, a bind's different... Maybe I'm being an idiot, but I'd like to
> have a neat & tidy argument against symlinks, saying symlinks
> introduce a "damned if you do, damned if you don't" kind of issue.
> Maybe the fact that no-one seems to like to keep the pwd in the kernel
> is argument enough.

wrt the question "what is the current directory",
can you explan how is bind different?

symlinks are problematic any way you slice it.
consider the number of programs that would
be in section 1 that have options, often in
multiples, for dealing with symlinks.

almost no unix programs (other than find)
bother with mount points.

- erik

Re: [9fans] Tvx update

[Steve Simon]

> almost no unix programs (other than find)
> bother with mount points.

Ok, only because it was in my final year exams, I know of one more
pwd needs to understand mount points (or did in v7) so it can step
over them - no doubt ther is a getwd() system call these days,
darn'ed new fangled things.

-Steve

Re: [9fans] Tvx update

[Ethan Grammatikidis]

On 28 May 2010, at 16:09, Steve Simon wrote:

>> almost no unix programs (other than find)
>> bother with mount points.
>
> Ok, only because it was in my final year exams, I know of one more
> pwd needs to understand mount points (or did in v7) so it can step
> over them - no doubt ther is a getwd() system call these days,
> darn'ed new fangled things.

There is a getcwd(), but in FreeBSD at least it's a library function,
not a system call. Perhaps the library has to look up mount points.

du (in at least FreeBSD, Gnu coreutils, busybox) also has an option to
not traverse mount points, making 2/5 of programs featuring directory
recursion also have an option to not traverse mount points. The others
are ls, grep, and diff.


--
Simplicity does not precede complexity, but follows it. -- Alan Perlis

Re: [9fans] Tvx update

[erik quanstrom]

> are ls, grep, and diff.

bsd: never an option too arcane to refuse.

- erik

Re: [9fans] Tvx update

[Ethan Grammatikidis]

On 28 May 2010, at 17:26, erik quanstrom wrote:

>> are ls, grep, and diff.
>
> bsd: never an option too arcane to refuse.

s/bsd/bsd|gnu/

Actually freebsd uses gnu grep and both get diff from the diffutils
package.

--
Simplicity does not precede complexity, but follows it. -- Alan Perlis

Re: [9fans] Tvx update

[Bakul Shah]

On Fri, 28 May 2010 12:51:36 BST Ethan Grammatikidis <eekee57@fastmail.fm>  wrote:
>
> On 27 May 2010, at 21:16, Bakul Shah wrote:
>
> > If BSD had
> > implemented ".." correctly (i.e. walk back up one level in
> > the given path), symlinks would have been more useful and
> > less surprising.
>
> This "correct" implementation of symlinks has never seemed right to
> me. Linux / Bash used to do it the "wrong" way as recently as 2001 if
> not more recently. I was much more comfortable with the "wrong" way,
> with symlinks just being a portal to another place entirely, and I
> wish I could pinpoint why. Having the path "faked" after following a
> symlink just feels badly wrong, and I wish I could put a semantic
> reason on it.

The kernel needs to implement this, not any user program.
When a single user program implements this and no others, it
is far more confusing!

By correct I meant maintaining the following:

    <prefix>/<component>/..[/<suffix>] == <prefix>[/<suffix>]
    /..[<suffix>] == /[<suffix>]

where <component> is not a literal ".." but can be any other
directory name or a symlink to anything (including "..").
This path string simplification is done left to right. Thus:

	a/b/c/../../d == a/b/../d == a/d

The kernel needs to keep the full path to $PWD in order to
perform this simplification with relative paths.  In effect
the kernel needs to strip out all .. from a given path before
interpreting it.  And of course the kernel must check that
<component> names an existent directory.

Given these changes symlinks will behave in an unsurprising
way.  Consider:

One user:
cd /tmp
mkdir -p a/b/c
ln -s a/b/c x

# Some other user at some other time:
cd /tmp/d
cd ..
/bin/pwd	# this should show /tmp. At present it shows /tmp/a/b
ln -s d/../a e
cd e		# this should succeed. At present it fails.
ln -s d/../../b f
cd f		# if there is no /b this should fail. At present it succeeds.

In a later email you said "Maybe the fact that no-one seems
to like to keep the pwd in the kernel is argument enough".  A
lack of popularity is not reason enough to reject this; one
must find/understand any technical reasons.  The days of
having to fit a kernel in 64KB are long gone (which could've
been a pragmatic reason once). I can't see a logical argument
as to why the kernel shouldn't keep the full path.  [Of
course, there may be one, but so far I have not seen it]

The only thing that can't be fixed is if the symlink target
disappears. This is the same as when (remote) target of your
bind (or mount) disappears.

Re: [9fans] Tvx update

[erik quanstrom]

> The kernel needs to keep the full path to $PWD in order to
> perform this simplification with relative paths.  In effect
> the kernel needs to strip out all .. from a given path before
> interpreting it.  And of course the kernel must check that
> <component> names an existent directory.

cf. /sys/src/9/port/chan.c:^/fixdotdotname and :/^walk.

- erik

Re: [9fans] Tvx update

[Ethan Grammatikidis]

On 28 May 2010, at 17:04, Bakul Shah wrote:

> On Fri, 28 May 2010 12:51:36 BST Ethan Grammatikidis <eekee57@fastmail.fm
> >  wrote:
>>
>> On 27 May 2010, at 21:16, Bakul Shah wrote:
>>
>>> If BSD had
>>> implemented ".." correctly (i.e. walk back up one level in
>>> the given path), symlinks would have been more useful and
>>> less surprising.
>>
>> This "correct" implementation of symlinks has never seemed right to
>> me. Linux / Bash used to do it the "wrong" way as recently as 2001 if
>> not more recently. I was much more comfortable with the "wrong" way,
>> with symlinks just being a portal to another place entirely, and I
>> wish I could pinpoint why. Having the path "faked" after following a
>> symlink just feels badly wrong, and I wish I could put a semantic
>> reason on it.
>
> The kernel needs to implement this, not any user program.
> When a single user program implements this and no others, it
> is far more confusing!

Indeed! In at least one of my early Linux installs (early to me, not
Linux 0.x) bash implemented this  but nothing else did, which is
probably the basis for my dislike of it, to be honest.

>
> By correct I meant maintaining the following:
>
>    <prefix>/<component>/..[/<suffix>] == <prefix>[/<suffix>]
>    /..[<suffix>] == /[<suffix>]
>
> where <component> is not a literal ".." but can be any other
> directory name or a symlink to anything (including "..").
> This path string simplification is done left to right. Thus:
>
> 	a/b/c/../../d == a/b/../d == a/d
>
> The kernel needs to keep the full path to $PWD in order to
> perform this simplification with relative paths.  In effect
> the kernel needs to strip out all .. from a given path before
> interpreting it.  And of course the kernel must check that
> <component> names an existent directory.
>
> Given these changes symlinks will behave in an unsurprising
> way.  Consider:
>
> One user:
> cd /tmp
> mkdir -p a/b/c
> ln -s a/b/c x
>
> # Some other user at some other time:
> cd /tmp/d
> cd ..
> /bin/pwd	# this should show /tmp. At present it shows /tmp/a/b
> ln -s d/../a e
> cd e		# this should succeed. At present it fails.
> ln -s d/../../b f
> cd f		# if there is no /b this should fail. At present it succeeds.
>
> In a later email you said "Maybe the fact that no-one seems
> to like to keep the pwd in the kernel is argument enough".  A
> lack of popularity is not reason enough to reject this; one
> must find/understand any technical reasons.

Aye, I'm trying to save myself some time in gaining an understanding,
but perhaps that's not such a good idea.

> The days of
> having to fit a kernel in 64KB are long gone (which could've
> been a pragmatic reason once). I can't see a logical argument
> as to why the kernel shouldn't keep the full path.  [Of
> course, there may be one, but so far I have not seen it]
>
> The only thing that can't be fixed is if the symlink target
> disappears. This is the same as when (remote) target of your
> bind (or mount) disappears.
>

Thanks for writing all this, & that goes for everyone involved the
discussion. I read and think about a lot more than I reply to, & often
save posts when I don't understand something immediately.

--
Simplicity does not precede complexity, but follows it. -- Alan Perlis

Re: [9fans] Tvx update

[EBo]

>> Don't change your use of symlinks. I meant more as a global thing: see
>> Korn's paper "Symlinks are a botch".
>
> Can I beg a specific title or reference?  My efforts with google turned
up
> primarily references to your original post (and a former US Ambassador
to
> Togo).

I'm personally betting on an anti economic slavery essay...

Re: [9fans] Tvx update

[EBo]

On Wed, 26 May 2010 16:58:51 -0700, ron minnich <rminnich@gmail.com>
wrote:
> On Wed, May 26, 2010 at 11:02 AM, Nick LaForge <nicklaforge@gmail.com>
> wrote:
>> I tried it and it is very fast to boot and run.  However, the wrapper
>> did not copy Tvx-root to my home dir.  Option 2 links to the package
>> install in /usr/local, but option 1 also links, but to the package
>> loopback mount in /tmp.
> 
> Is this because option 2 is the persistent install and option 1 is
> not? Note the loopback mounts are readonly squashfs mounts.

that sounds like the the persistent home or local is not set up.  It
should have been there by default when it was rebooted.  The three most
likely problems is missing persistent home or usbwait, but it might also
require persistent local (but I did not things so -- I'll try to remember
to test that after some rest)

>> The only files in my home are in the top
>> directory (the license, etc.).  Writing to the symlinks fails.  And so
>> does reading utf8 filenames.
> 
> Be careful .. did you set up a persistent /home? I'm not sure what you
> have done. There's no substitute at some point for seeing how things
> work at tinycore.org.

I haven't seen anything like this, but there are some weird things which
can happen when not setting up the proper boot flags.  Another one that can
cause oddities is not adding usbwait=5 or maybe 10.  This forces the boot
sequence to let the usb drive state catch up with the rest of the machine. 
Try adding that too and see if it helps.

> Thanks for the testing. I have found one thing out ... I don't think
> 9vx is SMP-safe. I've had some real problems building in /sys/src,
> where 9vx gets a segv on "cpu 5" or whatever. If I boot with only one
> cpu, there are no problems. Worth keeping in mind ... I may modify my
> version of 9vx to wire itself to cpu 0, as well as its children, and
> see if the failures I'm seeing go away. That's a pointer at least to
> where the issues may be.

One of the things I noticed fairly consistently.  When recompiling the
system various programs are rebuilt/installed while the system is running. 
Unfortunately 9vx does not deal gracefully with this and sometimes
segfaults.  The solution that has always worked for me, as gross as it is,
is to simply restart 9vx and restart the build.  I forget how often I had
to do this before it completely rebuilt the system but I think ti was
either two or three.  Once it rebuilt everything I can run a mk clean and
then mk all and install then it works without incidence.  I wonder if there
is some weird interaction between my build settings and the target machine.
Maybe I/we can explore that avenue.

> Also ...
> 
>>sed '43 s/H/L/' Tvx
> 
> Sorry, you lost me.
> 
>>Why are there TWO ways for cp to follow symlinks?  Why are there
>>symlinks it all?  (That should do it.)
> 
> They're not going away I bet, but what are you talking about here?

If you are referring to the symlinks in the /tmp directory, not that is by
TCL's design and they consider it a feature not a bug.  So, no I do not
think that TCL will ever get rid of them.


  EBo --

Re: [9fans] Tvx update

[ron minnich]

On Thu, May 27, 2010 at 4:54 AM, EBo <ebo@sandien.com> wrote:

> One of the things I noticed fairly consistently.  When recompiling the
> system various programs are rebuilt/installed while the system is running.
> Unfortunately 9vx does not deal gracefully with this and sometimes
> segfaults.  The solution that has always worked for me, as gross as it is,
> is to simply restart 9vx and restart the build.


I'm fairly certain this is a bug in the vx32 runtime. I can make
failures occur less frequently by booting with smp turned off. Other
ways to explode 9vx include doing a huge commit in sysfromiso.

There's a problem in there somewhere, I'm still trying to work out how
to even debug it ... valgrind is helpless in the face of the vx32
runtime :-)

ron

Re: [9fans] Tvx update

[EBo]

> I'm fairly certain this is a bug in the vx32 runtime. I can make
> failures occur less frequently by booting with smp turned off. Other
> ways to explode 9vx include doing a huge commit in sysfromiso.
>
> There's a problem in there somewhere, I'm still trying to work out how
> to even debug it ... valgrind is helpless in the face of the vx32
> runtime :-)

Yep, vx32 spanks valgrind and sends it home to mama...

I figure I would not worry about it to much until I can reproduce it, or
it looked like it would cause a real problem.

Have you been able to do do anything and cause it to repeatedly die in the
same place?


  EBo --

Re: [9fans] Tvx update

[ron minnich]

On Thu, May 27, 2010 at 7:07 AM, EBo <ebo@sandien.com> wrote:

> Have you been able to do do anything and cause it to repeatedly die in the
> same place?

I can drive it over the edge on SMP if I move a lot of data through it.

It's harder to blow it up on non-SMP but a mk all in /sys/src will do the job.

ron

Re: [9fans] Tvx update

[EBo]

>> Have you been able to do do anything and cause it to repeatedly die in
>> the
>> same place?
>
> I can drive it over the edge on SMP if I move a lot of data through it.

What's your best guess to where to start looking at the problem?

> It's harder to blow it up on non-SMP but a mk all in /sys/src will do
the
> job.

That's how I break it, but I thought it had to do with overwriting
programs while running them.

One test I guess I can run is to rebuild everything and do a diff between
the old and new executables to make sure that they are producing the
binaries between the machines.  I'll see if I can come up with some other
ideas, but I will unlikely be able to spend much time on it as I really
should be focusing on hpd related stuff...


  EBo --

Re: [9fans] Tvx update

[ron minnich]

On Thu, May 27, 2010 at 7:11 PM, EBo <ebo@sandien.com> wrote:

> That's how I break it, but I thought it had to do with overwriting
> programs while running them.

Interesting, but I doubt that's it. Reason being that I can also drive
it to destruction by doing a very large tar pipeline or HG commit.

But what occurred to me is the common thread: lots of I/O to the file system.

Maybe it's as simple as a buffer overflow in devfs-posix.c?

ron

Re: [9fans] Tvx update

[erik quanstrom]

>
> Maybe it's as simple as a buffer overflow in devfs-posix.c?
>

that's trivial to test.  try lotsafiles with ramfs.

- erik

[9fans] tvx update

[EBo]

I'm pleased to announce that the latest update of tvx has been uploaded to
TinyCoreLinux.

Please give it a try and let me know if you find any bugs.


  Best regards,

  EBo --

Re: [9fans] tvx update

[Ryousei Takano]

On Sun, Jun 13, 2010 at 5:43 PM, EBo <ebo@sandien.com> wrote:
>
> I'm pleased to announce that the latest update of tvx has been uploaded to
> TinyCoreLinux.
>
> Please give it a try and let me know if you find any bugs.
>
TinyCoreLinux 3.0 was released on the end of last month.
I could not find the tvx package from the repository of TCL 3.0.
Do you have plan to update the tvx package?

Best regards,
Ryousei

Re: [9fans] tvx update

[EBo]

> TinyCoreLinux 3.0 was released on the end of last month.

Sorry, I was so focused on other things that I missed the announcement.

> I could not find the tvx package from the repository of TCL 3.0.
> Do you have plan to update the tvx package?

yes, I will update it for TCL-3.0, but I'll need a couple of weeks before
I can get at this.  As a note, there were several changes made to 9vx that
happened just after the last release which ended up breaking things, and it
will take a little effort to make sure it is all back to rights.

If I have not gotten back to this by Sept. 6'th (the extended deadline for
iwp9 submissions), then poke me again and I'll see if I can break away
enough time release an update.

  EBo --