Re: Bug#479764: shell builtin mv fails to move files across devices

Re: Bug#479764: shell builtin mv fails to move files across devices

[Clint Adams]

On Tue, May 06, 2008 at 03:42:08PM +0100, martin f krafft wrote:
> lapse:~/debian|master|debian.ch% echo $ZSH_VERSION
> 4.3.6-dev-0+0428
> lapse:~/debian|master|debian.ch% zmodload zsh/files
> lapse:~/debian|master|debian.ch% which mv
> mv: shell built-in command
> lapse:~/debian|master|debian.ch% mv /tmp/debian.ch_T-Shirt_20080422-poll.png .
> mv: /tmp/debian.ch_T-Shirt_20080422-poll.png: invalid cross-device link

Anyone up for changing this in a future version?

    Note  that  this  mv  will not move files across devices.  Historical
    versions of mv, when actual renaming is impossible, fall back on copying
    and removing files; if this behaviour is desired, use cp and rm manually.
    This  may  change  in  a future version.

Re: Bug#479764: shell builtin mv fails to move files across devices

[Bart Schaefer]

On May 6,  4:03pm, Clint Adams wrote:
}
} Anyone up for changing this in a future version?
} 
}     Note  that  this  mv  will not move files across devices.  Historical
}     versions of mv, when actual renaming is impossible, fall back on copying
}     and removing files; if this behaviour is desired, use cp and rm manually.
}     This  may  change  in  a future version.

I'd rather we didn't.  rename(2) is atomic, but there's no reasonable way
to make cp+rm be so, and I think it's better if the user is forced to be
aware of the issue.

Also note that "cp" is (intentionally) NOT one of the commands supplied
by the zsh/files module, so changing mv means implementing cp.

Re: Bug#479764: shell builtin mv fails to move files across devices

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 913 bytes --]

also sprach Bart Schaefer <schaefer@brasslantern.com> [2008.05.06.1645 +0100]:
> I'd rather we didn't.  rename(2) is atomic, but there's no
> reasonable way to make cp+rm be so, and I think it's better if the
> user is forced to be aware of the issue.
> 
> Also note that "cp" is (intentionally) NOT one of the commands
> supplied by the zsh/files module, so changing mv means
> implementing cp.

What, if I may ask, is the purpose of zsh/files?

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems
 
you can't assign IP address 127.0.0.1 to the loopback adapter,
because it is a reserved address for loopback devices.
                                  -- micro$oft windoze xp professional

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Bug#479764: shell builtin mv fails to move files across devices

[Peter Stephenson]

On Tue, 6 May 2008 17:55:55 +0100
martin f krafft <madduck@debian.org> wrote:
> also sprach Bart Schaefer <schaefer@brasslantern.com> [2008.05.06.1645 +0100]:
> > I'd rather we didn't.  rename(2) is atomic, but there's no
> > reasonable way to make cp+rm be so, and I think it's better if the
> > user is forced to be aware of the issue.
> > 
> > Also note that "cp" is (intentionally) NOT one of the commands
> > supplied by the zsh/files module, so changing mv means
> > implementing cp.
> 
> What, if I may ask, is the purpose of zsh/files?

Its primary purpose has been as a fallback on a system where the shell
is working but nothing much else is.  Hence it provides the limited,
easy-to-implement things you're likely to need to fix what needs
fixing.  Moving things between file systems typically isn't something
you need (though in any given case it may be, as well as a whole load of
other stuff the shell doesn't provide).

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: Bug#479764: shell builtin mv fails to move files across devices

[Phil Pennock]

[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]

On 2008-05-06 at 17:55 +0100, martin f krafft wrote:
> What, if I may ask, is the purpose of zsh/files?

If I correctly recall the gist of a verbal conversation about 11 years
ago with the author, in a university environment with heavily multi-user
systems which were often abused by people writing My First Fork Bomb, it
was to have the availability to load in sufficient builtins to make it
easier to deal with systems which won't let you fork() anymore more; to
do basic recovery work and have the commands available; to avoid fork()
overheads when dealing with a loaded single-CPU Solaris box, etc.

But mostly, IIRC, to build a static version of the shell with the module
already loaded, to put on a boot/recovery diskette and so reduce what
was needed.

It pre-dates busybox, which has since become the common way of doing
this.

It wasn't intended to provide full versions of the commands, the
commands which are there don't implement all the POSIX options, etc.
It's intended to provide enough to be useful in emergencies.

Most people shouldn't need to use it.

Mind, it's availability was one of the factors which led to the author
successfully convincing me to switch shells to zsh.  :^)  There were
times when it _really_ mattered.

It's probably worth adding a note to the documentation that the
implementations are not complete (for Standards compliance) and that the
module should probably only be loaded for emergency recovery situations;
loading it for routine use is premature over-optimisation.

-Phil

[-- Attachment #2: Type: application/pgp-signature, Size: 163 bytes --]

Re: Bug#479764: shell builtin mv fails to move files across devices

[Vincent Lefevre]

On 2008-05-06 13:55:23 -0700, Phil Pennock wrote:
> Most people shouldn't need to use it.

I use it *by default* just because of the Linux kernel bug^Wlimitation
concerning the argument list (I often exceed it). But perhaps I should
write my own shell function as a wrapper to coreutils and use xargs or
zargs when necessary.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: Bug#479764: shell builtin mv fails to move files across devices

[Phil Pennock]

On 2008-05-07 at 23:30 +0200, Vincent Lefevre wrote:
> On 2008-05-06 13:55:23 -0700, Phil Pennock wrote:
> > Most people shouldn't need to use it.
> 
> I use it *by default* just because of the Linux kernel bug^Wlimitation
> concerning the argument list (I often exceed it). But perhaps I should
> write my own shell function as a wrapper to coreutils and use xargs or
> zargs when necessary.

Would it make more sense for your usage if Zsh used the new "features"
capabilities to resolve this?  Would this also help with the Debian
situation, in addition to documentation notes?

They were added by Peter after I sent a proposed patch renaming
zsh/stat's stat to zstat, with a stat deprecated backwards-compatible
name, since a stat(1) external command has come along in the years since
zsh's stat module was introduced with incompatible behaviour.

Peter just looked at the logical conclusion and put in a lot more work
to do the Right Thing.

For zsh/files it's not quite as simple as adding a z in front of the
names, since we already have zmv, but if there were alternate names
which would allow you to use Features to load only those names?

-Phil

PS: why single Linux out for doing what every Unix variant does?

Re: Bug#479764: shell builtin mv fails to move files across devices

[Vincent Lefevre]

On 2008-05-07 14:47:45 -0700, Phil Pennock wrote:
> Would it make more sense for your usage if Zsh used the new "features"
> capabilities to resolve this?

Yes. IIRC, the only features (standard or not) I really miss are
the -v option (verbose), the -n option for ln, the -u option for
mv, and mv across devices.

> For zsh/files it's not quite as simple as adding a z in front of the
> names, since we already have zmv, but if there were alternate names
> which would allow you to use Features to load only those names?

Names are not really important for me since it is easy to write a
shell function like:

rm() { command rm $@ }

or

rm() { builtin rm $@ }

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: Bug#479764: shell builtin mv fails to move files across devices

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

also sprach Vincent Lefevre <vincent@vinc17.org> [2008.05.07.2350 +0100]:
> rm() { command rm $@ }

"$@" :_)

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems
 
"one should never do anything that
 one cannot talk about after dinner."
                                                        -- oscar wilde

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Bug#479764: shell builtin mv fails to move files across devices

[Vincent Lefevre]

On 2008-05-08 00:03:36 +0100, martin f krafft wrote:
> also sprach Vincent Lefevre <vincent@vinc17.org> [2008.05.07.2350 +0100]:
> > rm() { command rm $@ }
> 
> "$@" :_)

I have SH_WORD_SPLIT unset, but that's true it's better not to depend
on this...

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: Bug#479764: shell builtin mv fails to move files across devices

[Stephane Chazelas]

On Thu, May 08, 2008 at 05:23:34AM +0200, Vincent Lefevre wrote:
> On 2008-05-08 00:03:36 +0100, martin f krafft wrote:
> > also sprach Vincent Lefevre <vincent@vinc17.org> [2008.05.07.2350 +0100]:
> > > rm() { command rm $@ }
> > 
> > "$@" :_)
> 
> I have SH_WORD_SPLIT unset, but that's true it's better not to depend
> on this...
[...]

SH_WORD_SPLIT... and GLOB_SUBST.

But even without those, $@ and "$@" are different as $@ removes
the empty elements (which shouldn't be a problem with rm
though).

-- 
Stéphane

Re: Bug#479764: shell builtin mv fails to move files across devices

[Mikael Magnusson]

2008/5/7 Vincent Lefevre <vincent@vinc17.org>:
> On 2008-05-06 13:55:23 -0700, Phil Pennock wrote:
>  > Most people shouldn't need to use it.
>
>  I use it *by default* just because of the Linux kernel bug^Wlimitation
>  concerning the argument list (I often exceed it). But perhaps I should
>  write my own shell function as a wrapper to coreutils and use xargs or
>  zargs when necessary.

This bug^Wlimitation has been fixed^Wraised in linux 2.6.23 from some
hundred kilobytes to 1/4 of ulimit -s, so by default 2MB but you can
make it limited only by available ram if you want.

-- 
Mikael Magnusson

ps I don't know how much of the cc i'm supposed to keep, but this is
sort of off topic to the debian bug anyway so i guess it doesn't
really matter.

Re: Bug#479764: shell builtin mv fails to move files across devices

[Vincent Lefevre]

On 2008-05-08 00:06:21 +0200, Mikael Magnusson wrote:
> This bug^Wlimitation has been fixed^Wraised in linux 2.6.23 from some
> hundred kilobytes to 1/4 of ulimit -s, so by default 2MB but you can
> make it limited only by available ram if you want.

I didn't know that (well, a few years ago I heard that something like
that was planned). Now, I still use some Linux machines that have a
2.6.18 kernel, and for some reason they probably won't be upgraded.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: Bug#479764: shell builtin mv fails to move files across devices

[Stephane Chazelas]

On Tue, May 06, 2008 at 04:03:03PM +0100, Clint Adams wrote:
> On Tue, May 06, 2008 at 03:42:08PM +0100, martin f krafft wrote:
> > lapse:~/debian|master|debian.ch% echo $ZSH_VERSION
> > 4.3.6-dev-0+0428
> > lapse:~/debian|master|debian.ch% zmodload zsh/files
> > lapse:~/debian|master|debian.ch% which mv
> > mv: shell built-in command
> > lapse:~/debian|master|debian.ch% mv /tmp/debian.ch_T-Shirt_20080422-poll.png .
> > mv: /tmp/debian.ch_T-Shirt_20080422-poll.png: invalid cross-device link
> 
> Anyone up for changing this in a future version?
> 
>     Note  that  this  mv  will not move files across devices.  Historical
>     versions of mv, when actual renaming is impossible, fall back on copying
>     and removing files; if this behaviour is desired, use cp and rm manually.
>     This  may  change  in  a future version.

Note that POSIX requires mv to be able to move files (and now
directories) accross file systems. Note that it's quite a
complex task as it involves recreating the files as they were
(permissions, ownership, ACLs, sparse files), handle conflicts,
handle hard links correctly, and handle the error cases
properly. It's not as simple as cp+rm, it would rather be pax
-rw && rm -rf 

-- 
Stéphane