[9fans] dcp - a deep copy script, better than dircp

[9fans] dcp - a deep copy script, better than dircp

[Ethan Grammatikidis]

I was never satisfied with dircp. It's practice of copying the contents
of one directory into another seemed limiting at best, obstructive at
worst. The recursive copy options of Gnu cp seemed much more elegant(!),
preserving the usual option syntax of cp and merely extending it slightly
to include directories.

It's no minor issue for me, I move directories around enough that the
lack of a convenient facility to do so would prevent regular usage of any
OS. Perhaps a _move_ as such would challenge the internal structure of
Plan 9 but a convenient deep copy will suffice, not automating removal
after copy just in case it goes wrong.

And so I wrote an rc script. It takes an unlimited number of source files
or directories and copies them all into a destination directory. (Note:
unlike dircp it doesn't copy the _contents_ of the sources, it copies
the sources themselves.) It differs from Gnu cp in that it will fail
if the destination does not exist or is not a directory. In this it
is more consistent, it will always (attempt to) copy src* into dest,
there is no ambiguity in the case of 2 arguments.

NOTE: The script as a whole has not been fully tested. However the
variable assignments which do all the clever stuff have been very
carefully tested.

Posted on the web at http://eekee.org.uk/plan9/scripts/dcp and pasted
here for discussion.

#!/bin/rc

# dcp src* dest
# Deep copy all items _src*_ into directory _dest_.
# Preserves permissions and ownership as far as
# possible, behaving similarly to Gnu cp -a.
#
# This differs from dircp which deep-copies
# the _contents_ of one directory into another.

if ( test $#* -lt 2 ) {
	echo 'usage: dcp from{file|dir} ... todir'
	exit usage
}

nsrc = `{ echo $#* 1 - p q | dc }

sources = $*(1-$nsrc)
# would you believe that line preserves spaces in
# arguments? rc ist gut!

dest = $*($#*)

tar -c $sources | @{builtin cd $dest && tar -xT}

# Note: dircp points its tar processes at the files
# /def/fd/1 and /dev/fd/0. Anyone know why?

Re: [9fans] dcp - a deep copy script, better than dircp

[erik quanstrom]

On Sun Jul 19 12:26:24 EDT 2009, eekee57@fastmail.fm wrote:
> I was never satisfied with dircp. It's practice of copying the contents
> of one directory into another seemed limiting at best, obstructive at
> worst. The recursive copy options of Gnu cp seemed much more elegant(!),
> preserving the usual option syntax of cp and merely extending it slightly
> to include directories.

see mkfs(8).  i keep /tmp/allproto around with the contents of '+'.
though i can't remember the last time i used it.

- erik

Re: [9fans] dcp - a deep copy script, better than dircp

[Richard Miller]

> i keep /tmp/allproto around with the contents of '+'.

There's also one in /sys/lib/sysconfig/proto/allproto,
but that takes longer to type.

Re: [9fans] dcp - a deep copy script, better than dircp

[Francisco J Ballesteros]

<{echo +}
works just fine.

On Sun, Jul 19, 2009 at 8:16 PM, Richard Miller<9fans@hamnavoe.com> wrote:
>> i keep /tmp/allproto around with the contents of '+'.
>
> There's also one in /sys/lib/sysconfig/proto/allproto,
> but that takes longer to type.
>
>
>

Re: [9fans] dcp - a deep copy script, better than dircp

[Ethan Grammatikidis]

On Sun, 19 Jul 2009 14:05:04 -0400
erik quanstrom <quanstro@quanstro.net> wrote:

> On Sun Jul 19 12:26:24 EDT 2009, eekee57@fastmail.fm wrote:
> > I was never satisfied with dircp. It's practice of copying the contents
> > of one directory into another seemed limiting at best, obstructive at
> > worst. The recursive copy options of Gnu cp seemed much more elegant(!),
> > preserving the usual option syntax of cp and merely extending it slightly
> > to include directories.
>
> see mkfs(8).  i keep /tmp/allproto around with the contents of '+'.
> though i can't remember the last time i used it.

Before I say anythign daft, what's '+'? It does not appear to be special on my system.


--
Ethan Grammatikidis

Those who are slower at parsing information must
necessarily be faster at problem-solving.

Re: [9fans] dcp - a deep copy script, better than dircp

[Charles Forsyth]

>Before I say anythign daft, what's '+'? It does not appear to be special on my system.

it's interpreted by mkfs in its proto file to mean all the substructure of a directory.
see mkfs(8).

Re: [9fans] dcp - a deep copy script, better than dircp

[Ethan Grammatikidis]

On Mon, 20 Jul 2009 10:55:21 +0100
Charles Forsyth <forsyth@terzarima.net> wrote:

> >Before I say anythign daft, what's '+'? It does not appear to be special on my system.
>
> it's interpreted by mkfs in its proto file to mean all the substructure of a directory.
> see mkfs(8).
>

Thanks. I looked over the man page last night & somehow missed the "Mkfs copies only those files that are out of date" part. Ah... that part would be more relevant to cej than me.

I'm primarily concerned with moving dirs and I already know what tar does. Mkfs seemed very confusing at first, it's confusingly-named for one thing (it does not make filesystems & wtf does mkext stand for), and IMHO default destinations belong in a script, not a compiled binary. Also I'm absurdly pleased with how terse dcp is. Apart from the option check there are 4 lines of code. :)

I have half a mind to write a new script using mkfs to get the functionality cej would like, but:

	file /bin/mkfs
	/bin/mkfs: cannot open: '/bin/mkfs' file does not exist

Dude, like, huh?

--
Ethan Grammatikidis

Those who are slower at parsing information must
necessarily be faster at problem-solving.

Re: [9fans] dcp - a deep copy script, better than dircp

[erik quanstrom]

> & [what] does mkext stand for), and

make extract, i assume.

>
> 	file /bin/mkfs
> 	/bin/mkfs: cannot open: '/bin/mkfs' file does not exist
>
> Dude, like, huh?

you must have missed the first few lines of the man page

          disk/mkfs [-aprvxU] [-d root] [-n name] [-s source] [-u
          users] [-z n] proto ...

          disk/mkext [-d name] [-u] [-h] [-v] [-x] [-T] file ...

this means mkfs is in /bin/disk plan 9 uses subdirectories
in /bin.  actually in the fs, mkfs is in the directory
/$objtype/bin/disk/

- erik

Re: [9fans] dcp - a deep copy script, better than dircp

[Ethan Grammatikidis]

On Mon, 20 Jul 2009 07:50:08 -0400
erik quanstrom <quanstro@quanstro.net> wrote:

> > & [what] does mkext stand for), and
>
> make extract, i assume.
>
> >
> > 	file /bin/mkfs
> > 	/bin/mkfs: cannot open: '/bin/mkfs' file does not exist
> >
> > Dude, like, huh?
>
> you must have missed the first few lines of the man page
>
>           disk/mkfs [-aprvxU] [-d root] [-n name] [-s source] [-u
>           users] [-z n] proto ...
>
>           disk/mkext [-d name] [-u] [-h] [-v] [-x] [-T] file ...
>
> this means mkfs is in /bin/disk plan 9 uses subdirectories
> in /bin.  actually in the fs, mkfs is in the directory
> /$objtype/bin/disk/

Ah, cheers. I do rather feel like I've been recommended to use a c-stoff/t-stoff powered rocket hopper where a family car would suffice though.


--
Ethan Grammatikidis

Those who are slower at parsing information must
necessarily be faster at problem-solving.

Re: [9fans] dcp - a deep copy script, better than dircp

[Steve Simon]

> ...c-stoff/t-stoff powered rocket...

<oblique UK reference>
I watched OU programs as a child too :-)
</oblique UK reference>

I suggest you consider why you are moving directories about, I have just got
out of the habit.

If I get a tar or a zip which contains dome data I need I just mount it with
fs/tarfs or fs/zipfs and look inside. If I want just a few files I cp them to
$home, if I want most of it then I extract the image to the destination where
it is needed.

Basicially I put a bit more thought up-front as to where I want to put stuff
and then put it there.

If you want dircp as a backup mechnism then can I suggest either fossil and venti,
or, mk9660(8).

mk9660 creates a dump-like heirarchy in a single ISO image, mergeing multiple dumps
into the single ISO, stripping dumplicate files as it goes; kudos to wkj I believe.

-Steve

Re: [9fans] dcp - a deep copy script, better than dircp

[Ethan Grammatikidis]

On Mon, 20 Jul 2009 15:00:43 +0100
"Steve Simon" <steve@quintile.net> wrote:

> > ...c-stoff/t-stoff powered rocket...
>
> <oblique UK reference>
> I watched OU programs as a child too :-)
> </oblique UK reference>

I didn't, my parents would never have a TV. ::) I got that from a book, The World's Worst Aircraft.

> I suggest you consider why you are moving directories about, I have just got
> out of the habit.

> Basicially I put a bit more thought up-front as to where I want to put stuff
> and then put it there.

Sure. Let me get a modest little cybernetic interface to one of those temporal vision thingummabobs they have these days and I'll call you right back.

Sorry if that was a bit harsh, but I've had far too much 'advice' to 'just do this easy little thing'... Computers are supposed to supplement the brain, to help, not require more (and in some cases quite impossible) work. To file anything so you can find it again requires experience in filing that particular information type. I'm constantly dealing with new data...


--
Ethan Grammatikidis

Those who are slower at parsing information must
necessarily be faster at problem-solving.

Re: [9fans] dcp - a deep copy script, better than dircp

[erik quanstrom]

> Sure. Let me get a modest little cybernetic interface to one of those temporal vision thingummabobs they have these days and I'll call you right back.
>
> Sorry if that was a bit harsh, but I've had far too much 'advice' to 'just do this easy little thing'... Computers are supposed to supplement the brain, to help, not require more (and in some cases quite impossible) work. To file anything so you can find it again requires experience in filing that particular information type. I'm constantly dealing with new data...

9fans interface:

1.  principles of operation

9fans is a forum where one can make statements
and ask questions.  those asking questions must
be prepared to accept answers. [...]

- erik

Re: [9fans] dcp - a deep copy script, better than dircp

[Dan Cross]

On Mon, Jul 20, 2009 at 11:02 AM, Ethan
Grammatikidis<eekee57@fastmail.fm> wrote:
> Sorry if that was a bit harsh, but I've had far too much 'advice' to 'just do this easy little thing'... Computers are supposed to supplement the brain, to help, not require more (and in some cases quite impossible) work. To file anything so you can find it again requires experience in filing that particular information type. I'm constantly dealing with new data...

Not to criticize, but I think the suggestion that Steve is making is
that, in order to better use the computer to supplement the brain and
help, it's best to use the tools of this particular computer system in
the most natural way, versus trying to use it as merely an improved
Linux or Unix or what have you.  Now, I'm not suggesting that you
don't understand Plan 9, or that the underlying reasons you are moving
directories around aren't valid, of course!  If you're data is new,
then good to go.  If not, then I think the advise is for some things
that you may find simplify your life, not make it harder.

Re: [9fans] dcp - a deep copy script, better than dircp

[Ethan Grammatikidis]

On Mon, 20 Jul 2009 11:19:16 -0400
Dan Cross <crossd@gmail.com> wrote:

> On Mon, Jul 20, 2009 at 11:02 AM, Ethan
> Grammatikidis<eekee57@fastmail.fm> wrote:
> > Sorry if that was a bit harsh, but I've had far too much 'advice' to 'just do this easy little thing'... Computers are supposed to supplement the brain, to help, not require more (and in some cases quite impossible) work. To file anything so you can find it again requires experience in filing that particular information type. I'm constantly dealing with new data...
>
> Not to criticize, but I think the suggestion that Steve is making is
> that, in order to better use the computer to supplement the brain and
> help, it's best to use the tools of this particular computer system in
> the most natural way, versus trying to use it as merely an improved
> Linux or Unix or what have you.

This is about my data, in files, on a filesystem. If Plan 9 has difficult restrictions in it's filesystem model then yeah, I have to use something else, but where else can I find namespaces and 9p? Ok, Inferno & Octopus; the latter I ought to have a look at, the former might work for me but I found Plan 9 a couple of years earlier, I feel a little bit more sure of where I am with Plan 9, and Inferno is frankly a bit messy. I suppose you guys think I'll make my Plan 9 installation 'a bit messy' if I carry on, eh? You may be right. :)

Ugh, you got me _thinking_. :) If you have time for a mini-essay on Plan 9 and human beings, read on.

What I really want to say is there's nothing natural about not letting user X reorganise his stuff. The world is change, the universe is changing constantly, if slowly. Humans are adapted to that change, some better than others, but more than that we're part of the changing world. We change, we grow, and we don't have all possible ideas pertaining to X the moment we acquire X. Some 'see' more than others, and quicker, and those I guess are better equipped to organize their data naturally as they acquire it, but give them a completely new field of data to work with and I'm sure they would initially be as disorganized as anyone else and would soon find they would have to reorganize.

I realize I don't want Plan 9 exactly as it is. I already knew I didn't want the available user interfaces _quite_ the way they are and this discussion provides me with something else to think about in that regard. However I do seriously want 9p, namespaces, process isolation, interface simplicity... The potential value these features have in relation to the goals I would like to achieve is incalculable! However, to achieve my goals I have to work with the rest of Plan 9. I have to bend it until I can use it to get where I want to go.

*sigh* Now you've got me thinking about seperate types of filesystems for user data and system data, or perhaps personal data and public data. I have other things to do, dammit. >:D


--
Ethan Grammatikidis

Those who are slower at parsing information must
necessarily be faster at problem-solving.

Re: [9fans] dcp - a deep copy script, better than dircp

[Jason Catena]

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

I've been thinking for a while that I don't quite like directories, as
organizing metadata.  Too restrictive: you can only really choose one place,
to find something you already have to know where to look (or search through
everything), and if you choose long, comprehensible names, your unique set
of metadata to find a file becomes very long.

Consider, by contrast, a service like Evernote.com (KPhotoAlbum also does
tags well, and delicious.com for web pages).  It has folders, yes, but
what's better is that it lets you assign keywords to the note (really just
arbitrary textual (or other) data).  The note's topic is in essence the
combination of all the keywords, so if you are interested in one of those
keywords in the future, looking for the keyword (or some combination of
keywords) would list all the notes with those keywords attached.

I think the latter approach would produce a more usable, less frustrating
filesystem than the former.

Jason Catena

[-- Attachment #2: Type: text/html, Size: 1103 bytes --]

Re: [9fans] dcp - a deep copy script, better than dircp

[erik quanstrom]

>
> I think the latter approach would produce a more usable, less frustrating
> filesystem than the former.
>

plan 9 seems to me to be firmly in the fs rather than
the database camp.

before i moved to plan 9, i thought that locate was a
very cool tool.  things had gotten mighty hard to find
in linux.  since i've moved to plan 9, i just don't have
that problem.  even when struck with real desperation
on coraid's worm, a find on main takes not too long:

minooka; cd /n/ila
minooka; time rc -c 'find . | wc'
 356164  356164 13987863
1.24u 1.38s 6.65r 	 rc -c find . | wc

- erik

Re: [9fans] dcp - a deep copy script, better than dircp

[Jack Johnson]

On Mon, Jul 20, 2009 at 2:41 PM, erik quanstrom<quanstro@quanstro.net> wrote:
> on coraid's worm, a find on main takes not too long:
>
> minooka; cd /n/ila
> minooka; time rc -c 'find . | wc'
>  356164  356164 13987863
> 1.24u 1.38s 6.65r        rc -c find . | wc

The FAQ also mentions:

du -a . | grep foo

Just out of curiosity, how does find vs du compare for you?

-Jack

Re: [9fans] dcp - a deep copy script, better than dircp

[erik quanstrom]

minooka; time rc -c 'du -a . | wc'
> On Mon, Jul 20, 2009 at 2:41 PM, erik quanstrom<quanstro@quanstro.net> wrote:
> > on coraid's worm, a find on main takes not too long:
> >
> > minooka; cd /n/ila
> > minooka; time rc -c 'find . | wc'
> >  356164  356164 13987863
> > 1.24u 1.38s 6.65r        rc -c find . | wc
>
> The FAQ also mentions:
>
> du -a . | grep foo
>
> Just out of curiosity, how does find vs du compare for you?

i know you can do it with du, but it seems a bit "cat -n"ish to me.
for comparison:

minooka; time rc -c 'du -a . | wc'
 356181  712362 14825492
1.91u 4.22s 8.66r 	 rc -c du -a . | wc

- erik

Re: [9fans] dcp - a deep copy script, better than dircp

[Dan Cross]

On Mon, Jul 20, 2009 at 6:56 PM, erik quanstrom<quanstro@quanstro.net> wrote:
> i know you can do it with du, but it seems a bit "cat -n"ish to me.
> for comparison:

This was why I wrote 'walk' a few years ago; du is the disk usage
tool, not a general file walker.

        - Dan C.

Re: [9fans] dcp - a deep copy script, better than dircp

[Federico G. Benavento]

btw, shouldn't mkfs use allproto by default like mk9660 does
or am I just missing something?

On Mon, Jul 20, 2009 at 6:55 AM, Charles Forsyth<forsyth@terzarima.net> wrote:
>>Before I say anythign daft, what's '+'? It does not appear to be special on my system.
>
> it's interpreted by mkfs in its proto file to mean all the substructure of a directory.
> see mkfs(8).
>
>



--
Federico G. Benavento

Re: [9fans] dcp - a deep copy script, better than dircp

[cej]

[-- Attachment #1: Type: text/plain, Size: 381 bytes --]

hi,

i'm quite happy with the 'cpdir' by Kenji Arisawa (thanks, Kenji!) on sources/contrib/arisawa.
However, your scripty seems fine, too. Could we add a switch to conform with gnu's cp -au? Just not to overwrite newer files. I don't know there is an option there in 'tar' (I can't see in  in tar(1)). At least we have the 'k' modifier, which could help.

Regards,
++pac.


[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 2620 bytes --]

Re: [9fans] dcp - a deep copy script, better than dircp

[Ethan Grammatikidis]

On Mon, 20 Jul 2009 11:32:11 +0200
<cej@gli.cas.cz> wrote:

> hi,
>
> i'm quite happy with the 'cpdir' by Kenji Arisawa (thanks, Kenji!) on sources/contrib/arisawa.
> However, your scripty seems fine, too. Could we add a switch to conform with gnu's cp -au? Just not to overwrite newer files. I don't know there is an option there in 'tar' (I can't see in  in tar(1)). At least we have the 'k' modifier, which could help.

Tar's -k modifier is the nearest I can find. Mkext has no such option and I've found no cpio (thankfully?). I don't know where to look for or suggest such an option. It may seem natural in tar, but there's a danger with archive utilities that their feature list may grow well beyond a comfortable size for the ordinary user.

Also a copy which omits files would be much faster if it did not read the files which it is going to omit, which rather rules out archive_util | archive_util usage. I think the option would be better added to arisawa's cpdir, but I'm too lazy. :)

--
Ethan Grammatikidis

Those who are slower at parsing information must
necessarily be faster at problem-solving.

Re: [9fans] dcp - a deep copy script, better than dircp

[arisawa]

Hello,

I found this mail today. sorry.

> Just not to overwrite newer files

cpdir does behave this way. 


2009/07/20 18:32、cej@gli.cas.cz のメール：

> hi,
> 
> i'm quite happy with the 'cpdir' by Kenji Arisawa (thanks, Kenji!) on sources/contrib/arisawa.
> However, your scripty seems fine, too. Could we add a switch to conform with gnu's cp -au? Just not to overwrite newer files. I don't know there is an option there in 'tar' (I can't see in  in tar(1)). At least we have the 'k' modifier, which could help.
> 
> Regards,
> ++pac.
> 
> <winmail.dat>

Re: [9fans] dcp - a deep copy script, better than dircp

[erik quanstrom]

On Thu Apr 17 08:54:02 EDT 2014, arisawa@ar.aichi-u.ac.jp wrote:
> Hello,
>
> I found this mail today. sorry.
>
> > Just not to overwrite newer files
>
> cpdir does behave this way.

mkfs already does this.  :-)

- erik