Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Watch for undocumented behavior of cp -R UNIX
The man page for cp says that cp -R directory1 directory2 "...copies the directory and the entire subtree connected at that point." But it doesn't always do that! For example, do the following:
$ cd /tmp
$ mkdir Afolder
$ cd Afolder
$ touch file1
$ touch file2
$ touch file3
$ cd ..
$ cp -R Afolder ~/
If you now cd to your home folder, you will find Afolder there, with files 1, 2, and 3 inside it. This is the documented behavior. But, now do the same experiment again, but this time use this copy command:
$ cp -R Afolder/ ~/
If you look in your home directory now, you will find files 1, 2, and 3 there, but no Afolder. Darwin takes the trailing / on Afolder as a flag to tell it to copy the contents only, and not the enclosing directory Afolder. This is either a bug, or a very annoying feature. You might ask who would put a trailing / on a folder anyhow? Well, if you use Tab completion, as in Afol [Tab] to complete the name automatically, the terminal will automatically add the trailing /.

The workaround is to watch carefully for the trailing /. But it is easy to get caught, which can be a disaster when you are copying a folder containing lots of files with names similar to ones in the target directory.
    •    
  • Currently 3.00 / 5
  You rated: 5 / 5 (5 votes cast)
 
[14,864 views]  

Watch for undocumented behavior of cp -R | 38 comments | Create New Account
Click here to return to the 'Watch for undocumented behavior of cp -R' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Watch for undocumented behavior of cp -R
Authored by: gullevek on Nov 16, '05 06:57:17AM

Yes, this is "evil". I had a bad experience this on a FreeBSD box where I than had to "hand pick" the files from my home directory that should have been in a subdir.

I never saw a similar thing in the GNU cp version. I think this is a BSD cp only issue. But its an evil one.



[ Reply to This | # ]
Standard Behavior
Authored by: forman on Nov 16, '05 07:09:21AM
This is the standard behavior of all command-line utilities, whose behavior varies depending on whether a directory or the contents of a directory are specified. You'll see the same behavior for programs such as mv, scp, rsync, etc.

The brief rule is this: without a trailing slash, the source directory itself is selected; with the trailing slash, the contents of that directory are selected.

Michael.

[ Reply to This | # ]
Standard Behavior
Authored by: signal15 on Nov 16, '05 07:13:10AM

I agree, this is standard (and useful) behavior. In fact, I think it actually is documented somewhere, I seem to remember reading this in some man pages years and years ago. cp has worked like this since at least 1990. :)



[ Reply to This | # ]
Standard Behavior
Authored by: owsla on Nov 16, '05 07:21:42AM

Frankly, if I wanted the files IN the directory, I would do something like:

cp mydirectory/* newdirectory/

Silently deciding that for me if I didn't specify the * is cruel.

That's interesting that non-GNU cp has been like this since 1990. I definitely think the rest of the Unix world should switch over.



[ Reply to This | # ]
Standard Behavior
Authored by: forman on Nov 16, '05 08:03:36AM
Your proposal of eliminating the directory-contents specifier "/" and instead choosing files in a directory manually using syntax such as mydirectory/* has the problem of omitting dot files.

Given that this behavior is a long-standing documented feature, this is not a software problem rather it is a user problem. The point of the parent post is to teach folks a command-line nuance, not call for the rewriting of dozens of applications to conform to the limitations of inexperienced users. Learn the trick and move on. :)

Michael.

[ Reply to This | # ]
Standard Behavior
Authored by: owsla on Nov 16, '05 10:18:10AM

Experience or inexperience isn't the issue ... it's GNU-style vs BSD-style. I've spent eight years in GNU-style environments. The first time BSD cp did this to me was really unnerving.

I just wish there was a way to change the behavior. Installing GNU cp is counter-productive on Tiger (unlike on Panther) because only the Apple-supplied cp has the resource fork patches.



[ Reply to This | # ]
Gnu inconsistent
Authored by: SOX on Nov 16, '05 10:48:01AM

GNU sometimes ignores the slash and sometimes uses the slash to have such behavious. Compare cp and rsync on GNU and you will see what I mean. At least the BSD pattern is consisent. THat said I never did like that confusion over what / does. I have to re-read the rsync manpage every time I use it!



[ Reply to This | # ]
Gnu inconsistent
Authored by: 3oz on Nov 18, '05 07:25:59AM

rsync is not written by the GNU Free Software Foundation. It is licensed with the GPL. GNU bundles many of the basic command line programs like 'cd, cp, mkdir, rm....' but rsync is written by Andrew Tridgell, the same guy that started the Samba project.



[ Reply to This | # ]
UTSL
Authored by: hayne on Nov 16, '05 11:12:43AM
Installing GNU cp is counter-productive on Tiger (unlike on Panther) because only the Apple-supplied cp has the resource fork patches.
Of course you could submit the appropriate patches to the GNU project or the Fink project (or just patch your own version of /sw/bin/cp) The Apple source code for 'cp' is available from www.opensource.apple.com (cp.c is under "file_cmds").
The code that handles the copying of resource forks is in copyfile.c (which is in "Libc")

[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: owsla on Nov 16, '05 07:11:51AM

Indeed, the GNU version of cp does not have this behavior. Quite strange.

Anothe reason this "feature" is evil is because bash tab-completion puts the trailing / on the directory name for you. Instead of being able to hit space and move on to the next directory, you have to remove the / when typing out cp commands with lots of directories. I've definitely been tripped-up by that when I'm not paying attention.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: the1truestripes on Nov 16, '05 10:00:18AM

FYI the tab completion in zsh strips the trailing / back off if you hit space or return just after the completion.

[ Reply to This | # ]

Watch for undocumented behavior of cp -R
Authored by: erikzred on Nov 16, '05 07:12:58AM

This seems normal to me. Maybe since my only "NIX" experience is with OS X I never knew there was another way to expect. Seems logical to me.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: TvE on Nov 16, '05 07:31:25AM

"The man page for cp says that cp -R directory1 directory2 "…
then WHY do you do somehing different like:
"cp -R directory1/ directory2/" ???

I mean - you're clearly adding characters to the command but still expect it to do what you THINK about.

I am fairly sure the difference is there for a reason, and I actually find it very logical:
"/blablabla/" means the content of the directory "/blablabla" and "/blablabla" is the before mentioned folder AND all it's content…



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: allanj37 on Nov 16, '05 07:47:36AM

Yes, this behavior is logical if you think about it, but the point is that it is still easy to accidently do the wrong one, especially with tab-completion.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: adrianm on Nov 16, '05 08:06:01AM

If you use zsh instead of bash, the trailing slash is removed after tab completion if you don't do anything else with it.

Reason to use zsh #472



[ Reply to This | # ]
more dangerous
Authored by: melvinwalker on Nov 16, '05 10:56:49AM

Which is exactly why command lines are considered more dangerous than a GUI.

If someone wants to experiment with the command line, do it in a user account that has a lot of restrictions first. That's not always possible, but it's safer than using your administrator account.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: auricgoldfinger on Nov 16, '05 12:02:52PM

Well, / is a delimiter to identify folder-files. So it actually isn't quite logical that you copy the directory under folder/ without copying the folder itself too.

And why you would do the cp -R folder/ thing?
mkdir temporary
cp -R tem<tab>

If you hit the tab then, bash will autocomplete to "temporary/"
Note the trailing slash?
That's why someone would like to add the /



[ Reply to This | # ]
Another Command With Similar Behavior
Authored by: bfad on Nov 16, '05 08:41:16AM
To add to the list, ditto also behaves similarily

[ Reply to This | # ]
Ditto and other mistakes by Apple
Authored by: Mattbot on Nov 16, '05 11:42:43AM

Ditto ignores the slash altogether and ONLY copies the contents of the source directory into the destination. Copying the source directory itself isn't possible, only sub-directories.

Ditto is a fine tool for making packages but a crap replacement for cp. Unfortunately, prior to Tiger, ditto was what Mac OS X used for CLI copying of files with resource forks. Which is why I think HFS+ aware cp was one of the biggest improvement in Tiger.

The real debate shouldn't be BSD cp v. GNU cp but why Darwin cp didn't work with HFS+ file systems since at least Mac OS X Beta. Or perhaps even why Apple stays with slow clunky HFS+ in the first place as there are several higher performance file systems available. Having to wait this long for a useful version of the *NIX standard file copying utility was absurd. Way to drop the ball, Apple.



[ Reply to This | # ]
BSD vs. SysV
Authored by: jordan.breeding on Nov 16, '05 08:45:10AM

As far as I can tell this is an issue between BSD and SysV versions of cp (and maybe some other utilities). On every BSD box I have used cp works the way it does in Darwin, which makes since. Solaris (SysV), Linux (GNU -- which is usually close to SysV), etc. all copy the directory is you use "dir" or "dir/", but copy the contents only when the argument is "dir/.". The SysV behavior makes more sense to me, but I have learned to just watch what I am doing on BSD systems.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: iordonez on Nov 16, '05 08:51:26AM

Your problem is the trailing / after BOTH Afolder and ~
# cp -R AFolder ~
works fine. This is standard bash behavior.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: hombre on Nov 16, '05 08:12:48PM

This is not correct. cp -R Afolder ~ and cp -R Afolder/ ~ produce the same (differing) results as in the hint.



[ Reply to This | # ]
And drag-n-drop
Authored by: kyngchaos on Nov 16, '05 08:55:33AM

Another thing that adds the trailing / for a folder - drag-n-drop a folder to Terminal. I use this all the time when running configure scripts on 'nix source, and running other programs in terminal that take a folder path as an option. Some things handle the trailing / OK, but others have problems with it.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: pediddle on Nov 16, '05 09:03:22AM
The version of cp included with Fink doesn't have this problem. Personally, I use all of Fink's GNU utilities in preference to BSD (/sw/bin is the first item in my PATH). In addition to not having this silly "feature", there are lots of other features GNU fileutils have that BSD ones don't, especially with find and grep.

[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: owsla on Nov 16, '05 10:21:53AM

Yes, this is what I did in Panther because I like the SysV/GNU tools more than the BSD ones. However, the Fink GNUtools don't have patches (afaik) to handle resource forks, unlike the new Tiger-versions of the BSD tools.

Hence, if you care more about resource forks, etc. than ease-of-use you should stick with the BSD ones in Tiger.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: greed on Nov 16, '05 09:17:56AM

Switch to the Z shell and tell it "setopt autoremoveslash". That way, when you tab-complete a directory, it removes the trailing slash when you hit the next key. (Hit slash then tab to accept the directory and start completing stuff inside it.)

I can't find a similar option in BASH.

You will find a similar trailing-slash convention used in rsync.



[ Reply to This | # ]
omitting trailing slash on completion
Authored by: sjk on Nov 21, '05 03:03:05PM

Adding "set mark-directories off" to ~/.inputrc will omit the trailing slash from directory completion in bash and other apps that use the readline library for completion.

In tcsh, add "unset addsuffix" to ~/.tcshrc or wherever you put commands that it processes shell variables. My ~/.tcshrc file has this line in it:

if (-r /usr/share/tcsh/examples/rc) source /usr/share/tcsh/examples/rc

... so I put "unset addsuffix" in ~/Library/init/tcsh/rc.mine along with other commands that override and extend the rc examples.



[ Reply to This | # ]
Or learn to avoid the problem
Authored by: tim-wood-MacOSXH on Nov 16, '05 10:37:29AM

I've lived in multiple unix worlds for a while and versions of this "problem" come. The basic problem is seen in the discussion here. Some people see the slash and say "ah, stuff inside the folder". Other people see the slash and say "ah, the folder". Surprise, surpise... those who programmed the different shells (tcsh, bash, sh, etc) are people, too.

Like a lot of things unix, you can come up with a simple approach that works across _all_ shells and not have to hack each computer you use.

An example is using cp -R instead of cp -r. Both mean recursive (e.g. go through all the nested folders) but a lot of unix commands only recognize the -R. Computers are so darn literal sometimes.

Back to the problem at hand... I avoid the problem by (begin really deep insight) explicitly telling the computer whether I want to deal with a folder _or_ all the stuff in it.

If I want the directory, I'll leave off the slash:

cp -R stuff ~/

If i want to the stuff in the directory, I'll add a slash and the wildcard:

cp -R stuff/* ~/

The final fly-in-the-ointment is that dot files (i.e. files whose names begins with a dash) are hidden from view by default and sometimes will not copy this way, so you may have to do a second copy by putting /.* on the end of the directory name:

cp -R stuff/.* ~/



[ Reply to This | # ]
Or learn to avoid the problem
Authored by: thadman08 on Nov 17, '05 05:50:57AM

You have to be careful with '~/folder/.*'. That covers the dot-files, yes, but it also covers 'dot' and 'dot dot', meaning the folder itselft and the parent folder.

Depending on the command you are using, this can affect a few more files than you might think.



[ Reply to This | # ]
Or learn to avoid the problem - better solution for dotfiles
Authored by: nickp on Nov 18, '05 08:58:00PM
As thadman08 pointed out, .* matches .. as well as the intended dotfiles.

My incantation to get around this was to use .[A-z]*

Un*x sux, btw. Fifty years of computer science and here we are arguing about how to copy some goddamn files ...

[ Reply to This | # ]

Watch for undocumented behavior of cp -R
Authored by: ojohns on Nov 16, '05 12:03:19PM

I had wondered whether this was standard BSD behavior. I am familiar only with
Linux and OSX. Evidently, BSD generally uses the trailing / as a flag, whereas Linux does not. May I suggest to the BSD community that someone should perhaps rewrite the BSD man page to describe what actually happens? :-)

---
ODJ



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: emagius on Nov 16, '05 06:38:46PM

It's standard UNIX behavior. When the GNU folks copied cp, they goofed. You'll see this with quite a few GNU utilities, actually.



[ Reply to This | # ]
Watch for undocumented behavior of cp -R
Authored by: Rogue.Monk on Nov 16, '05 08:17:29PM
Some say this is the way it should work. Others say they hate it this way.

If you don't like it you can always right a script to strip the trailing slash and then alias "cp -R" to it.

#this might need tweaking--was pieced together quickly
#Automator rocks for rapid shell script prototyping
for f in "$@"
do
	cp -R ${f%/}
done


Someone might want to tweak this further to ensure things like options are being passed down correctly.

The point is, Unix is designed to be tweaked. It gives you the most possible options and then lets you customize for your own needs.

Don't let it dictate how you use it.


[ Reply to This | # ]
Yes!
Authored by: forman on Nov 16, '05 09:43:23PM

What he said. :)



[ Reply to This | # ]
No!
Authored by: nickp on Nov 18, '05 09:12:10PM
After many years of sysadmin'ing, I ended up forsaking almost all such customizations, for the simple reason that I couldn't help out any of my users if I was too habituated to something nonstandard. Nor did I feel like running classes on 'suggested hacks for your initfiles'.

Jef Raskin, in his excellent The Humane Interface, goes into this whole area in more detail, but the short version is "if the system requires customization, then it is poorly designed."

In a similar vein, he argues against giving users (even expert users) multiple ways of getting the same thing done -- the cognitive overhead of figuring out which one to use is surprisingly large.

None of this is to argue that customizability, per se is evil, but that using it to paper over fundamental design flaws is. Some days un*x seems to me to be one huge design flaw ...

Raskin's book is available from Amazon.

[ Reply to This | # ]

Watch for undocumented behavior of cp -R
Authored by: pjungwir on Mar 22, '09 01:31:18PM

This is a nice workaround, but shouldn't the code be this?:

#!/bin/bash

cp -R "${@%/}"



[ Reply to This | # ]
tar works just as well in most cases
Authored by: devlogic on Nov 16, '05 10:38:16PM
I dunno; I never really got to like using 'cp -R'; I learned a trick with multiple piped tar's a while back, and have pretty much used it exclusively since then:

tar cf - [source] | (cd [destination] && tar xvf -)
does a great job for everything I've ever needed it for. Plus it has the added benefit that it can be piped through things like ssh (or rsh if you're brave):
tar cf - [source] | ssh username@other.server tar xvf - -C [destination]
Toss a '-p' in the arguments for each tar, and you even get the permissions translated across. Do the command as root (on both sides, mind, if you're using the ssh cross-server method), and the ownership isn't even modified; UID and GID are preserved. Of course if you are moving files between servers, it's necessary to remember that these commands use numeric UID/GIDs and not usernames; user "johnsmith" might be uid 501 on one system, but 502 or 1006 on another.

[ Reply to This | # ]
tar works just as well in most cases
Authored by: derekhed on Nov 18, '05 11:54:40AM

So... do you put the ending slash on [source] and [destination]? :-)



[ Reply to This | # ]