Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Flatten a directory structure UNIX
There may be times when you want to consolidate all the files in a directory and its sub-directories (or a folder and its sub-folders) into a single directory or folder. For example, you may have a folder with sub-folders for years, and other sub-folders in each year folder for months, and you may want to move files in the month folders all to the top level.

Doing this manually is a complex and time-consuming process. While you might be able to do this by using a search - for example, if all the files are, say, Excel files, you can search for Excel files in the top folder, then just copy them all to a new folder - if there are lots of different types of files, this wouldn't make things easier.

Fortunately, there's a way to do this from a command line. On the BedroomLAN blog, Alexios presents two commands that will do this:

cd $ROOT_DIRECTORY
find -type f -print0 | xargs -0 -I%%% cp %%% $FLAT_DIRECTORY


Replace $ROOT_DIRECTORY with the top level directory containing all the sub-directories and files, and replace $FLAT_DIRECTORY with the directory you wish to contain all the files. Note that this command will overwrite any files with the same name, so if you don't have uniquely named files, it's not ideal.

You can also use the ln command instead of the cp command, and this will not overwrite files, but will give error messages if there are duplicate file names. See the blog post for more details on this. H/t to robg for pointing this out.
    •    
  • Currently 1.57 / 5
  You rated: 1 / 5 (7 votes cast)
 
[9,111 views]  

Flatten a directory structure | 12 comments | Create New Account
Click here to return to the 'Flatten a directory structure ' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
There's also a GUI tool for this
Authored by: tempel on Feb 26, '13 08:18:03AM

My program Find Any File (http://apps.tempel.org/FindAnyFile/) can do that, too, for those who dislike the command line interface:

After starting FAF, drag the folder icon that you want to flatten onto the top popup menu of the window (where it lists your available disks).

Then change the search to:
Name - matches pattern - *
I.e, enter an asterisk in the text field.
Finally, press the Find button.

You'll get a Finder-like window listing all items in the folder, where you can then move, copy and perform other common tasks just like in the Finder.

Edited on Feb 26, '13 08:18:38AM by tempel



[ Reply to This | # ]
Flatten a directory structure
Authored by: mmm-five on Feb 26, '13 09:35:54AM

Can't you just find items that are not folders by using "NOT:folder" in the folder find window, and then copy/move those to the root of the folder you're searching/flattening.



[ Reply to This | # ]
A few notes
Authored by: lullabud on Feb 26, '13 09:48:32AM
To be safe, that "cp" really should be "cp -n" so you don't overwrite destination files with the same name. (eg: moving 1/foo.jpg 2/foo.jpg to the current directory would end up with just 2/foo.jpg as foo.jpg)

Another way to get all the files into the same dir is to change slashes to another character, like an underscore. Here's an example in bash: find . -type f | sed 's/^\.\///' | while read -r X ; do echo mv "${X}" "${X//\//_}" ; done ; # filesystem regex's always look wonky with slash delimiters

Yet another way to do massive renames like this is with one of my favorite perl scripts, prename. (You can find it by googling "perl prename") With prename, you could flatten a two-level directory like this: prename 's^/^_^g' */* # perl and many others now allow you to use something other than a slash regex delimiter, which is useful for filesystem operation legibility

[ Reply to This | # ]
A few notes
Authored by: excarnate on Mar 01, '13 02:15:57PM

You're moving, he is copying, big difference.



[ Reply to This | # ]
Flatten a directory structure
Authored by: adrianm on Feb 26, '13 09:58:19AM

You can do it in Finder too.

Just make a query such as Name matches "" in the directory where you want to start looking and it'll find everything.

So, in a Finder window, type Cmd+F and you end up with the search criteria bar showing Kind is Any.
Change that to Name matches "" (empty double quotes).

That's it. Do what you will with the files.

---
~/.sig: not found



[ Reply to This | # ]
Flatten a directory structure
Authored by: macher on Feb 27, '13 04:46:05AM
I am curious could something like exporting folders with names like year-month-date into a structure like year/month/data/datahere be possible and how.
---
I have my life with me but my heart is at home.


[ Reply to This | # ]
Flatten a directory structure
Authored by: excarnate on Mar 01, '13 02:24:14PM

A basic approach would be to locate a folder (e.g. use ls if everything is at the same level, or find /the/path -type f if you wanted to find all subdirectories), parse out its name into 3 parts, YYYY, MM, DD (handling anything like having/not having a leading 0 to suit your needs). mkdir -p /wherever/$YYYY/$MM/$DD Then move or copy or link the files into that directory.

That assumes I understood the question correctly :)



[ Reply to This | # ]
Flatten a directory structure
Authored by: Rishifter on Feb 27, '13 09:39:07AM

Exactly what I've done many times. I think this has something to do with Finder, by default as far as I can remember, not searching the current directory and instead searching the entire mac. That setting is found in Finder Preferences > Advanced > "When performing a search". I always keep it to "Search the current folder" and when that is set just searching in any Finder window searches that folder which then makes it simple to search for all files.

It is interesting to know there are multiple ways of doing things like the terminal command in the original hint or as I was thinking of AppleScript someone already posted a script to do just that. But I always believe the easiest way is the best way, which in this case is in spotlight in Finder already. The terminal and AppleScript of course have their advantage in automating those things.



[ Reply to This | # ]
Flatten a directory structure
Authored by: Maxikubik on Feb 27, '13 08:29:42AM
This is of course also possible with an AppleScript:

--flattens a folder structure by moving all files from all sub-folders to the selected folder
--files with the same name will cuase an error message
tell application "Finder"
	set theTopFolder to (choose folder)
	set theFiles to a reference to every file of (entire contents of folder theTopFolder)
	set theNewFolder to make new folder at theTopFolder with properties {name:"Flattened Files"}
	move theFiles to theNewFolder
end tell
Just use it as is or modify it to your liking!

[ Reply to This | # ]
Flatten a directory structure
Authored by: arcticmac on Feb 27, '13 09:53:23AM

An interesting side effect of using 'ln' instead of 'cp' (really, you want 'cp -n' if you're going to take that approach) is that ln makes hard links instead of copies. Effectively, you get another pointer to the file's data, rather than a whole new copy of the file. In many cases, this is an advantage because hard linking is faster than copying, and you won't need nearly as much disk space to make a bunch of new hard links. On the other hand, if you wanted to flatten and then make changes to the flattened copy while leaving the originals alone, you'd definitely want to use 'cp -n' instead.



[ Reply to This | # ]
Flatten a directory structure
Authored by: lsequeir on Mar 05, '13 05:45:03AM

Actually, the find command can also execute a command on each file it finds - that's what the -exec option is for.

Also, I think there was a "." missing in the original hint (the first argument to find is usually the directory where the search starts; after doing cd then you would want to start at the current directory).

Here is a shorter version (and, to my mind, clearer). Just replace (or set) $ROOT_DIRECTORY and $FLAT_DIRECTORY and run:


find $ROOT_DIRECTORY -type f -exec cp -n {} $FLAT_DIRECTORY \;


(the two braces {} are find's convention for the current found file; the semicolon ends the exec command and must be escaped to prevent the shell from interpreting it)

---
Luís



[ Reply to This | # ]
Flatten a directory structure
Authored by: leichter on Mar 18, '13 03:51:36AM
While I've sometimes used one-liners such as those being described here, I've all too often regretted it: A minor typo and files end up in the wrong place or with almost but not quite the intended names. And then you're left with coming up with a new command to fix things up. (Even worse, your one-liner fails part way through, and you now have two different problems to solve: The original one for the remaining files, and a new one for the mis-directed files.)

Two approaches make this safer:
1. Don't get fancy with one-liners. Use find to generate a list of input files; then convert it into a list of shell commands to carry out the necessary actions. Finally, when you've looked everything over and are satisfied with it, feed the commands to a shell ($ sh <command-file).

You can build the list of commands using whatever tools are comfortable and appropriate: sed, cut/paste, awk, tr (handy for changing upper to lower case, or '/' to '_'), emacs, vi. Sometimes a mix of tools is best - e.g., sed to do most of the work, emacs to fix up a few special cases.

2. If you do use a one-liner, check it first. Easy approach: Put echo in front of the command that actually renames or moves files or in any other way changes stuff. That way, nothing is actually done but you can see the commands that would have been executed. All too often, you'll find yourself feeling relief for not having "let it rip" without testing.
---
-- Jerry

Edited on Mar 18, '13 03:55:08AM by leichter


[ Reply to This | # ]