Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Use tar to quickly create segmented files UNIX
With today's increased Flash storage sizes, there may not be many times when you need to split files. However, when you do need to do so, I find that tar is a good tool (this hint is also good if you want to split an archive before burning to DVDs).

The following use of bash's brace expansion makes creating multi-volume tar files easy. To create a multi-file archive of a given length, use this:

tar --tape-length=102400 -cMv --file=tar_archive.{tar,tar-{2..100}} [files to tar]

To extract from the archive:

tar -xMv --file=tar_archive.{tar,tar-{2..100}} [files to extract]

To create DVD-sized volumes, use --tape-length=4588544. In the above examples, replace [files to tar] and [files to extract] with the location(s) of the file(s) you wish to segment.

[robg adds: I haven't tested this one.]
    •    
  • Currently 2.55 / 5
  You rated: 1 / 5 (11 votes cast)
 
[14,185 views]  

Use tar to quickly create segmented files | 8 comments | Create New Account
Click here to return to the 'Use tar to quickly create segmented files' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Use tar to quickly create segmented files
Authored by: mzs on Mar 23, '09 08:34:22AM

man split

That is the portable way to do what you want.



[ Reply to This | # ]
Use tar to quickly create segmented files
Authored by: TonyT on Mar 23, '09 11:14:17AM

Not if archiving a directory.



[ Reply to This | # ]
Use tar to quickly create segmented files
Authored by: mzs on Mar 24, '09 07:54:24AM

That is why you pipe cpio or tar into split, optionally first piping it into gzip or bzip2.

tar cf - foo | bzip2 -c --best - | split -b 1440k - foo.tar.bz2



[ Reply to This | # ]
Use tar to quickly create segmented files
Authored by: TonyT on Mar 24, '09 10:04:46AM

I really don't understand how piping 3 programs with:
tar cf - foo | bzip2 -c --best - | split -b 1440k - foo.tar.bz2
is preferable to:
tar -L 1440 -cMj --file=foo.tar-{1..99} foo



[ Reply to This | # ]
Use tar to quickly create segmented files
Authored by: mzs on Mar 24, '09 10:40:29AM

Like I said it's not portable. -j and -L are not in all tar programs. In fact -L is most likely something about symlinks in bsdtar considering how the other commands there work. It's best to train your fingers and write your scripts so that you do not have to download and build lots of GNU versions of stuff before you can use a new computer or one someone else admins.



[ Reply to This | # ]
Use tar to quickly create segmented files
Authored by: amusingfool on Mar 23, '09 09:03:49AM

Does this support (and use intelligently) the z or j flags?



[ Reply to This | # ]
Use tar to quickly create segmented files
Authored by: TonyT on Mar 23, '09 11:20:38AM

yes. compression (-j or -z) works with -M (multi-volume) (its just another option)



[ Reply to This | # ]
Use tar to quickly create segmented files
Authored by: kamath on Mar 21, '13 10:41:59PM

Just because someone might be searching:

In 'Create split zip files from the command line' it was suggested that the options used in this hint have gone walkabout in 10.8.

GNU tar has been renamed 'gnutar', and 'bsdtar' is now the target of the /usr/bin/tar link.

Just change 'tar' to 'gnutar' and you should be good to go.



[ Reply to This | # ]