Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

A script to prevent damage from rm -rf malware UNIX
After reading about the malware / Trojan Horse business that has been flying around lately, I realized that there is a fairly easy way to protect against this kind of thing. I wrote a script which duplicates a folder (like a user's Home directory) using hard links. The result is a complete backup of your home directory that takes up very very little additional space, because it is actually pointing to the contents of your original files. The practical upshot of running the script is that if you accidentally run (or some unfriendly software runs for you) rm -rf ~, then it will still remove the contents of your home directory ... but you will have a full backup of your files exactly as they were before running the command. I've called the script shadowmirror, and it works like this. You run it periodically to keep all the files in backup directory in sync:
 shadowmirror /Users/yourUserName /Users/backups/yourBackupName 
This will make sure everything in your home directory is backed up. It does not remove any files you might have deleted, though, unless you run it like this:
 shadowmirror -d /Users/yourUserName /Users/backups/yourBackupName 
This will remove any files that are in the backup that you have removed from your home directory. So, if you run the first version nightly, and the second version weekly, or monthly, you will always have a fairly up-to-date backup of your home directory with very little cost in disk space. For example, a 20GB home directory required about 58M of drive space. Using this has the added benefit of giving you a way to recover files that you yourself may have accidentally deleted. Some things to note:
  • Hard links work in such a way that files will not be removed from the disk until all links to them have been removed. This means that you will NOT recover disk space by emptying the trash until you run the -d version of this command. But it's also what makes it possible to recover from that rm -rf on your home directory.
  • Files WILL stay in sync between the backup and your home directory in real time, as long as the app editing them does not delete and recreate the file, which many apps do. Which means that this does not let you recover from mangling the contents of your files in all cases.
  • The first time you run it, it will take a long time, as it is initially creating the backup. Subsequent runs will be faster, as only new files will need to be relinked
  • You should NOT edit things inside your backup directory. First, any new files created there will be removed upon the next time you run with -d. Also, in many cases it will do exactly the same as editing the file in your home directory, so it is of very questionable usefulness.
  • You can not make your backup within your source directory. It must be in a completely separate location.
  • The source and backup directories MUST live on the same disk. Hard links do not function across volumes. This also means that currently this will not work if you are using File Vault.
[robg adds: Due to the length of this script, I have uploaded it to the macosxhints' file collection; click here to view and copy the source. Remember to make the file executable (chmod +x shadowmirror) and store it somewhere on your path. I have not tested this script, but the idea of using hard links for a live backup is a good one, I think...]
    •    
  • Currently 1.50 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (2 votes cast)
 
[10,354 views]  

A script to prevent damage from rm -rf malware | 39 comments | Create New Account
Click here to return to the 'A script to prevent damage from rm -rf malware' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
A script to prevent damage from rm -rf malware
Authored by: merlyn on May 18, '04 11:14:25AM
You don't need that script. You can do it with the built-in rsync command rather easily:

rsync -a --delete --link-dest=$HOME $HOME /backup/path/for/home
This creates a hard-link tree under /backup/path/for/home pointing at $HOME, just as the overly-complicated Perl script also accomplishes, presuming that /backup/path/for/home and $HOME are on the same disk.

[ Reply to This | # ]
Better than mere rsync...
Authored by: greebly on May 18, '04 12:45:43PM

If you're a die-hard UNIX head, there's a fantastic bit of python scriptwork called rdiff-backup. Why do I make the "die-hard UNIX head" proviso? There's a bit of console work and compiling to do if you want to install this. Adventurous non-UNIX people are welcome to try it out (learn something new)! I'd be happy to help, if you're interested. Once's it's installed it's easier than easy to use:

$ rdiff-backup /Users /Path/to/other/directory/Users

Typing this at a command prompt as root (or putting sudo in front) will backup and mirror /Users to the destination folder. Running it again in say 24 hours will bring the mirror up-to-date with any changes, and it will move changed files to an increment directory.

It's like rsync in that it keeps a mirror of whatever directory you tell it to backup, but it also keeps "diffs" of the changes, like an incremental backup. I have an NFS server in my basement, and I have a cron job backing up the /Users folder every night, for each of my Macs. I can also have it remove the incrementals older than 21 days (any time amount is possible).

It has backup and restore capabilities. I used to back up to an old tape drive, but with disk space so cheap, it's easier and cheaper to just use disks. I can restore to an earlier version of a file if something happens.

I haven't tried to mirror a whole disk, so I don't know if such a thing were bootable, but I imagine it wouldn't be hard to make it so.

Long story short, the only sure way to protect against file deletions is to keep backups. I have a .Mac account, but find that Apple's Backup utility doesn't do it for me. I like having incremental backups and a live mirror on a hard disk, networked or otherwise.

Note: (It's necessary for the destination folder to exist before running the script and it's useful to have the same name and permissions as the source, eg. /Users permissions are drwxrwxr-t (1775 in Octal), so you would want /Path/to/other/directory/Users [just the Users folder] to have the same permissions. This would make a full restore as easy as just clicking and dragging or a cp -Rp).

2nd Note: If anyone is interested in helping me, I think it would be rather easy to make an AppleScript wrapper for this, where backups and restores and such would become more accessible to Joe User. Let me know.

---
--++-- Aaron Mildenstein --++--

Do not meddle in the affairs of Dragons,
for you are crunchy, and taste good with ketchup.

[ Reply to This | # ]

A script to prevent damage from rm -rf malware
Authored by: PurpleHaze on May 18, '04 01:08:10PM

Wouldn't you need the -R flag to preserve structure ? or is that implied in -a (archive mode) ?



[ Reply to This | # ]
rsync doesn't work that way.
Authored by: jk806 on May 18, '04 06:30:06PM

Hi,

As the author of that overly complicated perl script, I'd like to point out that the command you provide does _not_ actually work. Believe me, I'm much too sensible (read: lazy) to write complicated perl scripts when a one-line shell command will do it. I tried it with rsync before I wrote the thing.

Incidentally I also used cpio, which will create the hard link tree as well, but the perl script runs faster after the first run, as CPIO completely rewrites all the links every time, whereas the perl script only makes updates to things that have changed.

Go ahead and try the rsync on a small directory, and check the inodes and your disk space. You'll notice that what you actually get is a _real_ backup of your files, not a hard-link tree. Still, I'm not immune to mistakes... so just to make sure I didn't miss something the first time round... I used your command on just the Pictures folder within my home directory...

I did my ~/Pictures folder to ~/t1/

Results: (ls -i1 on files in both locations)

Computer [205]~>ls -i1 t1/Pictures/Finder*
1771681 t1/Pictures/Finder4.jpg
1771682 t1/Pictures/Finder5.jpg

Computer [206]~>ls -i1 Pictures/Finder*
1439742 Pictures/Finder4.jpg
1439743 Pictures/Finder5.jpg

Note the inodes are different. My disk space decreased, coincidentally, by the size of my pictures folder.

If you can find a way to make rsync actually do it, please post it.

jk806



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: Fil Machi on May 18, '04 10:06:45PM

The example rsync command line by merlin contains an error. Try this:

rsync -a --delete --link-dest=$HOME $HOME/ /backup/path/for/home

Note the trailing slash '/' on $HOME/. Without it rsync creates /backup/path/for/home/X/*, where X is the name of your home directory, and rsync then can not match up paths correctly between the backup directory and the link-dest directory.

With the trailing slash rsync creates /backup/path/for/home/* and --link-dest works as expected.

Also, the stock rsync that comes with OS X does not copy the resource fork of HFS+ files. So backups of HFS+ directories created using it can not really be called backups. Install and use the resource fork aware version of rsync, aka rsyncx from versiontracker (http://www.versiontracker.com/dyn/moreinfo/macosx/16814).



[ Reply to This | # ]
rsync still doesn't work
Authored by: jk806 on May 18, '04 11:45:45PM

I did as you mentioned, adding the /, same result. New inodes. It makes a backup just fine, but not using links.

As I mentioned, I tried using rsync several times using link-dest, and could not get it to create the tree using links no matter which way I ran it.

I like rsync and use it regularly, but for this purpose, it doesn't seem to work.

In any case, shadowmirror does what's needed, on a stock OS X system (or any other unix system) without additional software. It does just as well at syncing as rsync for these purposes, and does it faster than rsync. Over 7x faster consistently in my tests, most likely due to the fact that rsync does more tests than simple inode comparisons and, of course, creating links is a lot faster than copying data.

For me, I'll use shadowmirror since I know it works, and don't want to spend any more time trying to figure out why rsync won't.



[ Reply to This | # ]
sweet.
Authored by: DanPritchard on May 19, '04 02:04:02PM

I think that is a very cool script. Nice job. Very elegant--exploiting the fact that deleting a file doesn't really delete the file. I love it.



[ Reply to This | # ]
Woulr rm -i help too?
Authored by: dombi on May 18, '04 11:37:21AM

If someone set in their .profile an alias for rm

alias rm='rm -i'

woudl that work against the "Trojan"? Basically this way every time you use the rm command, the Terminal would ask you if you are sure that you would like to delete the files.



[ Reply to This | # ]
Woulr rm -i help too?
Authored by: tang on May 18, '04 11:52:30AM

It depends on how the trojan was written. For instance, the writer could just refer to the full path of the rm command. /bin/rm (or wherever it is by default) and the alias would not work. Also, what is to stop the trojan from just aliasing rm back to rm before they execute? or if the trojan even loads your .bash_profile or not...



[ Reply to This | # ]
Woulr rm -i help too?
Authored by: lagroue on May 18, '04 11:55:15AM

Hmmm, no.
The -f option has precedence over the -i :

$ touch toto
$ rm toto
remove toto? n
$ rm -f toto
$



[ Reply to This | # ]
Woulr rm -i help too?
Authored by: semios on May 19, '04 04:58:56PM

No. Aliases do not apply to non-interactive scripts in bash.



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: rhowell on May 18, '04 12:04:12PM
I wonder if its possible for Apple to rewrite the rm command, and any other Applescript commands or even the kernel's way of unlinking files, so that any invocation of such a command will move the files to your Trash can (or to the root Trash can, etc.). And then, emptying the Trash can will always require the User's password, or an Administrator's if its at the root level. Furthermore, the password prompt will always be prefaced with a statement that the User is about to empty the Trash can (making it difficult to spoof anyone into giving up passwords for other reasons). Granted, this makes emptying the Trash a little more cumbersome, but essentially prevents any third-party app from permanently deleting files.

Am I being too naive? Is this too difficult to implement? Would there be work-arounds?

[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: sapporo on May 18, '04 12:57:18PM

Remember that a malware program does not have to use the /bin/rm binary at all, and could easily achive the same without using external programs in any programming language.
And even if you make it impossible to unlink files, a malware program could just overwrite you files with gibberish.



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: robmorton on May 18, '04 03:31:47PM

I really hope this never happens. If a user has the ability to delete a file, they also need to take responsibility for what is run on their file space. AppleScripts and shell scripts are far too handy to start limiting them because of the potential issues. These issues have existed since AppleScript was created, so I do not see the issue. Sure there was no rm-rf command in OS 9, but you could get around that and did not have to worry about as many pesky permissions getting in the way of really messing up a system.



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: ptwithy on May 18, '04 04:07:52PM

I would think this would be very possible. I used to do it as a standard procedure when I had to manage Unix installations for non-wizards. It is simply a matter of shadowing the kernel unlink call in the C runtime with a subroutine that moves rather than unlinks then re-link all the commands that do unlink operations (e.g., not just rm, but mv, which will unlink a duplicate target.) This saved me from having to search backups many times...



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: MaxMarino on May 18, '04 12:14:52PM

Nope, f and i have *same* precedence, the last one wins

rm -rif == rm -rf

rm -rfi == rm -ri



[ Reply to This | # ]
even simpler
Authored by: SOX on May 18, '04 12:29:51PM

find ~ | cpio -dpl /shadow_dir/

this will do the same thing as the script.



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: Han Solo on May 18, '04 01:51:40PM
OK, so shadowmirror will not work across Volumes. I am looking for a relatively transparent way to mirror my User directory to another Volume. (A networked Windows file server via SMB or AFP, if it matters.) I am already duplicating the drive locally every night to another drive in my MDD G4 with CCC, but that might not help much if my office gets flooded again. (Long story....)

Anyway, which of the above — rsync, rdiff-backup, and/or cpio — would satisfy those criteria? Any info would be helpful before I invest the time into figuring one of them out with my meager Unix knowledge.... TIA.

[ Reply to This | # ]

RE: Network backup question
Authored by: greebly on May 18, '04 02:21:48PM

I think that all 3 meet your criteria, but rsync and rdiff-backup are going to be easier to use and understand.

The bottom line is that each of those needs access to the Volume at the console level. If that can be accomplished, then you're fine.

However, you're going to need the target volume to support full UNIX permissions and ownership compatibility. SMB unfortunately doesn't do this properly, mostly because SMB/CIFS doesn't account for it in the protocol properly (UNIX permissions, not Windows ones). AFS should probably foot the bill. NFS certainly does.

---
--++-- Aaron Mildenstein --++--

Do not meddle in the affairs of Dragons,
for you are crunchy, and taste good with ketchup.



[ Reply to This | # ]
remote backup
Authored by: SOX on May 18, '04 02:22:15PM

rsync, rdiff, and cpio all have a problem with HFS+ and that is resource forks and metadata are destroyed. For 99% of your osx files this makes no difference but you will end up regretting it eventually. macosxlabs' RsyncX instead of the built in rsync takes care of HFS+, but is like rysnc plagued with silent malloc failures on deep directory trees that make it unsuited for unattended usage. If you actually check your error logs to notice those sorts of failures then RsyncX is probably the single best solution.

RsyncX is pretty close to your only choice if you cannot mount the remote volume on your local computer.

if you are able to mount the remote volume (as opposed to ssh-ing to the remote computer) and it's an hfs volume you could use Ditto or cpMac with the copy resource flag enabled.

If you are mirroring things that mostly dont change then the above two methods are inefficient. Instead use rysncX (with the caveat I noted.) Personally I use psync which is also more efficient and which does not have the deep path failure mode. It is slightly slower than rsyncX and wont work on things you cannot mounton the local machine.

Also for things you can mount you can use silverkeeper (free) from Lacie and (free) CarbonCopy Cloner (which uses a variety of methods under the hood including psync). Both of these are fast and work well. It's very easy to automate silverkeeper and maintain exclude lists using its GUI interface. I've had severe problems with silverkeeper but most people I know report complete success. But it does not fail silently.

psync has a nice feature of being able to stash your metadata and protection factors when you are stroring on a remote volume that is not HFS aware. They can be restored later.

with psync its:
mount remote disk
pysync the local to the remote directory


then as an added bonus on the remote machine I normally execute
find /mybackup | cpio -dpl /backup.1
before I run psync. this gives me a snapshot of the previous generation backup that takes almost no added disk space. you can keep as many of these as you want for weekly, monthly backups....the only storage space it uses is to store the differences.




[ Reply to This | # ]
remote backup
Authored by: SOX on May 18, '04 02:24:42PM

note that while cpio has problems with meta data, oddly enough this is not a problem for the prescribed usage of making hard links. this only affects the primary copy. that is dont use CPIO to make a backup copy. use it to make a hardlink image. make the bakcup copy with something else.



[ Reply to This | # ]
remote backup
Authored by: greebly on May 18, '04 02:35:28PM

Hmmm. Perhaps I will have the backup performed to my other Mac with an HFS+ volume, and make .dmg's of that volume from time to time.

I don't think it's going to be a HUGE problem though. Most of the important data is backed up, and those are documents, etc. which, as you say, are part of that 99% of files that won't be affected.

---
--++-- Aaron Mildenstein --++--

Do not meddle in the affairs of Dragons,
for you are crunchy, and taste good with ketchup.



[ Reply to This | # ]
remote backup
Authored by: SOX on May 18, '04 06:59:36PM

still will lose the creator code if the suffix is missing.



[ Reply to This | # ]
remote backup
Authored by: greebly on May 18, '04 02:57:56PM
According to this post, the metadata problem has been solved in recent versions of rdiff-backup.

That's been my experience so far as well. I haven't had a problem with my backups, having checked the logs.

---
--++-- Aaron Mildenstein --++--

Do not meddle in the affairs of Dragons,
for you are crunchy, and taste good with ketchup.

[ Reply to This | # ]

RE: Network backup question
Authored by: greebly on May 18, '04 02:27:15PM

2nd note: Let me reiterate again, trying to store UNIX files on a windows file server is a disaster waiting to happen if you are trying to make a completely recoverable file. The best way to do this would be to make a tar backup (or a dmg image or something like it) and save that file on the windows server. Permissions and such will be lost in a file copy style backup (like rsync, rdiff, etc.) and if you're only wanting to backup document files and such, that's not going to be a problem. A full restore, however, would be messed up beyond belief.

---
--++-- Aaron Mildenstein --++--

Do not meddle in the affairs of Dragons,
for you are crunchy, and taste good with ketchup.



[ Reply to This | # ]
RE: Network backup question
Authored by: SOX on May 18, '04 07:02:02PM

the native Tar and zip are not hfs aware either. however, you can use the new apple zip-archive feature and transport that. (dont expand it on the remote file system or you may lose the meta data)



[ Reply to This | # ]
Complication: Removing Files
Authored by: googoo on May 18, '04 02:14:27PM

One complication with this method is when you WANT to remove a file immediately. Suppose you have a large file (i.e., digital video) that you would like to remove. You would have to remove the "original" as well as the hard-linked "backup." If you do not remove the backup, the file will remain on your hard drive until the backup script removes it.

-Mark



[ Reply to This | # ]
Complication: Removing Files - what about secure delete
Authored by: hamarkus on May 18, '04 07:05:57PM

Would secure delete remove the files immediately, or would it just overwrite the files with gibberish, with the disk space only being reclaimed after the -d version of the script was run?



[ Reply to This | # ]
Complication: Removing Files - what about secure delete
Authored by: _merlin on May 18, '04 09:06:04PM

Depends on whether Apple's implementation sets the file length to zero before deleting the file. I'm not at my Mac now, but on this Sun, the space is only reclaimed when you remove all the links (yes, the file is overwritten with garbage from /dev/random first).



[ Reply to This | # ]
Good for rm, but not others
Authored by: neier on May 18, '04 07:04:44PM
This is only a slight remedy, as there are still many nasty things that anyone familiar with unix could do to your data if you only backed it up via hard links.

echo " " > MyImportantFile would kill the original AND any hard links. Creative use of cp would also write over the original contents.

[ Reply to This | # ]

Good for rm, but not others
Authored by: gshenaut on May 18, '04 07:59:19PM

You could use "rm -rfP ~" and it would delete or overwrite all your files, including those you have "backed up" with hard links.

Also, as others have said, this isn't a unix problem. You could use applescript to delete selected folders and then to empty the trash, you'd
get the same effect.

One approach to a fix would be to do force web browsing into a carefully constructed, chrooted (or jailed) sandbox. That is, the browser and programs it spawns would have access only to a safe, backed up piece of your file system.

Greg Shenaut



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: digitol on May 18, '04 07:30:35PM

Couldn't one just find the rm tool and set it as sudo users only tool.



[ Reply to This | # ]
PEBKAC: Why hard links/aliases aren't the solution
Authored by: robophilosopher on May 18, '04 10:58:43PM
There are two ways we can approach this kind of "malware." The first is to do things like this, or like aliasing 'rm,' which is to say, temporary fixes to prevent specific abuses. These 'fixes' will fail as soon as a Trojan runs
find ~/ -type f --exec cat {} \; 
(disclaimer: I'm not 100% sure on my syntax there, particularly how that redirection will work in the --exec clause. plus, a dd would be more effective, right?) or similar instead of rm. You have to think anybody with half a brain writing this sort of bad stuff will be using actual interaction from the program, rather than system calls or shell commands. The more important issue is resisting viruses and trojans' entrance points. This is what mac is fairly good at, what UNIX is great at, and what gives OS X the track record it has against viruses. (Yes, Windows is a bigger target, and that's a contributing factor, but I stand by my statement.) Fixing the 'problem' that is involved in much of the (potential) malware I've seen is to maybe introduce a GUI-level warning when executing a program that has an "open with" extention, or something like that, to prevent un-vigilant people from doing un-wise things...

You create hard links, I can write to the inodes. You don't run my program, I can't do squat.

PS: check acronymfinder for the subject line if you're interested.

[ Reply to This | # ]
PEBKAC: Why hard links/aliases aren't the solution
Authored by: SOX on May 19, '04 10:24:37AM

How does one write to inodes. Usually one writes to files that get unlinked before being written, preserving the hardlink.



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: roballen on May 19, '04 04:08:56AM

The company I work for develop software on Redhat, SCO, AIX etc. To prevent accidental deletion of certain key directories we create a file called "-i" or some other such switch. It basically prevents rm -f -f * type commands, as it interpreted as an option to the command.

To remove the actual file, create a blank file "touch testfile", and then "rm testfile -i".

This will not prevent a file or directory from being explicitly removed, however it will stop wildcard deletions, and accidental typos.

Works well for our office.

Rob



[ Reply to This | # ]
A script to prevent damage from rm -rf malware
Authored by: abriening on May 19, '04 09:43:59AM
What about rm -rfP ~ ? Won't the -P flag shred(overwrite 3x) the data making your backup useless? I've read that the -P flag doesn't work on HFS+ so this may not be a concern for most.

The Trojans will evolve.

[ Reply to This | # ]
Sheep in wolf's clothing
Authored by: bq on May 19, '04 11:07:21AM
The trojan horse that's been getting all the attention was essentially cooked up by Intego to sell their security software. An article in Wired (http://www.wired.com/news/mac/0,2125,63000,00.html) essentially debunks it.

What you shouldn't worry about
As of this writing, no working model for self-spreading malware (i.e. virus or worm) for OS X has been identified. In fact, the trojan horse (i.e. malicious code masquerading as something else, in this case an MP3 file) exists only in theory -- no actual version of this has been seen in the wild. And, even when it does appear in the wild, it can't spread itself.

What you should worry about
First, the concept that Intego pointed out is easily created. Someone could (and probably will) create a bit of malicious code and post it somewhere for the unsuspecting to download. Of course, only a couple of people will be affected before the thing is taken down and the author immediately found, but still, it's possible. Exercise standard caution when downloading stuff and you should be fine.

Second, eventually someone will create self-spreading software for OS X. Keep your eyes and ears peeled.

Third, backups are always a good idea. No backups is always a bad idea.

[ Reply to This | # ]

Safety Net
Authored by: anjoschu on May 19, '04 12:19:26PM
I fully agree with you.

Another thing to keep in mind is that malware can only do the things the user who launched it can do (unless it exploits some security hole in Mac OS X). So, as most of us know, it is not a good idea to work with an Admin account (admittedly makes less of a difference if you're the only user). If you are particularly paranoid about downloading malware, you should be rather safe when doing the following:

- Copy the suspicious file to the /Users/Shared folder (or make something like /Users/Shared/Downloads your Download Folder)
- Create some test user (re-usable), call it canary or whatever. :)
- Fast user switch to canary and open/examine the file you downloaded from there.

[ Reply to This | # ]

A script to prevent damage from rm -rf malware
Authored by: gidrac on May 21, '04 11:23:56AM

Buy a Lacie, keep good backups, no worries :-)



[ Reply to This | # ]