Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Store logs in safe location UNIX
I have gotten so many good tips from this site, I am grateful to post something back for the community. After a recent post regarding a compromised OS X box, it became painfully clear that having some forensic evidence of how your computer was compromised is very important. If your rig is broken into and someone gets root access, a quick rm -rf / could ruin your day, as well as erase all traces of how the bad guys broke in. You might be able to reinstall everything from good backups, but what about the original vulnerability?

One common technique to keep logs safe is to use a centralized logging system where each computer sends its logs to a separate, hardened computer for safer storage. I have no second computer, so I decided to burn my logs periodically to CD (via incremental burns, to get the most out of each CD), and use the read-only aspect of the medium to keep them safe.

Here is how I did it:
  1. Create the script below and make it executable.
  2. Put a blank CD in your drive and have your Finder 'ignore' it.
  3. Add an entry to your root crontab to fire off the script.
  4. Check your emails to make sure your disk isn't going to be full the next time you run it.
#!/bin/sh
#
# Back up logs to a CD incrementally. Saves as 'log n' in reverse order,
# renaming each as a higher increment and laying down a new file 'log'
# How the hell does it do this?
#
# May as well use a timestamp for our temp file.
stamp=`date +%y%m%d-%H%M%S`

# Create the image
hdiutil create -quiet -srcfolder /var/log/ /tmp/Log_$stamp.dmg

# Wait for it...?
sleep 5

# Now burn the image
hdiutil burn -quiet /tmp/Log_$stamp.dmg -noeject

# Clean up after yourself
rm /tmp/Log_$stamp.dmg

# Let the drive mount. 15 works for me, but yours may be different.
sleep 25

# Report back to cron how much space is used.
free=`du -ch /Volumes/log* | grep -i Total`
echo "$free used on log Backup CD"
Fire this puppy off from root's crontab nightly (or however often you feel you need to). You will receive an email telling you how much space you have used, so you can make sure you put a new blank in when the first one gets too full.

Caveats: I am no shell expert, and I'm sure this could be tidied up. I also have not tested what happens when your CD is full, but since we are working with copies of the logs, you won't suffer any catastrophic loss.
    •    
  • Currently 2.60 / 5
  You rated: 2 / 5 (5 votes cast)
 
[7,352 views]  

Store logs in safe location | 4 comments | Create New Account
Click here to return to the 'Store logs in safe location' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Store logs in safe location
Authored by: DaMacGuy on Oct 06, '04 02:22:15PM

This is a cool tip for several other purposes too (say, making automated CD backups of a project, etc).

Here are two other suggestions for preserving log files though...

1) have cron mail them to a Gmail account as attachments
2) mount and copy the files over to your iDisk (assuming you have a .Mac account).

I'm not a shell scripting pro (I'm weak at even hacking other folks scripts), but I'm pretty sure both of these techniques can be accomplished.

-DaMacGuy

---
-DaMacGuy



[ Reply to This | # ]
Replacement script
Authored by: derekhed on Oct 06, '04 03:28:05PM
I accidentally left my cron entry to run this script after I ejected my black CD. I forgot all about it until I noticed a dozen or so CRON, sh, and hdiutil entries in my ps list.
I rewrote the script, wrapping everything inside a check for the existence of writable CD media. The next version could include a test for free space on the CD after creating the disk image. So here it is, version 1.01:

#!/bin/sh
#
# Back up logs to a CD incrementally. Saves as 'log n' in reverse order,
# renaming each as a higher increment and laying down a new file 'log'
# How the hell does it do this?
#
# Wrap the whole process in a test for CD Media

if drutil status | grep -i "No Media Inserted" > /dev/null 2>&1
   then
      echo "No Media found. Failing."
      exit
   else
      echo "Media found"
      # No luck with appendable, but you could try adding it.
      if drutil status | egrep "overwritable|blank" > /dev/null 2>&1
      then
         echo "Medium is writable"
         # May as well use a timestamp for our temp file.
         stamp=`date +%y%m%d-%H%M%S`

         # Create the image
         hdiutil create -quiet -srcfolder /var/log/ /tmp/Log_$stamp.dmg

         # Wait for it...?
         sleep 5
         
         # Now burn the image
         hdiutil burn -quiet /tmp/Log_$stamp.dmg -noeject
         
         # Clean up after yourself
         rm /tmp/Log_$stamp.dmg
         
         # Let the drive mount. 15 works for me, but yours may be different.
         sleep 25
         
         # Report back to cron how much space is used.
         free=`du -ch /Volumes/log* | grep -i Total`
         echo "$free used on log Backup CD"
      else
         echo "Can't write to CD Medium. Failing."
         exit
      fi # End check for writable Media
fi # End check for inserted Media


[ Reply to This | # ]

Store logs in safe location
Authored by: Nik_Doof on Oct 07, '04 07:51:10AM
Or you can setup syslog to send to a remote syslog server, nice en secure :)

Quick entry into /etc/syslog.conf:

*.* @127.0.0.1

(replace 127.0.0.1 with ip of syslog server)

here is a quick guide to setting up a remote syslog server on Linux.

[ Reply to This | # ]
You may not want to store logs for too long
Authored by: nickp on Oct 07, '04 02:18:32PM

This is a cool idea, and thanks to Derekhed for working it out and posting it!

I want to add one caveat, however (which may only be appropriate in a large multi-user environment). Back when I used to do sysadmin work, as a matter of policy we *deliberately* made sure we didn't keep logs for more than two weeks ... and that the logs weren't accidentally being backed up somewhere.

If a machine ever becomes the focus of a legal investigation (perhaps only because it was broken into and then used to cause problems elsewhere), you will be amazed at how broad and inclusive the power of a subpoena is.

The first time you waste four entire days spinning old backup tapes in order to comply with one, you will understand. If you have a record, it can be subpoena'ed -- and they invariably ask for everything, and you must comply -- it's much better to be able to say, in all honesty, "we don't have that record." Subpoenas are frequently just fishing expeditions, and they don't really need or use all the data they request. That doesn't excuse the sysadmin from the legal requirement to fulfill the request, however.

Perhaps simply snapping the logfile CD's in half after a month or two is sufficient. Another thing to keep in mind is that OS-X logfiles are extremely chatty, with all sorts of potentially personal information (like what videos you've been watching with VLC, for instance ...)



[ Reply to This | # ]