Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Manage the periodic logs System
There has been some discussion here about the periodic maintenance tasks and making them run:

10.4: Reschedule periodic maintenance tasks
A simple shell script to run system maintenance tasks

What hasn't so far been addressed is the logs themselves. I got very interested in these as I suddenly (under 10.4) couldn't account for 5GB on my hard disk. It turns out that the primary culprit was the daily.out log file that had simply grown out of all proportions. The periodic logs are never cleaned up, and thus just grow in size each and every time the periodic maintenance tasks are run. For example:
[21:02:42@/]$ ls -alOFG /private/var/log/*.out
-rw-r--r--  1 root  wheel  - 509376 17 Jul 03:15 /private/var/log/daily.out
-rw-r--r--  1 root  wheel  -    676  1 Jul 07:52
/private/var/log/monthly.out
-rw-r--r--  1 root  wheel  -  10023 12 Jul 03:25
/private/var/log/weekly.out
I did a clean install of 10.5 Leopard in November 2007, so in a little over half a year, the daily.out log has grown in size to 0.5GB. So in the time before 10.6 Snow Leopard comes out, it'll grow to about 1.5GB in size. I do know that hard drive space is cheap these days, but still...

I've found two ways of handling this (there are of course more solutions -- find the one that suits your needs).

Solution #1: Don't log -- let's be honest, who actually reads these logs? Run the following as root (sudo). This will redirect the output from the periodic tasks into the computational equivalent of a black hole:
sed -E -n -e '/_output/p' /private/etc/defaults/periodic.conf | sed -E -e 's!".*"!"/dev/null"!' /private/etc/periodic.conf
rm /private/var/log/*.out
Solution #2: Only keep the last log, just in case you need it. Save the following script somewhere (remember to chmod +x the script) and then link to it in three places. Here's the script:
[07:42:52@Scripts]$ cat deleteOut.bash
#!/bin/bash

#
# delete the old log file
#

where=$(pwd)
echo rm /private/var/log/${where#*/}.out
The script finds out from where it's been run from, and on the basis of this, deletes the appropriate log file. Thus, the script is generic and I don't have to maintain more than one version of it. Here are the links to create:
sudo ln -s ~bjarne/Scripts/deleteOut.bash /private/etc/periodic/daily/000.deleteOut
sudo ln -s ~bjarne/Scripts/deleteOut.bash /private/etc/periodic/weekly/000.deleteOut
sudo ln -s ~bjarne/Scripts/deleteOut.bash /private/etc/periodic/monthly/000.deleteOut
[robg adds: I haven't tested this one.]
    •    
  • Currently 1.78 / 5
  You rated: 5 / 5 (9 votes cast)
 
[14,974 views]  

Manage the periodic logs | 10 comments | Create New Account
Click here to return to the 'Manage the periodic logs' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Manage the periodic logs
Authored by: wallybear on Jul 25, '08 07:56:15AM
This happens because your Mac doesn't perform periodic tasks. It's a long story and you can find it explained in macosxhints previous posts and all around the web.
Leopard and Tiger _should_ perform periodic tasks skipped (they should be run in the middle of the night, when (often) our Macs are sleeping or shut down, but very often they don't (see http://www.thexlab.com/faqs/maintscripts.html).

You can use the free PseudoAnacron (http://www.jaw.it/pages/en/x_misc.html) to automatically resolve the problem of your periodic skipped tasks.

[ Reply to This | # ]
Manage the periodic logs
Authored by: BlackPenguin on Jul 25, '08 08:28:55AM

[21:02:42@/]$ ls -alOFG /private/var/log/*.out
-rw-r--r--  1 root  wheel  - 509376 17 Jul 03:15 /private/var/log/daily.out
-rw-r--r--  1 root  wheel  -    676  1 Jul 07:52
/private/var/log/monthly.out
-rw-r--r--  1 root  wheel  -  10023 12 Jul 03:25
/private/var/log/weekly.out
I'm not sure if this is what you had intended... but your daily.log isn't 0.5 GB. The daily.log is weighing in at about 0.5 MB (497 KB, more precisely). With the ls command you used, the sizes are reported in bytes. Try the -h flag.

[ Reply to This | # ]
Manage the periodic logs
Authored by: Black on Jul 25, '08 08:40:03AM

Actually, in this case, I'd say the problem was exactly the opposite. The log this hint is most concerned with is daily.out which is appended to by the daily periodic task. If the periodic tasks weren't running, this log wouldn't be growing. I wonder how many people are still having problems with periodic not running in 10.5. Launchd has been doing its job and rescheduling these for me properly for quite some time, and my computer is almost always asleep when the jobs are scheduled to run.

That said, the author of this hint might want to actually take a peek inside his logs if it has grown to .5GB in that amount of time. In about half that time, mine has grown to a whopping 130K. If it is spitting out that much verbiage, it might be time to actually see what it is saying...



[ Reply to This | # ]
Manage the periodic logs
Authored by: Black on Jul 25, '08 08:44:49AM

Opps, this was meant as a reply to the first comment... Anyway, BlackPenguin is quite right, I didn't look at the output of the ls very carefully - I just took the BjarneDM's word for it. ~500k sounds just about right and certainly nothing to worry about.



[ Reply to This | # ]
Manage the periodic logs
Authored by: arcticmac on Jul 25, '08 11:29:41AM

Really, you should not be disabling logging. Sure you don't want them now, but really, that's true of most log files until something screws up. And if it's something really bad, you'll kinda want to know.

And if you had 0.5GB log files, it would be time to read through them all to figure out why something was that badly broken. Fortunately yours are, as pointed out by others, more like 0.5MB.

As far as keeping those logs under control goes, newsyslog (which is run hourly by launchd) should, at least in theory, be responsible for rotating the periodic logs. In practice however it does seem that they are actually rotated, and there's no indication in the config files that it's actually configured to do so. I have three systems here, and all of them have entries in daily.out from the first day they were installed (though granted, it wasn't all that long ago). I think the easiest fix might just be to add entries to /etc/newsyslog.conf for them, and while you're at it you might look into the issue with system.log not being rolled over.



[ Reply to This | # ]
Manage the periodic logs
Authored by: rflorence on Jul 25, '08 11:53:26AM
It is not a good idea to redirect the output of the periodic commands to /dev/null; valuable debugging and system information is lost forever. It is also not a good idea to turn an orderly Unix filesystem into spaghetti with symbolic links from system files to user directories. If the size of the periodic files is bothersome, they can be automatically rotated with a simple script installed as /etc/daily.local:
#!/bin/sh
# script to rotate logs
# to maintain historical logs, uncomment the three lines

rotatelog ()
{
    log=$1
    size=`ls -l $log | awk '{print $5}'`
    if [ $size -gt 50000 ] 
    then
#	[ -f $log.2.bz2 ] && mv $log.2.bz2 $log.3.bz2
#	[ -f $log.1.bz2 ] && mv $log.1.bz2 $log.2.bz2
#	[ -f $log.0.bz2 ] && mv $log.0.bz2 $log.1.bz2
	bzip2  $log.0.bz2 
	chmod 0644 $log.0.bz2
	> $log
    fi
}
cd /var/log
rotatelog daily.out
rotatelog weekly.out
rotatelog monthly.out
The script rotates the logs when they are >50K. If you uncomment the commented lines in the code, the script will save the four most recent versions of each log; otherwise it will save only the most recent log. The script is run automatically by the periodic daily command; probably best to install it with execution permission (chmod +x /etc/daily.local).

[ Reply to This | # ]
Easy use of periodic
Authored by: fab23 on Jul 26, '08 08:23:05AM
The whole periodic stuff probably comes from FreeBSD, as it is very familiar to me and does have man pages:
fabian@flashback:~$ apropos periodic
periodic(8)              - run periodic system functions
periodic.conf(5)         - periodic job configuration information
fabian@flashback:~$
To learn more about periodic use man 8 periodic for the program itself, or man 5 periodic.conf for the config file.

I guess two options would be helpful, for example if the computer is not running at nights, you can run periodic manually with the following commands at the appropriated time:

fabian@flashback:~$ sudo periodic daily
fabian@flashback:~$ sudo periodic weekly
fabian@flashback:~$ sudo periodic monthly
And if you do not want to create log files, you can change options in the /etc/periodic.conf file (does not exists and needs to be created). For example to send an e-mail instead, put the following lines into /etc/periodic.conf (change the e-mail address to your own):
daily_output="user@example.com"
weekly_output="user@example.com"
monthly_output="user@example.com"
daily_status_security_output="user@example.com"
Or you can send the log output to nirvana:
daily_output="/dev/null"
weekly_output="/dev/null"
monthly_output="/dev/null"
daily_status_security_output="/dev/null"
The defaults for all options are in /etc/defaults/periodic.conf (do not make any changes in this file). If you want to change something, just add the corresponding option to /etc/periodic.conf.

[ Reply to This | # ]
Manage the periodic logs
Authored by: Fairly on Jul 26, '08 03:29:38PM

"Who reads these logs?" Yes exactly. If you're an admin then you'd better read them. Even ordinary users should consult them. But what's always made me laugh is how people get hysterical about their logs working properly but have never but never looked into them and wouldn't know what to look for if they did.

This is a great article.



[ Reply to This | # ]
Manage the periodic logs
Authored by: rachaeljellicoe on Jul 26, '08 03:59:16PM

Well, I attempted to do the solution #1 but when it came to removing the files it said I wasn't authorized. I am not sure how to use the sudo command. Wikipedia was no help. I tried the su command but don't know the password. I tried my user password and the administrator password and that didn't work. And then I tried to log in as the administrator but could not remember where I told Leopard not to display it. Help!!

Rachael

---
A wanna be geek. A student but not blond



[ Reply to This | # ]
Manage the periodic logs
Authored by: jeremyp on Jul 27, '08 09:07:30AM

sudo is easy to use. Just type sudo and then the command you want to run on the same line (as in the example at the top). You will be prompted for a password which is your user account password that you are logged in as. On the standard Mac OSX install, only administrators are allowed to use sudo.

However, don't bother in this instance. My daily.out file is currently 242 Kbytes and that's since March 20. If it gets to 0.5Gb there's something wrong with your system and you'll need daily.out to find out what.



[ Reply to This | # ]