macosxhints.com Unix tips and tricks

Find users that use the most disk space

Thu, 11 Apr 2013 07:30:00 -0700

Find which users take up the most disk space. This can of course be used to indentify any number of users so I picked 10 as an arbitrary number.

If you are already root (unlikely, but possible) you can remove the sudo -s part. Here's the command:

sudo -s du -sm /Users/* | sort -nr | head -n 10

Generally you would have to make sure that you use sudo -s or it will give a few Permission denied errors before finally spitting out the results, and they may be incorrect. The directory structure should start with the /Users Directory and then it will recursively perform the operation. This may miss any folders outside of the normal user space, but there shouldn't normally be any user data there.

[crarko adds: I tested this, and it works as de ...

Flatten a directory structure

Tue, 26 Feb 2013 07:30:00 -0800

There may be times when you want to consolidate all the files in a directory and its sub-directories (or a folder and its sub-folders) into a single directory or folder. For example, you may have a folder with sub-folders for years, and other sub-folders in each year folder for months, and you may want to move files in the month folders all to the top level.

Doing this manually is a complex and time-consuming process. While you might be able to do this by using a search - for example, if all the files are, say, Excel files, you can search for Excel files in the top folder, then just copy them all to a new folder - if there are lots of different types of files, this wouldn't make things easier.

Fortunately, there's a way to do this from a command line. On the BedroomLAN blog, Alexios presents two commands that will do this:

cd $ROOT_DIRECTORY find -type f -print0 | xar ...

Extract names and emails from a text file

Wed, 20 Feb 2013 07:30:00 -0800

I have a recurring need to extract full names and email addresses from a plaintext archive of email messages. The archive is created by selecting a bunch of emails in Mail, copying them, pasting into TextEdit, and converting to plain text.

For each message in the file, the first line contains the information I wanted:

From: Joe Example <joe@example.com>

I wanted one email address per line, suitable for pasting into another location. I am far from an expert with the bash shell, but here's what I came up with�I imagine there are many more efficient ways to do this, as I'm sure experienced perl, sed, awk, etc. users may point out. Note that this is highly dependent on the format created by Apple's Mail app in OS X 10.8.

grep 'From:' /path/to/archive.txt | cut -f2 -d\< | cut -f1 -d\> | pbcopy

The grep bit pulls out the entire From: line, then the first cut ...

Control the Finder with the Terminal

Thu, 13 Dec 2012 07:30:00 -0800

There are several ways to open a Terminal window to the current directory in the Finder. But wouldn't it be useful if you could do the reverse and open a Finder window to the current Terminal directory? Well, you can, and you can completely control the Finder from the terminal.

I have put the code on github with full instructions on how to set it up. It works by using bash_completion, .bash_profile with some applescript to control the Finder, .inputrc and .bash_aliases.

Here are some of the features:

Changing a directory in the Terminal opens the same directory in the Finder.
You can change the Finder window view from the Terminal (column, list, icon views).
It is case insensitve, you can press Tab for menu completion, and Shift-Tab to expand bash aliases.
Open a Terminal directory to the current Finder window.

This code will work with both the Terminal ...

See how long a given process has been running

Wed, 28 Nov 2012 07:30:00 -0800

I wanted to find out how long a certain background process had been running. There's a column for CPU Time in Activity Monitor, but that's not real clock time.

It turns out you can get this information with ps, via the etime keyword. So to get a list of every running process, in decreasing order of run time, just use this command:

ps -ax -o etime,command -c

To see the results for a single process, just add a grep at the end for the process' name. For example:

$ ps -ax -o etime,command -c | grep AppleVNCServer
03-08:09:16 AppleVNCServer

So on my Mac, the AppleVNCServer has been running for three days, eight hours, nine minutes, and 16 seconds. I have a need to do this pretty regularly, so I turned it into a simple command line app:

#!/bin/bash

# Display the time a given process has been running
# Use the process name when calling the command

ps -ax -o etime,command -c | grep $1

I saved tha ...

See details of an app's virtual size

Fri, 19 Oct 2012 07:30:00 -0700

If you use top in Terminal, you may occasionally see apps with huge VSIZE values. I know this because Witch, one of our apps, is an example of such—it's VSIZE can exceed 11GB.

In trying to figure out why this was so (short answer seems to be: we can't control it, but it's not a problem), I ran across an interesting command, vmmap. This command will spew out a ton of detail about virtual memory usage. Stringing a couple Unix commands together, though, you can extract just the summary portion of the report. In Terminal, run this command:

vmmap -resident PID# | grep "Summary for" -A 999

Replace PID# with the process ID you're interested in observing, and you'll get a nice summary of its virtual memory state. You can see what's virtual, and what's actually resident. Here's a bit of the output for Witch, for example:

...
REGION TYPE                      VIRTUAL RESIDENT
====== ...

Quickly eject all disks from the command line

Wed, 05 Sep 2012 07:30:00 -0700

Instead of using the following to eject all disks:

osascript -e 'tell "Finder" to eject (every disk)'

I've found that this is much faster:

find /Volumes -maxdepth 1 -not -user root -print0 | xargs -0 diskutil eject

Run this command from Terminal, or in a shell script, to eject all your local disks.

To eject network volumes, this code works:

find /Volumes -maxdepth 1 -not -user root -print0 | xargs -0 umount

[kirkmc adds: The problem with this hint is that each line of code works for different types of volumes. If you have both local and network volumes, you need to use both; you could, of course, stick them together with a semi-colon separating them so they are, in effect, a single command.. In my tests, it's not really faster than the osascript code above, but I'm posting it because some may find it useful. If you have different results in speed or efficacy, please post in the comments.]

Send Terminal output to iCloud

Tue, 21 Aug 2012 07:30:00 -0700

Imagine that you run a script automatically on your Mac, and that you want to check the result of that script. There are many ways you could do this, such as remotely connecting to the Mac, or sending the results by e-mail. But with iCloud, you can also save the output to a file and put it on iCloud, where you can access it with your favorite iCloud-compatible text editor on another Mac, or on an iOS device.

To do so, simply send the output of the script to a file like this:

~/Library/Mobile\ Documents/com\~apple\~TextEdit/Documents/filename.txt

So, to save a list of a directory's contents, you'd use this:

ls -al > ~/Library/Mobile\ Documents/com\~apple\~TextEdit/Documents/list.txt

That saves a file called list.txt in TextEdit's Documents folder. Look inside the ~/Library/Mobile\ Documents folder for the paths to other apps you have that can use iCloud. Each folder in the Mobile ...

Command-drag to Terminal to change directory

Mon, 30 Jul 2012 07:30:00 -0700

If you hold down the Command key while dragging an icon to a Terminal window, it will cd to either the directory you dragged (if it's a folder) or the directory containing the item you dragged (if it's a file).

I don't know when this feature appeared; it might not be specific to 10.8, but that's where I discovered it.

[kirkmc adds: This is a really neat feature. Also, if you Command-drag an application or other bundle, Terminal will cd to inside that bundle. This is great if you need to access files within an application bundle, for example. The comments are saying that this is indeed 10.8 only.]

Script to delete huge sparsebundle images

Thu, 19 Jul 2012 07:30:00 -0700

I needed to restart a backup for one of my Macs, which had an 800 GB disk image on my Time Capsule. It turned out to be impossible to delete the disk image - in Finder, it would just not move to the Trash, even after waiting for several hours.

I was looking for advice and most of the suggestions involved reformatting the Time Capsule, but I couldn't do this, as I had other Macs being backed up there.

I looked for a way to do this from the command line. The Time Machine disk image - a sparsebundle, a special type of disk image that can increase in size as needed - is actually a package, which has "bands" inside, many smaller files that are named hexadecimally. In this case, there were thousands of them. So many, that even deleting them from the command line didn't work - commands like rm find limitations in the lengths of lists parsed by recursion or wildcards, so a basic command like rm -f didn't work.

I ended up writing a li ...

Save all of your Terminal activity to a file

Fri, 11 May 2012 07:30:00 -0700

I use Terminal everyday, and I find it a good idea to log everything I you. It makes it much easier to undo your mistakes when you know what those mistakes were. Here's how I do this:

Open Terminal's preferences.

Go to Settings, then Shell. You can choose here to run a command at startup. You could create a simple log of your session using the following:

/usr/bin/script ~/Desktop/Terminal.log

This will log everything you do and append it to the log file.

I like to keep my history so instead I wrote this small script that archives previous sessions by renaming the file with a date/time string. I then set Terminal.app to run at startup the following command:

~/Desktop/logger.sh

Here's the script:

#!/bin/bash
# logger.sh
# Log everything you do in Terminal.

#* Formatted date & time string.
FORMATTED_DATE=`/bin/date "+%Y-%m-%d%H%M%S"`

#* Archive the previous file
/bin/cp -f ~/Desktop/Terminal ...

Automatically restart network on lost connection

Tue, 17 Apr 2012 07:30:00 -0700

Often times, for whatever reason, the Internet connection on my Mac will lock up. Only restarting the Airport or Ethernet interfaces seem to resolve the issue in a timely manner. (This is different from a router crash which requires a manual restart of the router - in my case an Airport Extreme.) I have developed some interconnected scripts that will automatically accomplish this task for me when attached to launchctl. This is important for me because I often have to access files over the internet and if the connection has locked up there is no way for me to access them without manual intervention.

The first script gets the active network interface. This is called by the second script which determines whether to run a matched script for either the Ethernet or AirPort interfaces. These third and fourth scripts restart either the Ethernet or AirPort connection respectively. These last two scripts can also be configured to send you an e-mail if this happens by altering the ' ...

Automatically reclaim memory from leaky programs

Wed, 11 Apr 2012 07:30:00 -0700

Sometimes applications have a nasty habit of claiming a lot of memory and never releasing it back to the system. Over time, this can cause affect performance. Most of the time, this can simply be fixed by quitting and restarting the offending program, but I wanted to find a more elegant solution.

After searching the Internet, I found few scripts that I could tweak to achieve what I wanted. Heres what I came up with. The first script is a python script I found at StackExchange by user drfrogsplat that gets information about system memory. The second is a bash script I wrote that runs the purge command if you have over a certain amount of inactive memory, in my case 500MB (probably overkill). Note that I think you need developer tools installed to use the purge command. There's probably an easier way to do this with vm_stat directly but I'm not good enough at awk/grep/sed to it figure out.

The bash script can be attached to ...

Getting and setting printer options from the command line for use with lpadmin

Fri, 16 Mar 2012 07:30:00 -0700

lpadmin and lpoptions don't interact with OS X as you might think, or at least as they are documented. Here are some helpful notes and a script.
There are some good hints for adding printers via the command line with lpadmin: Managing multiple printers via the command line.
However, there is still confusion surrounding the setting of printer options from the command line, as a poster to Debian bugs pointed out back in 2006: lpoptions documentation doesn't. After doing some testing, here are the two main points to note:

If you use lpadmin and specify options with "-o", the PPD is altered and OS X will recognize the options for the printer.

However, if you setup the printer using lpadmin without any options, and later use lptoptions to set the options, they are not written to the PPD and the GUI is unaware of the pr ...

10.7: Fix crash when running shell scripts from script menu

Wed, 14 Mar 2012 07:30:00 -0700

It seems that 10.7 no longer wants to run shell scripts from the script menu. This may also happen in 10.6, but I do not have an install to test it with.

No matter what script I try, the SystemUIServer crashes. And somewhere in the crash report is this ominous message:

USING_FORK_WITHOUT_EXEC_IS_NOT_SUPPORTED_BY_FILE_MANAGER

This might have something to do with Apple tightening privileges for spawning processes, but it can be easily fixed by enclosing the shell script in AppleScript.

do shell script "sh ~/script.sh"

[kirkmc adds: I haven't tested this; I don't run shell scripts from the script menu, so I haven't seen crashes.]

Control+T in Terminal shows time snapshot

Fri, 27 Jan 2012 07:30:00 -0800

Not sure if this was available before 10.7, but hitting Control+T while running a command in the Terminal will show what process is executing, the load, the PID of the process and its user and kernel time.

I was running a script and accidentally hit Control+T instead of Command+T to create a new tab. I was surprised at what I got. Here is an example of what gets printed:

# buildOrder.py
load: 2.51  cmd: p4 15179 running 0.00u 0.00s
load: 2.23  cmd: p4 17962 waiting 0.01u 0.00s
load: 2.53  cmd: Python 15167 running 94.68u 66.33s
load: 2.60  cmd: Python 15167 running 150.71u 101.82s

[crarko adds: I wasn't able to reproduce this, but it may be due to the briefness of the running command. Give it a try and post a comment about your results. Try it in Snow Leopard too if you ...

Convert Clipboard contents to plain text

Tue, 17 Jan 2012 07:30:00 -0800

There was a recent article on the Macworld main site about apps you can buy that will strip formatting from text on the clipboard, leaving you with plain text that you can paste into documents without it having the wrong font, etc.

Although some of them do this automatically, I thought it's a bit much to pay for such a simple utility that can be done in one line on the CLI.

So here's a shell script that will convert the contents of the clipboard to plain text.

#!/bin/bash
#
# Convert contents of clipboard to plain text.

pbpaste | textutil -convert txt -stdin -stdout -encoding 30 | pbcopy

exit 0

The encoding ...

Compress all the PNG files in your Home directory

Wed, 23 Nov 2011 07:30:00 -0800

PNG image files are often not compressed with the optimal compression setting, however a one line command can fix this (and save some space).

Basically, one just needs to have MacPorts and the optipng port installed, and run the following command:

mdfind -0 -onlyin ~ "kMDItemContentType=public.png" | xargs -n 5 -0 optipng -preserve

For more details, see this blog entry.

[crarko adds: I haven't tested this one. Note that to use MacPorts you also need to have Xcode installed to compile the ports. If you already have MacPorts set up give this hint a try and let us know how efficient th ...

Try the GIMP version 2.7.3 in OS X

Thu, 17 Nov 2011 07:30:00 -0800

In a previous hint, I suggested to install the development version of the GIMP (which also has the new, well-known single window mode) in an Ubuntu virtual machine: which obviously wasn't an optimal solution.

Now, there is another option: install the GIMP 2.7.3 via MacPorts; here's how to do it.

First of all, if you don't have it yet, you need to install MacPorts, available from here.

It is always a good thing to do a selfupdate; in the Terminal type:

$ sudo port selfupdate

Now, we can install the GIMP's development version (currently available in version 2.7.3):

$ sudo port install gimp2-devel

There are also many, many dependencies, and thus many packages to compile form source, so you must be patient; it can take several hours (for me, on a 2010 Core 2 Duo Mac mini Server with 8 GB of RAM, it took approximately 5 hours).

Once installe ...

10.7: Removing a Delay in SSH commands to some hosts

Tue, 25 Oct 2011 07:30:00 -0700

Using SSH to connect to some servers can introduce long delays from the time the command is issued until the connection with the server is established. This hint describes two potential problems and offers a solution.

In connecting to a local server, my ssh command would take about 20 seconds to complete. Non-Macintosh clients would not see this delay. After searching for the web and slogging through a number of posts that suggested how to change the server to fix ssh delays, I pieced together enough information to find out how to change my client configuration to solve the problem.

The first, and largest cause of delays, was my client and the server working to authenticate me using 'GSSAPI' (Kerberos) authentication. That seemed to be a good 15-18 seconds of the delay. I fixed this problem by adding a configuration file for my user that disabled GSSAPI authentication for the host. The file I added is ~/.ssh/config and I created it to read: ...