Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Control multiple ssh connections via a master Terminal UNIX
Many Unix users may be familiar with ClusterSSH. This tool allows you to open up ssh terminal sessions to many hosts (like when administering a cluster), and then send commands to all of the connected machines simultaneously. csshX is a free, open source tool inspired by ClusterSSH, but designed to work with the OS X Terminal.app (rather than X11's xterm). Once downloaded, the csshX script can be run where it is, or copied into /usr/local/bin/ (or any other directory on your path) for convenience.

The basic command line is:
csshX hostname1 hostname2 hostname3 [...]
This will open three Terminal windows, and ssh into hostname1, hostname2, and hostname3, connecting to one host per window. A fourth (red colored) controller Terminal window will also be opened (this screenshot demonstrates how things look). Any commands typed into the controller window will be sent to all three hosts. You can also send commands directly to only one of the host terminals by selecting its window directly.

For advanced use, menus can be brought up by pressing Control-A. These menus provide you with options such as disabling/enabling input to individual windows, minimizing all the windows, retiling the windows, etc.

[robg adds: This worked as described in testing with my, um, two-machine cluster, and I can see how this would be very useful for those working with a larger number of machines.]
  Post a comment  •  Comments (12)  
  • Currently 2.20 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (10 votes cast)
 
[24,823 views] Email Article To a Friend View Printable Version
Unix command files, UTF-8, and the byte order mark UNIX
"A little knowledge is a dangerous thing" as they say. A long story for a problem people may rarely if ever encounter, but here goes:

I love TextWrangler for editing all kinds of text files. I set it to save in UTF-8 (with the initial byte order mark, or BOM) set by default. I discovered that the BOM makes Safari read HTML as Unicode automatically, without the need for a charset declaration, or messy entity codes for special characters. So now I can just type HTML freely in any languages and scripts I want.

Now over to Terminal: On my old Mac, I had a few default aliases set up for tcsh. I learned that now in Leopard the default shell is bash, which I am happy to note supports Unicode in pathnames seamlessly, but which uses a very different structure for keeping default aliases. I found my old ~/Library » init » tcsh » aliases.mine file and did my research: I copied the file, saved it as ~/.bash_alias, and created ~/.bash_profile to source it.

But nothing would work. I got the strangest errors, like -bash: source: command not found. Say what?! The command is right there in /usr/bin/ where it belongs! I dug for answers on the net for hours, and kept trying things. Eventually I noticed that when I executed ~/.bash_alias myself on the command line, all but the first of my aliases loaded. When I changed the file to start with a blank line, all aliases loaded, with one error about an empty command. Ahha! So the problem turned out to be the file format: the BOM made the first word of the first line into nonsense. So I resaved both of my dot-files in "UTF-8, no BOM" mode, and all is well.

Moral of the story: Though we know "There ain't no such thing as plain text," Unix requires command files to be as close to it as possible.

  Post a comment  •  Comments (17)  
  • Currently 2.33 / 5
  You rated: 4 / 5 (9 votes cast)
 
[15,066 views] Email Article To a Friend View Printable Version
10.5: A fix for failing SSH Bouncing UNIX
As a Linux SysAdmin, of course my favorite desktop is OS X! On my MacBook Pro, I run CentOS using VMware. In my work, every day I use a fantastic technique called SSH Bouncing, both from within the VMware Linux environment and from Terminal on the Mac itself.

Explained in simple terms, assume you're on a computer, A, and you want to ssh into a computer (say, C) that is isolated from the world on a private network. If a computer B exists (and you already have an account on it) that has two network interfaces, one facing the internet (or at least the network you're on) and the other facing the private network, SSH Bouncing let's you ssh directly from computer A to computer C in one jump, with no added software or weakening of your security setup, as it requires the already-existing account on computer B.

This greatly simplifies file transfer, since you don't have to do the two step "SSH Hopping" of A » B, B » C. And popping a graphical X-Window off of C » A isn't possible at all without this technique. See this article for all the technical details on setting this up. It's very straightforward, and you'll need to understand it first for this hint to be useful.

Anyway, the point of this hint is that I used this all the time under Tiger, and when I upgraded my home and work Macs to Leopard, this broke. Specifically Mac » Mac » (destination) bouncing broke. I kept getting "Connection closed by remote host" errors.
read more (254 words)   Post a comment  •  Comments (14)  
  • Currently 2.13 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (8 votes cast)
 
[11,982 views] Email Article To a Friend View Printable Version
Use tar to quickly create segmented files UNIX
With today's increased Flash storage sizes, there may not be many times when you need to split files. However, when you do need to do so, I find that tar is a good tool (this hint is also good if you want to split an archive before burning to DVDs).

The following use of bash's brace expansion makes creating multi-volume tar files easy. To create a multi-file archive of a given length, use this:

tar --tape-length=102400 -cMv --file=tar_archive.{tar,tar-{2..100}} [files to tar]

To extract from the archive:

tar -xMv --file=tar_archive.{tar,tar-{2..100}} [files to extract]

To create DVD-sized volumes, use --tape-length=4588544. In the above examples, replace [files to tar] and [files to extract] with the location(s) of the file(s) you wish to segment.

[robg adds: I haven't tested this one.]
  Post a comment  •  Comments (8)  
  • Currently 2.55 / 5
  You rated: 1 / 5 (11 votes cast)
 
[14,299 views] Email Article To a Friend View Printable Version
10.5: Compare versions of files on Time Machine backup UNIX
There are (or have been) a number of version control systems en vogue over time -- CVS, SVN, Git, etc. I try to keep up with them and use them where possible, but I don't put everything I do in version control. Since I am on a Mac running Leopard, I do have (and use) Time Machine. So I wanted to see if I could use my Time Machine backups to do some quick comparisons between source files.

The result is the following quick and dirty python script. Note that you can find the original version of this script in this post on my blog; you may want to check there for an updated version. Here's the code:
#!/usr/bin/env python
# encoding: utf-8
"""
Created by Preston Holmes on 2009-02-23.
Copyright (c) 2009 __MyCompanyName__. All rights reserved.
"""

import sys
import os
import getopt
import difflib
import time
import pdb
from subprocess import Popen, PIPE

# if you set time_machine_path, it should be to the full path of this machines backup drive:
# ie '/Volumes/TM_Drive/Backups.backupsdb/Joes-Mac/'
# if not set explicitly - the script will use the first TM drive it finds, 
# and the first host folder it finds - which should work for most cases
# Will Not work with network or disk image based time machine backups

time_machine_path = None

verbose = False
cmd = 'tell application "Finder" to name of (path to startup disk)'
boot_volume = Popen('osascript -e '%s'' % cmd,shell=True,stdout=PIPE,stderr=PIPE).communicate()[0][0:-1]



help_message = '''
Call this script with the path to one or more text based files as arguments
'''
header = '''

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html>

<head>
    <meta http-equiv="Content-Type"
          content="text/html; charset=ISO-8859-1" />
    <title></title>
    <style type="text/css">
        table.diff {font-family:Courier; border:medium;}
        .diff_header {background-color:#e0e0e0}
        td.diff_header {text-align:right}
        .diff_next {background-color:#c0c0c0}
        .diff_add {background-color:#aaffaa}
        .diff_chg {background-color:#ffff77}
        .diff_sub {background-color:#ffaaaa}
    </style>
</head>

<body>
'''
footer = '''
    <table class="diff" summary="Legends">
        <tr> <th colspan="2"> Legends </th> </tr>
        <tr> <td> <table border="" summary="Colors">
                      <tr><th> Colors </th> </tr>
                      <tr><td class="diff_add">&nbsp;Added&nbsp;</td></tr>
                      <tr><td class="diff_chg">Changed</td> </tr>
                      <tr><td class="diff_sub">Deleted</td> </tr>
                  </table></td>
             <td> <table border="" summary="Links">
                      <tr><th colspan="2"> Links </th> </tr>
                      <tr><td>(f)irst change</td> </tr>
                      <tr><td>(n)ext change</td> </tr>
                      <tr><td>(t)op</td> </tr>
                  </table></td> </tr>
    </table>
</body>

</html>
'''
class Usage(Exception):
    def __init__(self, msg):
        self.msg = msg

def find_versions(path):
    #pdb.set_trace()
    global verbose
    print 'looking for versions of %s' % path
    #print time.strftime("%m/%d/%Y %I:%M:%S %p",time.localtime(os.path.getmtime(fname)))
    backups = os.listdir(time_machine_path)
    backups.sort()
    pathlist = [os.path.join(time_machine_path,b,boot_volume,path[1:]) for b in backups]
    pathlist.append(path)
    versions = []
    mod_times = []
    for f in pathlist:
        if verbose:print 'nLooking for modified version in %s' % f
        if os.path.exists(f):
            if verbose:print 'File Exists'
            mod_time = time.localtime(os.path.getmtime(f))
            if not mod_time in mod_times:
                if verbose:print '****************** unique version **********************'
                mod_times.append(mod_time)
                versions.append({mod_time:f})
    return versions
    
def getTMLocation():
    global time_machine_path
    if time_machine_path and os.path.exists(time_machine_path):
        return True
    volumes = os.listdir('/Volumes')
    #hostname = os.uname()[1].split('.')[0]
    #cmd = 'scutil --get ComputerName'
    #machine_name = Popen('osascript -e '%s'' % cmd,shell=True,stdout=PIPE,stderr=PIPE).communicate()[0][0:-1]
    for v in volumes:
        if os.path.exists(os.path.join('/Volumes',v,'Backups.backupdb')):
            backupsdb = (os.path.join('/Volumes',v,'Backups.backupdb'))
            time_machine_path = os.path.join(backupsdb,os.listdir(backupsdb)[0])
            return True
            # candidate_path = os.path.join('/Volumes',v,'Backups.backupdb',machine_name)
            # if os.path.exists(candidate_path):
            #     time_machine_path = candidate_path
            #     return True
    return False
    
def main(argv=None):
    global verbose
    if not getTMLocation():
        print 'No Time Macine Backup Found'
        sys.exit()
    print 'time machine path: ' + time_machine_path
    if argv is None:
        argv = sys.argv
    try:
        try:
            opts, args = getopt.getopt(argv[1:], "ho:v", ["help", "output="])
        except getopt.error, msg:
            raise Usage(msg)
    
        # option processing
        for option, value in opts:
            if option == "-v":
                verbose = True
            if option in ("-h", "–help"):
                raise Usage(help_message)
            if option in ("-o", "–output"):
                output = value
        if not args:
            raise Usage(help_message)
        differ = difflib.HtmlDiff(tabsize=4)
        html = header
        for path in args:
            path = os.path.join(os.getcwd(),path)
            html += '<h1>Changes for %s</h1' % os.path.basename(path)
            versions = find_versions(path)
            if len(versions) < 2:
                html += '<h2>Less than 2 Versions found on Time Machine Backup</h2>'
            else:
                html += '<h2>%s versions found</h2>' % len(versions)
            for i in range(0,len(versions)):
                if i: #skip the first
                    d1 = time.strftime("%m/%d/%Y %I:%M:%S %p",versions[i-1].keys()[0])
                    d2 = time.strftime("%m/%d/%Y %I:%M:%S %p",versions[i].keys()[0])
                    html = html + '<h2>changes from %s to %s</h2>' % (d1,d2)
                    l1 = open(versions[i-1].values()[0]).readlines()
                    l2 = open(versions[i].values()[0]).readlines()
                    table = differ.make_table(l1,l2,context=True,numlines=3)
                    html += table
        html += footer
        o = open('/tmp/diff.html','w')
        o.write(html)
        o.close()
        import webbrowser
        webbrowser.open('/tmp/diff.html')
    except Usage, err:
        print >> sys.stderr, sys.argv[0].split("/")[-1] + ": " + str(err.msg)
        print >> sys.stderr, "t for help use –help"
        return 2


if __name__ == "__main__":
    sys.exit(main())
In the unlikely event that I ever have time, this would be a cool pyObjC project -- add a file browser panel, date versions picker, and a webkit view (with some better CSS).

[robg adds: To use this script, save the file (filecomp.py or whatever), make it executable (chmod a+x filecomp.py), and then run it in Terminal, passing the path to a text file as an argument. Assuming you saved it somewhere on your path, that would look something like: filecomp.py ~/Documents/myfile.txt. It seemed to work in my testing, though thanks to a recent mistake on my part, I don't have much Time Machine history to dig through.]
  Post a comment  •  Comments (4)  
  • Currently 1.67 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (6 votes cast)
 
[9,970 views] Email Article To a Friend View Printable Version
Fix a broken Perl install after 2009-01 Security Update UNIX
It appears that if you have used CPAN to augment your default Perl installation, then Security Update 2009-001 "kills" Perl, so your scripts won't run. There is a more discussion about this in this Apple Discussions thread.

The fix mentioned in that thread that worked for me was the following, all done in Terminal:
$ mkdir -p /SourceCache
$ cd /SourceCache
$ curl -O http://cpan.mirror.solnet.ch/authors/id/G/GB/GBARR/IO-1.2301.tar.gz
$ tar xzf IO-1.2301.tar.gz
$ cd IO-1.2301
$ perl Makefile.PL
$ make
$ make install
Note that for some of the above commands, you must be logged in as root, or run using sudo.

[robg adds: According to the linked discussion, only those who have updated certain core Perl modules via CPAN will have this problem. In this post (linked in the above discussion), the author states more specifically that the problem is that "the Security Update brings (old) IO.bundle with version 1.22, but your IO.pm has been updated to the latest 1.23 on CPAN shell." The author mentions you may have to make similar fixes to the Scalar::Util and Storable modules. I haven't tested this one.]
  Post a comment  •  Comments (4)  
  • Currently 2.70 / 5
  You rated: 5 / 5 (10 votes cast)
 
[9,918 views] Email Article To a Friend View Printable Version
Determine multiple display layout via bash script UNIX
I searched far and wide, but only found third party utility references (an app called cscreen, for one) to get the current display layout programmatically in a script (AppleScript / bash). This could be easily munged into AppleScript, but here's what I came up with for a bash function that output something like:

Display Unit: 0 with height 900 and width 1440 is positioned at (0, 0)
Display Unit: 1 with height 1024 and width 1280 is positioned at (503, -1024)


However, you can massage the output to suit your needs. The script only returns the last set-up displays. In other words, if a display is hot-unplugged, the script will still report the last layout set in System Preferences.

Here's the script; it's only been tested on 10.5.6:
#!/bin/sh

DisplayListMac()
 {
    defaults read /Library/Preferences/com.apple.windowserver 'DisplaySets' | awk 'BEGIN { aDepth=0; iDepth=0; aNum=0; iNum=0; dID="" }
    /\(/ {
            ++aDepth
            iNum=0
#            print "D:" aDepth
         }
    /\)/ {
            --aDepth
            ++aNum
#            print "D:" aDepth " N:" aNum
         }
    /\{/ {
            ++iDepth
#            print "I:" iDepth
            if (dID != "")
             {
                print dUnit ":" dX ":" dY ":" dW ":" dH
                dID=""
             }
         }
    /\}/ {
            --iDepth
            ++iNum
#            print "I:" iDepth " N:" iNum
         }
    /=/ {
#            print "K:" $1 ":" $3 ":"
            if (aNum == 0)
             {
                # Remove semicolon
                v=substr($3, 1, length($3) - 1)
                if ($1 == "Height" )
                    dH = v
                else if ($1 == "Width" )
                    dW = v
                else if ($1 == "OriginX" )
                    dX = v
                else if ($1 == "OriginY" )
                    dY = v
                else if ($1 == "DisplayID")
                    dID = v
                else if ($1 == "Active")
                    dAct = v
                else if ($1 == "Depth")
                    dDepth = v
                else if ($1 == "Unit")
                    dUnit = v
             }
        }
'
 }

for i in $(DisplayListMac); do
    u=${i%%:*}
    rest=${i#*:}
    x=${rest%%:*}
    rest=${rest#*:}
    y=${rest%%:*}
    rest=${rest#*:}
    w=${rest%%:*}
    rest=${rest#*:}
    h=${rest%%:*}

    echo "Display Unit: $u with height $h and width $w is positioned at ($x, $y)"
done
[robg adds: To use the script, copy and paste the above into vi, emacs, BBEdit, TextEdit, or other pure-text editor. Save the file with a meaningful name (screeninfo). Then, in Terminal, cd to the directory where you saved the file, and run this command to make the file executable: chmod a+x screeninfo. You can then run the script (assuming it's been saved somewhere on your path) by typing screeninfo (or from within the current directory with ./screeninfo). It worked fine on my dual-monitor setup.]
  Post a comment  •  Comments (6)  
  • Currently 1.00 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (5 votes cast)
 
[11,054 views] Email Article To a Friend View Printable Version
Set file's date based on filename via bash and Automator UNIX
I have tons of radio show recordings, which all automatically have the date they were recorded in the filename, as so: Reelin and Rockin with Z and D -[2008-10-18].mp3. However, for a podcast, I want the actual creation/modification date to reflect the date of the show. Unfortunately, whenever I actually get around to editing the audio (to cut out previous/following DJs), the date does not reflect the date it was recorded.

I now have 50 or so files with the incorrect dates, and would like this to be automatic at some point anyway. So I figured out how to write a Bash shell script (which can be run via an Automator plug-in) to set the dates correctly using the filename. So now I can just Control-click on the files and choose the script from the Automator section of the contextual menu. (And now I can tweak it to use as a Folder Action, to run automatically when I save the edited audio file!)

Here's the shell script:
for f in "$@"
do

if [ -n "$f" ]; then
  fullpath=$f
else
  echo "No filename specified"
  echo "Usage: KCSB-SetDate <filename>"
fi

fname=`basename "${fullpath}"`

year=`echo $fname | cat | awk -F - '{print $2}' | awk -F [ '{print $2}'`
month=`echo $fname | cat | awk -F - '{print $3}'`
day=`echo $fname | cat | awk -F - '{print $4}' | awk -F ] '{print $1}'`

fulltime="${year}${month}${day}0000"

touch -t "$fulltime" "${fullpath}"

done
The automator action is:
  • Finder: Get selected Finder items
  • Automator: Run Shell Script with 'Shell' set to bash, and 'Pass input' set to as arguments, and the code itself as shown above (replace any existing code in the script box with the code from above).
I then saved this as a Plug-In for Finder. To make it a bash shell script instead, remove the for/do/done loop, and change $f to $1 (for files as arguments)

Other things I had to learn to get this working:
  • The awk pipes: the -F option lets you specify a delimiter/separator, and so -F - makes a dash (-) the separator (with the backslash to make sure bash doesn't try to interpret it, just in case). Then the {print $n} commands to awk make awk print out the nth string, if each string were separated by the - delimeter we specified.
  • To debug a shell script from within Automator, you must add the View Results object at the end, so you can see the results of echos and outputs etc.
[robg adds: I haven't tested this one.]
  Post a comment  •  Comments (0)  
  • Currently 2.36 / 5
  You rated: 1 / 5 (11 votes cast)
 
[10,489 views] Email Article To a Friend View Printable Version
One way to cut and paste between X11 and Aqua UNIX
Quite often when reading macosxhints, I read a hint that involves a terminal command, usually a defaults write hidden preferences trick. The easiest way to enter these commands is to copy them from Safari, and paste them on to the command line in Terminal. The problem is that most of the time I'm using xterm running under X11, and it doesn't support cut and paste with Aqua.

One can get around this problem by entering the following at the shell prompt.
pbpaste | sh
That's it. pbpaste is a shell command that writes the contents of the clipboard to standard output; piping it into the shell causes the shell to execute the command that was on the clipboard. pbcopy works in the other direction -- text sent to the standard input of pbcopy ends up on the clipboard. For example:
echo "Hello Aqua" | pbcopy
puts Hello Aqua on the clipboard.

[robg adds: We've discussed pcbopy and pbpaste before, but never in the context of using them to get the clipboard in X11.]
  Post a comment  •  Comments (8)  
  • Currently 1.89 / 5
  You rated: 5 / 5 (9 votes cast)
 
[14,576 views] Email Article To a Friend View Printable Version
Set pbcopy to use UTF-8 by default UNIX
After much frustration, I realized that the shell command pbcopy was destroying non-ASCII characters in an AppleScript that I was trying to write. Although pbcopy is an extremely convenient way to take the standard input and place it on the clipboard, I also needed access to accented characters.

Luckily, I found this blog post that explained everything. The important part was a comment there that explained how to set a default encoding by adding the following line to my .profile file:
export __CF_USER_TEXT_ENCODING=0x1F5:0x8000100:0x8000100
While this fixed the problem while working in the Terminal, it did not fix the problem within AppleScript. I did find that I could add the aforementioned export command inside the do shell script, and everything seemed to work as expected. An example is shown below.
do shell script "export __CF_USER_TEXT_ENCODING=0x1F5:0x8000100:0x8000100; cat -s " & quoted form of (fileToCopy) & " | pbcopy"
This has been working great for me, but one of the posts on the blog does claim a problem. The blog author also wrote a replacement for pbcopy and pbpaste that is also supposed to alleviate these issues. I have not yet tried it, however.
  Post a comment  •  Comments (14)  
  • Currently 2.50 / 5
  You rated: 1 / 5 (12 votes cast)
 
[13,861 views] Email Article To a Friend View Printable Version