Mounting a shared file system for Xgrid

Jun 10, '10 07:30:00AM

Contributed by: SOX

Xgrid is Apple's cluster computing system built into the OS. Typical jobs that use Xgrid are data intensive tasks that need remote access to input to files and/or large databases. Unfortunately, for security reasons, on a distributed Xgrid network the jobs running on agents can't access /Volumes and thus can't access remote files systems that are mounted in the usual way. So where can a remote file system be mounted that Xgrid agents can access on demand?

One of the few places on an agent computer that the Xgrid job is allowed to read from is /tmp so you mount it in /tmp. At first this seems unsuitable since any directory created for the mount point won't last past a reboot (/tmp is erased every reboot). But an unexpected feature of the mount process on OS X as opposed to Linux is that it will auto_create a directory for you if it is missing when the mount command is issued. This makes this possible to work using just the standard automounter.

For example, suppose an NFS disk server named myDiskServer on the local network is exporting a directory found on its disk at /Volumes/MacHD/export_directory. Then to import this shared NFS file system just append the following line to the end of /etc/auto_master on each agent computer:

/private/tmp/xsfs myDiskServer.local:/Volumes/MacHD/export_directory
Then after you execute 'automount' (or reboot) you will find you can access the disk server from an Xgrid job at /tmp/xsfs/.

Note this assumes that the NFS disk server is allowing exports to that agent computer. How to set up a disk server itself is beyond the scope of this hint and varies from Linux, to Windows, to Mac OS X. But if you are using an OS X Snow Leopard Server as the fileserver then you definitely need to check out this recipe to avoid pitfalls .

Alternatively, instead of NFS, there are previous hints for SMB and AFP automounting. (But heed this precaution for SMB) Also here is an informative discussion of why NFS may be sometimes better for Xgrid. But, regardless of the protocol, the key point is that mounting the filesystem in /tmp will work for Xgrid.

I'd like to give a hat tip to the new TenGrid.com wiki for the solution to this.

[crarko adds: I haven't tested this one.]

Comments (3)


Mac OS X Hints
http://hints.macworld.com/article.php?story=20100609114725470