Move and click the mouse via code

May 14, '08 07:30:03AM

Contributed by: Anonymous

Most OS X apps can have their interface elements accessed by AppleScript, and so anything can be scripted. Unfortunately, not all apps use an Aqua interface, especially games with game-engine-rendered interface elements. This means AppleScript can't find buttons, etc., to click on, and you're out of luck trying to automate actions.

Recently, I was given the unpleasant task of automating actions in such unfriendly apps. I just about went crazy trying to find a way to write a simple script with coordinates for mouse click events.There's nothing available within AppleScript for pushing the mouse around, and I couldn't find any command-line utilities either. Out of necessity, I wrote my own.

This is my first time writing anything that even resembles Cocoa, and my first time writing code that compiles in nearly 10 years. I'm a scripting guy, and I just wanted to be able to write a script. What I wrote is a little command-line app called click that invokes CGPostMouseEvent from the CGRemoteOperation.h header file. It takes coordinates as command line arguments, moves the mouse to that position, then clicks and releases the mouse button. I think this could even be modified to click, hold, and drag, but I haven't taken it that far because I don't need to. Here's the source:

// File: 
// click.m
//
// Compile with: 
// gcc -o click click.m -framework ApplicationServices -framework Foundation
//
// Usage:
// ./click -x pixels -y pixels 
// At the given coordinates it will click and release.


#import <Foundation/Foundation.h>
#import <ApplicationServices/ApplicationServices.h>


int main(int argc, char *argv[]) {
  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
  NSUserDefaults *args = [NSUserDefaults standardUserDefaults];


  // grabs command line arguments -x and -y
  //
  int x = [args integerForKey:@"x"];
  int y = [args integerForKey:@"y"];


  // The data structure CGPoint represents a point in a two-dimensional
  // coordinate system.  Here, X and Y distance from upper left, in pixels.
  //
  CGPoint pt;
  pt.x = x;
  pt.y = y;


  // This is where the magic happens.  See CGRemoteOperation.h for details.
  //
  // CGPostMouseEvent( CGPoint        mouseCursorPosition,
  //                   boolean_t      updateMouseCursorPosition,
  //                   CGButtonCount  buttonCount,
  //                   boolean_t      mouseButtonDown, ... )
  //
  // So, we feed coordinates to CGPostMouseEvent, put the mouse there,
  // then click and release.
  //
  CGPostMouseEvent( pt, 1, 1, 1 );
  CGPostMouseEvent( pt, 1, 1, 0 );


  [pool release];
  return 0;
}
Save the above code as click.m, open Terminal, and switch to the folder where you saved the source. Then compile the program by typing gcc -o click click.m -framework ApplicationServices -framework Foundation. Don't be intimidated by needing to compile this as there are more comments than code. It is a very short program that does one simple task.

I'm sure I'm not the only one out there who needs to script mouse events, so hopefully this will brighten somebody's day. The only gotcha here is that scripts that invoke this will likely break if they are then used on a machine with a different screen size. Such is life when scripting the unscriptable.

[robg adds: You'll need Xcode installed to compile this code, but I've already done that, so here's a pre-built binary for those who want the program but don't have Xcode. Usage is as noted in the comments in the code: click -x 100 -y 200 would move the mouse horizontally 100 pixels and vertically 200 pixels, and then send a mouse click event. I tested it, and it works as described.]

Comments (29)


Mac OS X Hints
http://hints.macworld.com/article.php?story=2008051406323031