Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Move and click the mouse via code Apps
Most OS X apps can have their interface elements accessed by AppleScript, and so anything can be scripted. Unfortunately, not all apps use an Aqua interface, especially games with game-engine-rendered interface elements. This means AppleScript can't find buttons, etc., to click on, and you're out of luck trying to automate actions.

Recently, I was given the unpleasant task of automating actions in such unfriendly apps. I just about went crazy trying to find a way to write a simple script with coordinates for mouse click events.There's nothing available within AppleScript for pushing the mouse around, and I couldn't find any command-line utilities either. Out of necessity, I wrote my own.

This is my first time writing anything that even resembles Cocoa, and my first time writing code that compiles in nearly 10 years. I'm a scripting guy, and I just wanted to be able to write a script. What I wrote is a little command-line app called click that invokes CGPostMouseEvent from the CGRemoteOperation.h header file. It takes coordinates as command line arguments, moves the mouse to that position, then clicks and releases the mouse button. I think this could even be modified to click, hold, and drag, but I haven't taken it that far because I don't need to. Here's the source:
// File: 
// click.m
//
// Compile with: 
// gcc -o click click.m -framework ApplicationServices -framework Foundation
//
// Usage:
// ./click -x pixels -y pixels 
// At the given coordinates it will click and release.


#import <Foundation/Foundation.h>
#import <ApplicationServices/ApplicationServices.h>


int main(int argc, char *argv[]) {
  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
  NSUserDefaults *args = [NSUserDefaults standardUserDefaults];


  // grabs command line arguments -x and -y
  //
  int x = [args integerForKey:@"x"];
  int y = [args integerForKey:@"y"];


  // The data structure CGPoint represents a point in a two-dimensional
  // coordinate system.  Here, X and Y distance from upper left, in pixels.
  //
  CGPoint pt;
  pt.x = x;
  pt.y = y;


  // This is where the magic happens.  See CGRemoteOperation.h for details.
  //
  // CGPostMouseEvent( CGPoint        mouseCursorPosition,
  //                   boolean_t      updateMouseCursorPosition,
  //                   CGButtonCount  buttonCount,
  //                   boolean_t      mouseButtonDown, ... )
  //
  // So, we feed coordinates to CGPostMouseEvent, put the mouse there,
  // then click and release.
  //
  CGPostMouseEvent( pt, 1, 1, 1 );
  CGPostMouseEvent( pt, 1, 1, 0 );


  [pool release];
  return 0;
}
Save the above code as click.m, open Terminal, and switch to the folder where you saved the source. Then compile the program by typing gcc -o click click.m -framework ApplicationServices -framework Foundation. Don't be intimidated by needing to compile this as there are more comments than code. It is a very short program that does one simple task.

I'm sure I'm not the only one out there who needs to script mouse events, so hopefully this will brighten somebody's day. The only gotcha here is that scripts that invoke this will likely break if they are then used on a machine with a different screen size. Such is life when scripting the unscriptable.

[robg adds: You'll need Xcode installed to compile this code, but I've already done that, so here's a pre-built binary for those who want the program but don't have Xcode. Usage is as noted in the comments in the code: click -x 100 -y 200 would move the mouse horizontally 100 pixels and vertically 200 pixels, and then send a mouse click event. I tested it, and it works as described.]
    •    
  • Currently 3.33 / 5
  You rated: 3 / 5 (12 votes cast)
 
[63,533 views]  

Move and click the mouse via code | 29 comments | Create New Account
Click here to return to the 'Move and click the mouse via code' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Move and click the mouse via code
Authored by: sweyhrich on May 14, '08 07:56:19AM

Sorry to be stupid, but where does this "click" binary go so Applescript programs can find it?

---
Steven Weyhrich
http://apple2history.org



[ Reply to This | # ]
Move and click the mouse via code
Authored by: ebaur on May 14, '08 08:28:45AM
You can place it in /usr/local/bin (I'm pretty sure that's in your default path). Then you can call it like any other command line program.

[ Reply to This | # ]
Move and click the mouse via code
Authored by: ghay on May 14, '08 07:59:21AM

Just a note to say that you are using cocoa objc to access carbon code.

There is a carbon example HIDTestTool on apple's site, which can be easily modified and some CGPostMouseEvents added to move the mouse.

Just if you prefer not mixing your carbon's and cocoa's.



[ Reply to This | # ]
Move and click the mouse via code
Authored by: ghay on May 14, '08 12:12:31PM

I should perhaps say that I was using the HID example to trap apple remote keypresses and map them to the mouse.

It isn't required to use anything from the HID example to set the mouse loc



[ Reply to This | # ]
Move and click the mouse via code
Authored by: Pedro Estarque on May 14, '08 08:07:47AM

Great one, thanks for sharing!
I would add a -h --help flag to be called when no arguments are given.



[ Reply to This | # ]
Move and click the mouse via code
Authored by: ob1cannoli on May 14, '08 10:09:45AM
Not to say your work is useless, but this works well for me:

activate application "Finder"
tell application "System Events"
	tell process "Finder"
		click at {15, 2}
	end tell
end tell


[ Reply to This | # ]
Move and click the mouse via code
Authored by: ghay on May 14, '08 10:17:04AM

Do you have "Enable Access for Assistive Devices" checked in your "Universal Access" system preferences?



[ Reply to This | # ]
Move and click the mouse via code
Authored by: ob1cannoli on May 14, '08 10:30:19AM

Yup. I'm on 10.5 and using an macbook/external monitor if that makes a difference..



[ Reply to This | # ]
Move and click the mouse via code
Authored by: ghay on May 14, '08 12:08:56PM

Your script will work with the setting on, but you are more than likely out of luck with it off, the original hint will work regardless.

Also, I haven't tested if telling the finder (or even system events) to click will work across into other applications.



[ Reply to This | # ]
Would this work outside of the Finder?
Authored by: jecwobble on May 14, '08 11:20:58AM

I would think that if you are in another active application that you want to click on, that activating the Finder would deactive the window you need the script to opperate on...



[ Reply to This | # ]
Move and click the mouse via code
Authored by: lotusone on Jul 27, '09 07:04:05PM

As mentioned in the post, this is for Apps that do not use standard Cocoa UI elements and thus can not be scripted using "click at." Finder is a Cocoa app, so this is not necessary, but try using "click at" on a web page in Safari using a Flash control and you'll be tearing your hair out.



[ Reply to This | # ]
Move and click the mouse via code
Authored by: Dr. T on May 14, '08 11:14:18AM

First, I wish to echo sweyhrich's comment about using Click within Applescripts.

I also cannot figure out how to make Click work via a Terminal command. I type command-shift-N to open a command line window, type 'click -x 90 -y 30', click the Run button, and get no detectable mouse click. Instead, a new Terminal bash window appears in the middle of my screen. Could this be due to owning a PowerPC Mac and running robg's binary (that may have been for Intel Macs)?



[ Reply to This | # ]
Move and click the mouse via code
Authored by: chrononaut on May 14, '08 11:35:13AM

This won't work unless you check "run command inside of shell" or actually run it from the command line. I don't know why. But other than that issue it works splendidly for me.



[ Reply to This | # ]
Move and click the mouse via code
Authored by: mjb on May 14, '08 05:48:44PM

Yes, robg's binary is not Universal, it's intel only.



[ Reply to This | # ]
Move and click the mouse via code
Authored by: sojourner on May 14, '08 06:24:45PM
there's another utility out there you might check out:
http://www.bluem.net/downloads/cliclick-en/

from the website:
"Cliclick" stands for "Command-Line Interface CLICK", which says it all—it is a a tiny shell / Terminal application that will emulate keyboard clicks at arbitrary screen coordinates.

[ Reply to This | # ]
Move and click the mouse via code
Authored by: iamsoclever on Jun 03, '08 11:25:38PM

Here's how to use this with AppleScript:

tell application "Terminal"
do shell script "/path/to/click click -x 300 -y 2"
end tell



[ Reply to This | # ]
Move and click the mouse via code
Authored by: rlaan on Jun 04, '08 04:35:27AM

In the OS9 days a little app existed called: Okidokie Pro. Okiedokie could click the default button of any dialog with a defaultbutton within a preset timeinterval. Does anything like that exist for OSX?

---
The box said, Windows 95 or better, so I bought OS X.



[ Reply to This | # ]
Move and click the mouse via code
Authored by: hot_wired13 on Jun 18, '08 07:18:29AM

OMG I SO LOVE YOU! THIS CODE IS A LIFE SAVER MAN!
THANK YOU THANK YOU THANK YOU THANK YOU!
I almost pulled my hair out when applescript did "half-selects" inside Adobe Flash CS3 - and by the way, that app is a NIGHTMARE to.. well, applescript.

This was like 3 days after googling for stuff. So, here, let me put some keywords for the rest of the guys who are stuck with the same problem:

Adobe Flash CS3 AppleScript Problem, Cannot select radio button, Applescript select button



[ Reply to This | # ]
Move and click the mouse via code
Authored by: alichino on Oct 23, '09 07:53:28AM

CGPostMouseEvent has been deprecated in SnowLeopard.

You can replace it with something like

CGEventRef mouseDownEv = CGEventCreateMouseEvent (NULL,kCGEventLeftMouseDown,pt,kCGMouseButtonLeft);
CGEventPost (kCGHIDEventTap, mouseDownEv);

CGEventRef mouseUpEv = CGEventCreateMouseEvent (NULL,kCGEventLeftMouseUp,pt,kCGMouseButtonLeft);
CGEventPost (kCGHIDEventTap, mouseUpEv );

With some minor adjustments (see Apple docs) you can also send wheel events, drag the mouse and the like.

Cheers



[ Reply to This | # ]
Move and click the mouse via code
Authored by: tje210 on Feb 02, '10 12:15:02AM

like it, like it, like it. one thing though.

how do i make the cursor move without clicking? i'd like to do a series of mouse clicks and movements to perform a task. that is, i need to click on my fan menu item, then select the fan speed option, then move it straight across, move it down, pause (long enough for the menu item to be highlighted) then click. reason i want to automate it is so i can control it with speech.

any help is appreciated.

## edit

i just realized... if i could just incorporate a small wait time between moving the mouse and clicking it, that would solve my problem. is it possible to do that simply in the program? sorry if that's a silly question... i'm still new to this code, and it all looks mostly chinese to me.

Edited on Feb 02, '10 12:22:56AM by tje210



[ Reply to This | # ]
Move and click the mouse via code
Authored by: rccharles on Feb 10, '10 11:57:45AM

Here are two sleeps routines you could add:

// Possible sleep routines
usleep(100000);
sleep(2);



[ Reply to This | # ]
Move and click the mouse via code
Authored by: rccharles on Feb 10, '10 12:05:45PM
I have developed a drag routine. I called it clickdrag>

// File: 
// clickdrag.m
//
// clickdrag will drag the "mouse" from the current cursor position to
// the given coordinates.
//
// derived from:
//  http://www.macosxhints.com/article.php?story=2008051406323031
//    &
//  http://davehope.co.uk/Blog/osx-click-the-mouse-via-code/
//
// Requires:
//   xtools
//
// Compile with: 
// gcc -o clickdrag clickdrag.m -framework ApplicationServices -framework Foundation
//
// Usage:
// ./clickdrag -x pixels -y pixels
//



#import <Foundation/Foundation.h>
#import <ApplicationServices/ApplicationServices.h>


int main(int argc, char *argv[]) {
  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
  NSUserDefaults *args = [NSUserDefaults standardUserDefaults];


  // grabs command line arguments -x and -y
  //
  int x = [args integerForKey:@"x"];
  int y = [args integerForKey:@"y"];


  // The data structure CGPoint represents a point in a two-dimensional
  // coordinate system.  Here, X and Y distance from upper left, in pixels.
  //
  CGPoint pt;
  pt.x = x;
  pt.y = y;

  // This is where the magic happens.  See CGRemoteOperation.h for details.

  // Get the current mouse location
  CGEventRef ourEvent = CGEventCreate(NULL);
  CGPoint ourLoc = CGEventGetLocation(ourEvent);
  
  // CGPostMouseEvent( CGPoint        mouseCursorPosition,
  //                   boolean_t      updateMouseCursorPosition,
  //                   CGButtonCount  buttonCount,
  //                   boolean_t      mouseButtonDown, ... )
  //
  
  
  

  // Start drag from current cursor location
  CGPostMouseEvent( ourLoc, TRUE, 1, FALSE );
  CGPostMouseEvent( ourLoc, FALSE, 1, TRUE );

  // End drag by moving to new location
  CGPostMouseEvent( pt, TRUE, 1, TRUE );
  CGPostMouseEvent( pt, FALSE, 1, FALSE );
  
  // Possible sleep routines
  //usleep(100000);
  //sleep(2);

  [pool release];
  return 0;
}
Here is the applescript code to test clickdrag.

(* 
 Required:
  Paintbrush
  http://paintbrush.sourceforge.net/downloads/
  
  Optional:
     Play Sound
   	http://microcosmsoftware.com/playsound/
  
     the XTools scripting addition gives AppleScript the ability to move the mouse. 
	
	http://www.lestang.org/osax/XTool/
	XTool-2.0-src.dmg.tgz 
*)
on run
	
	-- Write a message into the event log.
	log "  --- Starting on " & ((current date) as string) & " --- "
	
	
	(* 
	Play Sound
   	http://microcosmsoftware.com/playsound/
	*)
	try
		set theLoc to ((path to system folder) as string) & "Library:Sounds:Funk.aiff"
		tell application "Play Sound"
			play theLoc repeat 4
		end tell
	end try
	
	tell application "Paintbrush"
		activate
	end tell
	delay 10
	
	-- Click on return for confirmation of painting size.
	try
		set see to "/Users/mac/clickcode/click -x 432 -y 242"
		log "see= " & see
		set results to do shell script see
	on error theError
		display dialog "click error 1 results for " & theError
	end try
	
	delay 3
	
	-- Draw 5 lines with pencil.  Assume pencil icon is clicked.
	repeat with i from 100 to 500 by 100
		try
			set see to "/Users/mac/clickcode/click -x 200 -y " & i
			log "see= " & see
			set results to do shell script see
		on error theError
			display dialog "click error 2 results for " & theError
		end try
		
		try
			set see to "/Users/mac/clickcode/clickdrag -x 579 -y 365"
			log "see= " & see
			set results to do shell script see
		on error theError
			display dialog "clickdrag error results for " & theError
		end try
	end repeat
end run


Edited on Feb 10, '10 12:10:08PM by rccharles


[ Reply to This | # ]
Move and click the mouse via code
Authored by: buzzwig on Feb 01, '12 04:35:53PM

Hi. I'd like to use a command line tool like this to drag by calling from AppleScript. But I don't know how to compile, and my attempts have been frustrating and have failed. Is there a compiled command line tool that I can use to click and drag? I use cliclick and mousetools, but they don't allow dragging, that I'm aware of. It would be wonderful to find a tool that provides this! Better yet, a tool that does the stuff the others do, but also drags, so I wouldn't have to have a multitude of little command line tools stacked up in my system. Whoever manages to accomplish this will be my hero!



[ Reply to This | # ]
Move and click the mouse via code
Authored by: sfurlani on Oct 29, '10 06:18:20AM

I know this is an old post, but how can I generate click events on remote machines??

Thanks...

about.me/stephenfurlani



[ Reply to This | # ]
Move and click the mouse via code
Authored by: TonyT on Jan 28, '11 09:50:56AM

Yes, I know, very old post, but here's a way to drag and drop with a python script:

#!/usr/bin/python
import sys
import time
from Quartz.CoreGraphics import * # imports all of the top-level symbols in the module

def mouseEvent(type, posx, posy):
theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)
CGEventPost(kCGHIDEventTap, theEvent)
def mousemove(posx,posy):
mouseEvent(kCGEventMouseMoved, posx,posy);
def mouseclickdn(posx,posy):
mouseEvent(kCGEventLeftMouseDown, posx,posy);
def mouseclickup(posx,posy):
mouseEvent(kCGEventLeftMouseUp, posx,posy);
def mousedrag(posx,posy):
mouseEvent(kCGEventLeftMouseDragged, posx,posy);

ourEvent = CGEventCreate(None);
currentpos=CGEventGetLocation(ourEvent); # Save current mouse position
mouseclickdn(60, 100);
mousedrag(60, 300);
mouseclickup(60, 300);
time.sleep(1);
mousemove(int(currentpos.x),int(currentpos.y)); # Restore mouse position

Edited on Jan 28, '11 09:56:30AM by TonyT



[ Reply to This | # ]
Move and click the mouse via code
Authored by: willwade on Apr 07, '13 07:12:17AM
Thanks Tony - if anyone else wants anything similar I've expanded on this here (But also keep an eye on PyUserInput which hopes to do more..)

[ Reply to This | # ]
Move and click the mouse via code
Authored by: JShoe on Apr 06, '11 08:14:08PM

Hey why won't this work on a PowerPC?



[ Reply to This | # ]
Move and click the mouse via code
Authored by: stretchphoto on Apr 09, '11 07:35:27PM

If anyone can help me I'd be very very grateful.

I'm trying to PREVENT the computer from receiving mouse events. I'm using an Intel Mac Mini to control a touchscreen in a kiosk. Once a viewer touches the screen to launch a video, I need to prevent the computer from receiving any other user input including screen touches (mouse events) until the video concludes.

I'm sort of desperate here, and this is for my master's capstone project.

Feel free to e-mail me, also stretch AT stretchphotography DOT COM

Thanks very much.



[ Reply to This | # ]
Move and click the mouse via code
Authored by: alego500 on Jun 11, '11 10:35:51AM

im also sorry to be stupid but where in the script do you place the x and y coordinates



[ Reply to This | # ]