Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Finding Control and Menu Items for use in AppleScript User Interface Scripting Apps
OS 10.7 Lion seems to have brought about a renewed interest in AppleScript User Interface Scripting. A big part of the difficulty in getting a GUI script to work is figuring out how exactly to address the controls in an applications window.

Apple's Accessibility Inspector.app, which comes with Xcode, is some help with this, but in my hands, it's always been awkward to use for figuring out what code I need to write to click a particular check box in say, a System Preferences Pane.

Here's a bit of AppleScript I wrote years ago that delivers up easy to understand and correct lists of an application's window and menu items in a format that may easily be cut and pasted directly into Scripts.
-- Entire Contents Demo - mini
-- BP ages ago or so

-- This'll get all the controls and structures associated with an App's window and menus
-- In a form which is easily pasteable into your own scripts
-- and show them in the result pane below.
--
-- Copy that into a text editor and change commas to returns to get an easily  readable list.
--
-- The script can take a long time if there are LOTS of window items, such as
-- in the "music" pane of iTunes. It may even time out if you have a huge iTunes library
-- The script'll process most App's UI structures in under a minute

set appname to "System Preferences" -------------------------- Set this to the App you want to look at

set winstuff to "defaultval"
set menustuff to "defaultval"

tell application appname
  activate
end tell

tell application "System Events"
  tell process appname
    set winstuff to entire contents of front window
    set menustuff to entire contents of menu bar 1
  end tell
end tell
--return winstuff & "rrrr" & menustuff -- comment this out to get just winstuff
return winstuff -- comment this out too to get just menustuff
--return menustuff 
Pointing the code at 'System Preferences' with the 'Desktop & Screens Saver' pane open will return, along with 128 other UI objects, this line describing the slider which controls the delay before the screen saver activates:
slider 1 of group 1 of tab group 1 of window "Desktop & Screen Saver" of application process "System Preferences" of application "System Events"
With the proper form of address in hand, it's a simple matter to construct an AppleScript that changes the delay before the screen saver is activated:
tell application "System Events"
  set value of slider 1 of group 1 of tab group 1 of window "Desktop & Screen Saver" of application process "System Preferences" to 180
end tell
The script will return information on whatever application window is frontmost, be it a document window, or an obscure dialog box.

I've looked for similar code online, and it doesn't seem to be commonly available.

[crarko adds: I tested some of this out; as mentioned, be prepared to do some massaging of the data that is returned (in AppleScript Editor's Results window) to make it readable.

Note: comments indicate this also works in versions before Lion.]
    •    
  • Currently 4.10 / 5
  You rated: 5 / 5 (10 votes cast)
 
[12,532 views]  

Finding Control and Menu Items for use in AppleScript User Interface Scripting | 15 comments | Create New Account
Click here to return to the 'Finding Control and Menu Items for use in AppleScript User Interface Scripting' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: benwiggy on Dec 12, '11 09:23:01AM

Can you use this technique to activate menu commands in applications that aren't very AppleScript aware?



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: ccjensen on Dec 12, '11 09:52:38AM

yes, all standard cocoa controls can be scripted using UI scripting (including the menu bar).



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: emendelson on Dec 12, '11 11:41:10AM

The Prefab UI Browser is a terrific commercial product that writes the AppleScript you need to add GUI scripting to scripts. It costs $$$ but there's a 30-day free trial period.

http://pfiddlesoft.com/uibrowser/

I am not connected with the vendor in any way whatever.



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: scottbayes on Dec 12, '11 11:58:25AM
Munging the output: To clean up the text of the output, you can use TextWrangler (or other free text processors, probably including TextEdit).

In TextWrangler, select the Find... menu item under the Search menu, and enter , in the Find: field and \n in the Replace: field. Make sure Grep is checked.

Hit the Next button to highlight the first ",", then Replace & Find. You'll see the first item split off making it easy(er) to read. Once you're happy the replacements are working as expected hit Replace All to get the rest.

This is not perfect: in Safari, for example, the History menu may contribute huge numbers of elements, some of which may contain "," internally, messing this up, but for most uses it works pretty well. You also may need to remove some "{" and "}" characters, etc.

I opened the resulting file in Numbers, making it easy to insert a descriptive column if I wish.

NB: MUNG is a "recursive" acronym. It stands for Mung Until No Good. It sometimes confuses civilians and Muggles.

[ Reply to This | # ]

Re: 10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: Uncle Asad on Dec 12, '11 02:08:58PM
In TextWrangler, select the Find... menu item under the Search menu, and enter , in the Find: field and \n in the Replace: field. Make sure Grep is checked.

Hit the Next button to highlight the first ",", then Replace & Find. You'll see the first item split off making it easy(er) to read. Once you're happy the replacements are working as expected hit Replace All to get the rest.

This is not perfect: in Safari, for example, the History menu may contribute huge numbers of elements, some of which may contain "," internally, messing this up, but for most uses it works pretty well.

Since all the items seem to end with 'of application "System Events", ', you can use something like '"System Events", ' as your find term and '"System Events"\n' as your replace term and avoid the "commas-in-item-titles" problem.

[ Reply to This | # ]
Re: 10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: Uncle Asad on Dec 12, '11 09:18:37PM

Er, and why in the world is this hint prefaced "10.7: "? The author's AppleScript works just fine on 10.5



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: StrawHousePig on Dec 12, '11 11:13:28PM

This is really not a 10.7 specific hint. Especially considering the author states he wrote this script years ago.



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: lazybaer on Dec 13, '11 03:53:30AM

is there really no way to get the result in a text file or array by applescript?



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: emendelson on Dec 13, '11 06:49:30AM

I asked about this script in Macscripter.net, and the experts there came up with two very useful ways of extending it to create a more manageable list of results:

http://macscripter.net/viewtopic.php?pid=146759



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: squalene on Dec 13, '11 07:01:32AM

As far as I can tell, the only way to get it is to write a GUI script that pulls the text out of the Applescript editor's results pane.
I've done that, but the result is UGLY, and not terribly reliable code.
See this thread at MacScriptor: http://macscripter.net/viewtopic.php?id=37674

Obviously the people at Apple took the time to write a very nice parser to generate the text in the first place. It's too bad they never added a getter for it in the editor's script dictionary.



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: bretperry on Dec 13, '11 11:04:22AM

You could add code at the end to put winstuff into a text file or an array, not sure how this would be helpful since you are just going to copy it and paste it elsewhere. Just open a text file for writing, set it's contents to winstuff, save and close the file.



[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: emendelson on Dec 15, '11 07:11:04AM

Nigel Garvey at MacScripter.net provided a script that generates text that can be be pasted into a text editor. In the script below, I added the tell block that pastes text into TextEdit:

property appName : "System Preferences" -- Your app name here.

on listToText(entireContents) -- (Handler specialised for lists of System Events references.)
	try
		|| of entireContents -- Deliberate error.
	on error stuff -- Get the error message
	end try
	
	-- Parse the message.
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to {"{", "}"} -- Snow Leopard or later.
	set stuff to text from text item 2 to text item -2 of stuff
	set AppleScript's text item delimiters to "\"System Events\", "
	set stuff to stuff's text items
	set AppleScript's text item delimiters to "\"System Events\"" & linefeed
	set stuff to stuff as text
	set AppleScript's text item delimiters to astid
	
	return stuff
end listToText

on main()
	tell application "System Events"
		tell application process appName
			set frontmost to true
			set {windowExists, menuExists} to {front window exists, menu bar 1 exists}
			set {winstuff, menustuff} to {missing value, missing value}
			if (windowExists) then set winstuff to my listToText(entire contents of front window)
			if (menuExists) then set menustuff to my listToText(entire contents of menu bar 1)
		end tell
	end tell
	
	-- the following block is the only part of this code that is NOT by Nigel Garvey
	tell application "TextEdit"
		activate
		make new document at the front
		set the text of the front document to winstuff & return & "-----" & return & menustuff
	end tell
	
	-- 	return {winstuff:winstuff, menustuff:menustuff}
end main

main()


[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: emendelson on Dec 15, '11 11:10:10AM

And here's a version of Nigel Garvey's script that prompts you to select an application to analyze. I only added the trivial stuff. The intelligent part is by Nigel Garvey.

on run {}
	-- begin lines NOT by Nigel Garvey
	set appAlias to choose file with prompt "Select an application:"
	set appName to name of (info for appAlias)
	set appName to (the reverse of every character in appName) as string
	if ((text 1 thru 4 of appName) as string) is not equal to "ppa." then
		tell me to activate
		display dialog "Please select an application only." buttons {"OK"}
		error number -128
	end if
	set appName to (text 5 thru -1 of appName)
	set appName to (the reverse of every character in appName) as string
	set appPosix to quoted form of POSIX path of appAlias
	do shell script "open " & appPosix
	tell me to activate
	display dialog "Display the window or tab in " & appName & " that you want to analyze and click OK." buttons {"OK", "Cancel"} default button 1
	if button returned of result is "Cancel" then
		error number -128
	end if
	-- end lines NOT by Nigel Garvey
	
	tell application "System Events"
		tell application process appName
			set frontmost to true
			set {windowExists, menuExists} to {front window exists, menu bar 1 exists}
			set {winstuff, menustuff} to {missing value, missing value}
			if (windowExists) then set winstuff to my listToText(entire contents of front window)
			if (menuExists) then set menustuff to my listToText(entire contents of menu bar 1)
		end tell
	end tell
	
	-- the following block is NOT by Nigel Garvey
	tell application "TextEdit"
		activate
		make new document at the front
		set the text of the front document to winstuff & return & "-----" & return & menustuff
	end tell
	-- 	return {winstuff:winstuff, menustuff:menustuff}
end run

on listToText(entireContents) -- (Handler specialised for lists of System Events references.)
	try
		|| of entireContents -- Deliberate error.
	on error stuff -- Get the error message
	end try
	
	-- Parse the message.
	set astid to AppleScript's text item delimiters
	set AppleScript's text item delimiters to {"{", "}"} -- Snow Leopard or later.
	set stuff to text from text item 2 to text item -2 of stuff
	set AppleScript's text item delimiters to "\"System Events\", "
	set stuff to stuff's text items
	set AppleScript's text item delimiters to "\"System Events\"" & linefeed
	set stuff to stuff as text
	set AppleScript's text item delimiters to astid
	
	return stuff
end listToText


[ Reply to This | # ]
10.7: Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: emendelson on Dec 15, '11 12:52:03PM

And there's now an even better version at or near the end of this thread:

http://macscripter.net/viewtopic.php?pid=146849#p146849



[ Reply to This | # ]
Finding Control and Menu Items for use in AppleScript User Interface Scripting
Authored by: _Gekko_ on Dec 15, '11 11:55:40PM

simple, clever, etc Really a great hint!



[ Reply to This | # ]