Help! Can AppleScript access context menu?

Question

Help! Can AppleScript access context menu?

Hello Apple fans,

I am a long time Mac user, but very new to AppleScript. I've been working through some online tutorials, learning some of the basics, and I've made some progress. However, I am completely stumped with what I'm trying to accomplish. Any help that you scripting experts can provide would be most appreciated.

This is what I'm trying to do: I would like to make an AppleScript that can access the right-click (or Control-click) context (or contextual) menu in Safari.

For example, if there is an image on the page in Safari, if you right-click it, a context menu pops up with various commands: "Open Link in New Tab", "Copy Link", "Save Image As...", etc. I'm using "Open Image in New Tab" just to test whether AppleScript is indeed accessing this menu. I'm hoping I can substitute another command once I prove that the script actually works.

This is what I've done so far:

I navigate to a web page with an image.
Using the application "UI Browser", I can find what the pathway to access that image using UI Scripting.
I can copy and paste that pathway into my AppleScript Editor.
I make some minor adjustments to the actions I'm hoping to perform.

This is what "UI Browser" gives me as a typical pathway to an onscreen image:

application "Safari"

standard window "NAME OF WEBPAGE" (window 1)

group (group 3)

group (group 1)

scroll area (scroll area 1)

HTML content (UI element 1)

group (group 6)

link (UI element 1)

image (image 1)

ACTIONS:

press

show menu

After getting the above pathway, I then right-click the image and choose "Open Image in New Tab" and this is what "UI Browser" displays:

ELEMENT:

Role: menu item

Title: "Open Image in New Tab"

Description:

Help:

Application: Safari

ELEMENT PATH (FROM LEAF ELEMENT):

menu item "Open Image in New Tab" (menu item 12)

menu (menu -9223372036854775808)

[MISMATCH-different children] application "Safari"

ACTIONS [2]:

cancel

press

So this is what my AppleScript looks like:

tell application "System Events"

tell process "Safari"

tell image 1 of UI element 1 of group 6 of UI element 1 of scroll area 1 of group 1 of group 1 of group 3 of window 1

click

end tell

When I run this script, it succesfully clicks the image (the image automatically changes when clicked, so I know it works). However, I cannot figure out how to access the context menu, as if I were right-clicking the image. That is what is stumping me: the context menu, and executing commands in it.

I've tried "perform action "ShowMenu"", or "perform action "Show Menu"". I've tried "click menu item 12 of menu 1", or "tell menu item 12 of menu 1" "press". I've tried "click "Open Image in New Tab"", and variations such as that. So far, I just can't figure it out!

If you've made it this far reading my question, thank you! If you can offer any assistance with this, I would sincerely appreciate it.

Thank you very much,

Melissa

iMac (27-inch Late 2009), Mac OS X (10.6.8)

Posted on Feb 21, 2012 8:40 PM

Reply

Answer 1

red_menace

Level 6

17,072 points

Feb 22, 2012 12:18 AM in response to Melissa Atwell

Once you click to get the contextual menu (and it appears), you need to go through the hierarchy of that menu. Since the contextual menu will change depending on what is under the mouse pointer, and items on a web page will change depending on its layout, what exactly are you wanting to do?

Reply

Answer 2

Feb 22, 2012 1:43 AM in response to red_menace

Thank you, red_menace, for your reply.

My goal is to have a script and/or script + Automator combination that can download images from some online galleries. For example, one gallery is set up so that if you click an image, it wil automatically advance to the next. I've figured out how to script and/or automate that. What I want to do is perform the Safari right-click "Save Image to <download location>" with AppleScript. I've already tried using Automator's "Get Image URLs" but that works intermittently and is basically useless.

At this point, I've figured out how to do everything I need, using Automator with AppleScripts inserted in the workflow ... EXCEPT the right-click commands. That absolutely mystifies me!

Thanks again for replying. I look forward to your response.

Reply

Answer 3

Pierre L.

Level 5

4,641 points

Feb 22, 2012 3:45 AM in response to Melissa Atwell

You might try Cliclick!

Here's an example:

open location "http://www.apple.com/"

delay 1

tell application "Safari" to activate

tell application "System Events" to tell process "Safari"

set theImage to image 1 of UI element 3 of UI element 1 of scroll area 1 of group 1 of group 1 of group 2 of window "Apple"

set {x, y} to position of theImage

do shell script "/usr/local/bin/cliclick c" & x & space & y

end tell

Message was edited by: Pierre L.

Reply

Answer 4

Pierre L.

Level 5

4,641 points

Feb 22, 2012 4:58 AM in response to Melissa Atwell

Using the same web page as in my previous post, here's a much simpler method which also seems to work:

open location "http://www.apple.com/"

delay 1

tell application "System Events" to tell process "Safari"

set theImage to image 1 of UI element 3 of UI element 1 of scroll area 1 of group 1 of group 1 of group 2 of window "Apple"

tell theImage to perform action "AXShowMenu"

end tell

(You must write "AXShowMenu" and not just "ShowMenu")

Message was edited by: Pierre L.

Reply

Answer 5

Feb 22, 2012 4:18 PM in response to Pierre L.

Thank you, Pierre L., that was very helpful! That successfully opens the right-click menu. So now, my question is how to activate a specific command in that menu?

When examining the image I want to click with "UI Browser", this is what I get when I right-click, then hover over "Open Image in New Tab":

ELEMENT:

Role: menu item

Title: "Open Image in New Tab"

Description:

Help:

Application: Safari

ELEMENT PATH (FROM LEAF ELEMENT):

menu item "Open Image in New Tab" (menu item 12)

menu (menu -9223372036854775808)

[MISMATCH-different children] application "Safari"

ACTIONS [2]:

cancel

press

I've tried a bunch of variations, but once again, I'm just not experienced enough to figure this out. If you can help me with this last bit of scripting, I'll be done!

Thank you,

Melissa

EDIT: Oh, and this is what Apple's "Accessibility Inspector" shows:

<AXApplication: “Safari”>

<AXMenuItem: “Open Image in New Tab”>

Attributes:

AXRole: “AXMenuItem”

AXRoleDescription: “menu item”

AXParent: “<AXMenu>”

AXEnabled: “true”

AXPosition: “x=926 y=892”

AXSize: “w=242 h=19”

AXTitle: “Open Image in New Tab”

AXHelp: “(null)”

AXSelected (W): “true”

AXMenuItemCmdChar: “(null)”

AXMenuItemCmdVirtualKey: “(null)”

AXMenuItemCmdGlyph: “(null)”

AXMenuItemCmdModifiers: “0”

AXMenuItemMarkChar: “(null)”

AXMenuItemPrimaryUIElement: “(null)”

Actions:

AXCancel - cancel

AXPress - press

Thanks!

Message was edited by: Melissa Atwell

Reply

Answer 6

Feb 22, 2012 4:56 PM in response to Melissa Atwell

I think I've stumbled onto something that works, although it's not incredibly elegant. Once the contextual menu has popped open, it can be navigated with keyboard commands. For example, pressing the letter "s" gets you in the vicinity of "Save Image As...". After that, pressing "Return" executes the command. Up and Down arrows can be used to select other menu items that are close by.

So, for now at least, it looks like I can accomplish what I've been wanting to do. Of course, it would be great to have a nice, clean, elegant script that executes the exact command that you want. If any of you do happen to figure this out, please please please let me know.

Thank you!

Melissa

Reply

Answer 7

Pierre L.

Level 5

4,641 points

Feb 22, 2012 5:59 PM in response to Melissa Atwell

For example, pressing the letter "s" gets you in the vicinity of "Save Image As...". After that, pressing "Return" executes the command. Up and Down arrows can be used to select other menu items that are close by.

Try this:

tell theImage to perform action "AXShowMenu"

keystroke "Open Image in New Tab" & return

Reply

Answer 8

Feb 22, 2012 7:49 PM in response to Pierre L.

Hi Pierre,

Thank you so much for your ongoing help. I tried your suggestion, but there is a bigger problem to this method.

As red_menace pointed out:

"...the contextual menu will change depending on what is under the mouse pointer, and items on a web page will change depending on its layout..."

That is exactly the problem. Every time I "click" the image, the next web page has a slightly different path to the image. In other words, the "image 1 of UI element 3 of UI element 1...", etc. changes slightly each time. The result is that AppleScript regularly encounters "Invalid Index" errors. It makes using that method unusable.

However ... you definitely provided me with a wonderful alternative: CliClick.

What a wonderful little utility! That was the trick. My problem is solved, and I cannot thank you enough for your help (as well as the creator of "CliClick")!

Here is the script that I ended up with. It works great 99% of the time. (There are a few instances where it doesn't "click" properly, but it's rare.)

Here it is (please don't judge too harshly):

display dialog "To hear audio confirmations, please make sure the speakers are on, and volume turned up." buttons {"OK"} default button 1

delay 1

display dialog "How many pages would you like to save?" default answer "1" buttons {"Cancel", "GO"} default button 2

copy the result as list to {text_returned}

set x to text_returned

tell application "Safari"

activate

end tell

delay 1

repeat x times

do shell script "/usr/local/bin/cliclick c1000 550"

delay 0.5

tell application "System Events" to tell process "Safari"

keystroke "s"

delay 0.5

key code 126

delay 0.5

key code 36

delay 1

end tell

do shell script "/usr/local/bin/cliclick 1000 550"

delay 3

end repeat

delay 0.5

activateapplication "Finder"

tell application "Finder"

reveal "<my download destination here>"

end tell

delay 0.5

beep

delay 0.5

say "Download Complete."

Thanks again to both red_menace, and Pierre L. for your help with this. I sincerely appreciate it!

Best wishes,

Melissa

Reply