Move a mouse with Apple script

Hello,


I need some apple script code to move a mouse to a certain location and then click. If possible could it record the location of the mouse, move it to the pre-assigned coordinates, click, and them move it back to where it was.


I am doing this so I can use multivid and Qlab to play different videos on different iOS devices. Multivid doesn't seem to let me have a cue list with different videos playing simultaneously.


I will have my iMac set up with 2 displays, one with Qlab and the other with multivid in full screen.


Thanks for the help

iMac, OS X Mavericks (10.9.1)

Posted on Jan 21, 2014 10:37 AM

Reply
12 replies

Jan 8, 2017 4:35 AM in response to willrobin

In general (in case others are searching the same topic), there's a simple solution to move the mouse cursor and make it click somewhere on the screen:


There's a scripting addition called "AppleScript Toolbox" (it's an osax). Get it at https://astoolbox.wordpress.com/. With that, you can simply write:


AST set mouse point location {10, 10} -- moves the mouse to the top left of the screen

AST clickat {20, 16} -- clicks into the menu bar and opens the Apple menu

Jan 21, 2014 11:07 AM in response to willrobin

You can also use this python script:

(this will click the Apple @ 30,12 -- adjust position as needed)


#!/usr/bin/python

from Quartz.CoreGraphics import * 
def mouseEvent(type, posx, posy):
        theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)
        CGEventPost(kCGHIDEventTap, theEvent)
def mousemove(posx,posy):
        mouseEvent(kCGEventMouseMoved, posx,posy);
def mouseclick(posx,posy):
        mouseEvent(kCGEventLeftMouseDown, posx,posy);
        mouseEvent(kCGEventLeftMouseUp, posx,posy);
ourEvent = CGEventCreate(None); 
currentpos=CGEventGetLocation(ourEvent);    # Save current mouse position
mouseclick(30, 12);
mousemove(int(currentpos.x),int(currentpos.y));  # Restore mouse position

Jan 22, 2014 10:44 PM in response to willrobin

Here's another code using rubycocoa you might try.


_rb_click({|:position|:{30, 10}, |:click|:1, |:restore|:true, |:prep|:false})

on _rb_click(desc)
    (*
        record desc : event descriptor record;
            full spec = {|:position|:pos, |:click|:k, |:button|:b, |:flags|:m, |:restore|:r, |:prep|:p}
            defaults   = {|:position|:{}, |:click|:0, |:button|:1, |:flags|:"", |:restore|:false, |:prep|:true}

            list pos : {x, y} or {}
                number x, y = x, y global coordinate of position
                {} denotes current location
            integer k : click count (0, 1, 2 or 3}; 0 denotes only to move mouse and exit
            integer b : button index (1 = left button, 2 = right button)
            string m : modifier flags; e.g. 'ck' = control + command
                a = capslock
                s = shift
                c = control
                o = option
                k = command
            boolean r : true to restore original mouse location, false otherwise
            boolean p : true to post preparatory left 1-click event to change UI context, false otherwise
        return list : {x, y} = mouse location at exit
    *)
    considering numeric strings
        if (system info)'s system version < "10.9" then
            set ruby to "/usr/bin/ruby"
        else
            set ruby to "/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby"
        end if
    end considering
    
    set defaults to {|:position|:{}, |:click|:0, |:button|:1, |:flags|:"", |:restore|:false, |:prep|:true}
    set {|:position|:pos, |:click|:k, |:button|:b, |:flags|:m, |:restore|:r, |:prep|:p} to desc & defaults
    if pos = {} then
        set {x, y} to {"%", "%"}
    else
        set {x, y} to pos
    end if
    if k is not in {0, 1, 2, 3} then error "invalid click count: " & k number 8000
    if b is not in {1, 2} then error "invalid button index: " & b number 8000
    if m = "" then set m to "%"
    
    do shell script ruby & " <<'EOF' - " & x & " " & y & " " & k & " " & b & " " & m & " " & r & " " & p & "
require 'osx/cocoa'
include OSX

if ARGV[0..1] == ['%', '%']
    pt = CGEventGetLocation(CGEventCreate(nil))        # current mouse location
else
    pt = CGPoint.new
    pt.x, pt.y = ARGV[0..1].map {|a| a.to_f}
end
clk, btn = ARGV[2..3].map {|a| a.to_i}
flg = ARGV[4]
res, prep = ARGV[5..6].map {|a| a == 'true'}

etype, mbtn = case btn
    when 1 then [KCGEventLeftMouseDown, KCGMouseButtonLeft]        # [1, 0]
    when 2 then [KCGEventRightMouseDown, KCGMouseButtonRight]    # [3, 1]
    when 3 then [KCGEventOtherMouseDown, KCGMouseButtonCenter]    # [25, 2]
    else raise ArgumentError, %[invalid mouse button: #{btn}]
end
mtable = {
    'a'    => KCGEventFlagMaskAlphaShift,
    's'    => KCGEventFlagMaskShift,
    'c'    => KCGEventFlagMaskControl,
    'o'    => KCGEventFlagMaskAlternate,
    'k'    => KCGEventFlagMaskCommand,
}
mf = flg.split(//).inject(0) { |mf, x| (m = mtable[x]) ? mf | m : mf }

src = CGEventSourceCreate(KCGEventSourceStateHIDSystemState)
tap = KCGHIDEventTap
pt0 = CGEventGetLocation(CGEventCreate(src))                    # current mouse location

# move mouse to target location
ev0 = CGEventCreateMouseEvent(src, KCGEventMouseMoved, pt, 0)    # move mouse
CGEventPost(tap, ev0)
if clk == 0
    puts pt.x, pt.y
    exit
end

# post preparatory left mouse click to change UI context (optional)
if prep
    ev1 = CGEventCreateMouseEvent(src, KCGEventLeftMouseDown, pt, KCGMouseButtonLeft)    # mouse left button down
    CGEventPost(tap, ev1)
    CGEventSetType(ev1, KCGEventLeftMouseUp)                    # mouse left button up
    CGEventPost(tap, ev1)
end

# post target mouse click(s) with given flags
ev = CGEventCreateMouseEvent(src, etype, pt, mbtn)                # mouse button down
CGEventSetFlags(ev, mf)                                            # set flags
CGEventSetIntegerValueField(ev, KCGMouseEventClickState, clk)    # set click count
CGEventPost(tap, ev)
CGEventSetType(ev, etype + 1)                                    # mouse button up
CGEventPost(tap, ev)

# restore mouse location (optional)
if res
    CGEventSetLocation(ev0, pt0)                                # restore mouse location
    CGEventPost(tap, ev0)
    puts pt0.x, pt0.y
    exit
end
puts pt.x, pt.y
EOF"
    set rr to paragraphs of result
    repeat with r in rr
        set r's contents to r as number
    end repeat
    return rr
end _rb_click

Jan 23, 2014 5:37 AM in response to Hiroto

Python requires less code, and easier to put into Applescript:


set x to 30
set y to 12
do shell script "python -c '
from Quartz.CoreGraphics import * 
def mouseEvent(type, posx, posy):
        theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)
        CGEventPost(kCGHIDEventTap, theEvent)
def mousemove(posx,posy):
        mouseEvent(kCGEventMouseMoved, posx,posy);
def mouseclick(posx,posy):
        mouseEvent(kCGEventLeftMouseDown, posx,posy);
        mouseEvent(kCGEventLeftMouseUp, posx,posy);
ourEvent = CGEventCreate(None); 
currentpos=CGEventGetLocation(ourEvent);  
mouseclick( " & x & "," & y & ");
mousemove(int(currentpos.x),int(currentpos.y));'"

Jan 23, 2014 10:41 AM in response to Hiroto

Hiroto wrote:


Hello


Your python code is shorter not because it is written in pyobjc but because it only handles the particular case of single-clicking the left mouse button without any modifier keys accompanied. I just wanted to provide a little more general-purpose handler. 🙂


All the best,

H


Yes, but that is all that the OP asked for, (I left out the extraneous code from the py script).

Anyway, looks like the OP didn't use any of our suggestions, maybe this discussion it will help someone else in the future.

Jan 23, 2014 11:54 AM in response to Tony T1

Thank you all for your help. I've just tried installing pyobjc as I need it for the Quartz module on python. However when I run the setup.py, I get this notice:


warning: no directories found matching 'Scripts'

warning: no directories found matching 'setup-lib'

warning: no directories found matching 'source-deps'

warning: no previously-included files matching '.DS_Store' found anywhere in distribution

warning: no previously-included files matching '*.pyc' found anywhere in distribution

warning: no previously-included files matching '*.so' found anywhere in distribution

Use '/usr/bin/clang' instead of 'gcc-4.2' as the compiler


What does this mean? and how can I fix it to install pyobjc?


Thank you

Jan 23, 2014 12:36 PM in response to willrobin

If you wish, here's a stripped down code only to move mouse, left single click and return to original location.


No aditional installation is required to use rubycocoa under 10.9. Also rubycocoa code is much faster than pyobjc code in my environment under 10.6.8.


Regards,

H


_left_1_click_and_return(30, 20)

on _left_1_click_and_return(x, y)
    considering numeric strings
        if (system info)'s system version < "10.9" then
            set ruby to "/usr/bin/ruby"
        else
            set ruby to "/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby"
        end if
    end considering
    do shell script ruby & " <<'EOF' - " & x & " " & y & "
require 'osx/cocoa'
include OSX
x, y = ARGV.map {|a| a.to_f}
src = CGEventSourceCreate(KCGEventSourceStateHIDSystemState)
tap = KCGHIDEventTap
pt0 = CGEventGetLocation(CGEventCreate(src))
pt1 = CGPointMake(x, y)
ev0 = CGEventCreateMouseEvent(src, KCGEventMouseMoved,    pt1, 0)
ev1 = CGEventCreateMouseEvent(src, KCGEventLeftMouseDown, pt1, KCGMouseButtonLeft)
ev2 = CGEventCreateMouseEvent(src, KCGEventLeftMouseUp,   pt1, KCGMouseButtonLeft)
ev3 = CGEventCreateMouseEvent(src, KCGEventMouseMoved,    pt0, 0)
CGEventPost(tap, ev0)
CGEventPost(tap, ev1)
CGEventPost(tap, ev2)
CGEventPost(tap, ev3)
EOF"
end _left_1_click_and_return

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Move a mouse with Apple script

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.