How to control the mouse in Mac using Python?

When I wanted to do it, I installed Jython and used the java.awt.Robot class. If you need to make a CPython script this is obviously not suitable, but when you the flexibility to choose anything it is a nice cross-platform solution.

import java.awt

robot = java.awt.Robot()

robot.mouseMove(x, y)
robot.mousePress(java.awt.event.InputEvent.BUTTON1_MASK)
robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK)

The easiest way? Compile this Cocoa app and pass it your mouse movements.

Here is the code:

// File:
// click.m
//
// Compile with:
// gcc -o click click.m -framework ApplicationServices -framework Foundation
//
// Usage:
// ./click -x pixels -y pixels
// At the given coordinates it will click and release.

#import <Foundation/Foundation.h>
#import <ApplicationServices/ApplicationServices.h>

int main(int argc, char **argv) {
  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
  NSUserDefaults *args = [NSUserDefaults standardUserDefaults];


  // grabs command line arguments -x and -y
  //
  int x = [args integerForKey:@"x"];
  int y = [args integerForKey:@"y"];

  // The data structure CGPoint represents a point in a two-dimensional
  // coordinate system.  Here, X and Y distance from upper left, in pixels.
  //
  CGPoint pt;
  pt.x = x;
  pt.y = y;


  // https://stackoverflow.com/questions/1483567/cgpostmouseevent-replacement-on-snow-leopard
  CGEventRef theEvent = CGEventCreateMouseEvent(NULL, kCGEventLeftMouseDown, pt, kCGMouseButtonLeft);
  CGEventSetType(theEvent, kCGEventLeftMouseDown);
  CGEventPost(kCGHIDEventTap, theEvent);
  CFRelease(theEvent);

  [pool release];
  return 0;
}

App called click that invokes CGPostMouseEvent from the CGRemoteOperation.h header file. It takes coordinates as command line arguments, moves the mouse to that position, then clicks and releases the mouse button.

Save the above code as click.m, open Terminal, and switch to the folder where you saved the source. Then compile the program by typing gcc -o click click.m -framework ApplicationServices -framework Foundation. Don’t be intimidated by needing to compile this as there are more comments than code. It is a very short program that does one simple task.

Another way? Import pyobjc to access some of the OSX framework and access the mouse that way. (see the code from the first example for ideas).

Just try this code:

#!/usr/bin/python

import objc

class ETMouse():    
    def setMousePosition(self, x, y):
        bndl = objc.loadBundle('CoreGraphics', globals(), 
                '/System/Library/Frameworks/ApplicationServices.framework')
        objc.loadBundleFunctions(bndl, globals(), 
                [('CGWarpMouseCursorPosition', 'v{CGPoint=ff}')])
        CGWarpMouseCursorPosition((x, y))

if __name__ == "__main__":
    et = ETMouse()
    et.setMousePosition(200, 200)

it works in OSX leopard 10.5.6

Try the code at this page. It defines a couple of functions, mousemove and mouseclick, which hook into Apple’s integration between Python and the platform’s Quartz libraries.

This code works on 10.6, and I’m using it on 10.7. The nice thing about this code is that it generates mouse events, which some solutions don’t. I use it to control BBC iPlayer by sending mouse events to known button positions in their Flash player (very brittle I know). The mouse move events, in particular, are required as otherwise the Flash player never hides the mouse cursor. Functions like CGWarpMouseCursorPosition will not do this.

from Quartz.CoreGraphics import CGEventCreateMouseEvent
from Quartz.CoreGraphics import CGEventPost
from Quartz.CoreGraphics import kCGEventMouseMoved
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseUp
from Quartz.CoreGraphics import kCGMouseButtonLeft
from Quartz.CoreGraphics import kCGHIDEventTap

def mouseEvent(type, posx, posy):
        theEvent = CGEventCreateMouseEvent(
                    None, 
                    type, 
                    (posx,posy), 
                    kCGMouseButtonLeft)
        CGEventPost(kCGHIDEventTap, theEvent)

def mousemove(posx,posy):
        mouseEvent(kCGEventMouseMoved, posx,posy);

def mouseclick(posx,posy):
        # uncomment this line if you want to force the mouse 
        # to MOVE to the click location first (I found it was not necessary).
        #mouseEvent(kCGEventMouseMoved, posx,posy);
        mouseEvent(kCGEventLeftMouseDown, posx,posy);
        mouseEvent(kCGEventLeftMouseUp, posx,posy);

Here is the code example from above page:

##############################################################
#               Python OSX MouseClick
#       (c) 2010 Alex Assouline, GeekOrgy.com
##############################################################
import sys
try:
        xclick=intsys.argv1
        yclick=intsys.argv2
        try:
                delay=intsys.argv3
        except:
                delay=0
except:
        print "USAGE mouseclick [int x] [int y] [optional delay in seconds]"
        exit
print "mouse click at ", xclick, ",", yclick," in ", delay, "seconds"
# you only want to import the following after passing the parameters check above, because importing takes time, about 1.5s
# (why so long!, these libs must be huge : anyone have a fix for this ?? please let me know.)
import time
from Quartz.CoreGraphics import CGEventCreateMouseEvent
from Quartz.CoreGraphics import CGEventPost
from Quartz.CoreGraphics import kCGEventMouseMoved
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseUp
from Quartz.CoreGraphics import kCGMouseButtonLeft
from Quartz.CoreGraphics import kCGHIDEventTap
def mouseEventtype, posx, posy:
        theEvent = CGEventCreateMouseEventNone, type, posx,posy, kCGMouseButtonLeft
        CGEventPostkCGHIDEventTap, theEvent
def mousemoveposx,posy:
        mouseEventkCGEventMouseMoved, posx,posy;
def mouseclickposx,posy:
        #mouseEvent(kCGEventMouseMoved, posx,posy); #uncomment this line if you want to force the mouse to MOVE to the click location first (i found it was not necesary).
        mouseEventkCGEventLeftMouseDown, posx,posy;
        mouseEventkCGEventLeftMouseUp, posx,posy;
time.sleepdelay;
mouseclickxclick, yclick;
print "done."

The python script from geekorgy.com is great except I ran into a few snags since I installed a newer version of python. So here are some tips to others who may be looking for a solution.

If you installed Python 2.7 on your Mac OS 10.6 you have a few options to get python to import from Quartz.CoreGraphics:

A) In the terminal, type python2.6 instead of just python before the path to the script

B) You can install PyObjC by doing the following:

1. Install easy_install from http://pypi.python.org/pypi/setuptools
2. In the terminal, type which python and copy the path up through 2.7

3. Then type easy_install –-prefix /Path/To/Python/Version pyobjc==2.3

   **ie. easy_install –-prefix /Library/Frameworks/Python.framework/Versions/2.7 pyobjc==2.3

4. Inside the script type import objc at the top

5. If easy_install doesn’t work the first time you might need to install the core first:

   **ie. easy_install --prefix /Library/Frameworks/Python.framework/Versions/2.7 pyobjc-core==2.3

C) You can reset your python path to the original Mac OS python:

• In the terminal, type: defaults write com.apple.versioner.python Version 2.6

***Also, a quick way to find out the (x,y) coordinates on the screen:

1. Press Command+Shift+4 (screen grab selection)
2. The cursor then shows the coordinates
3. Then hit Esc to get out of it.

Your best bet is to use the AutoPy package. It’s extremely simple to use, and cross-platform to boot.

To move the cursor to position (200,200):

import autopy
autopy.mouse.move(200,200)

The pynput library seems like the best currently maintained library. It allows you to control and monitor input devices.

Here is the example for controlling the mouse:

from pynput.mouse import Button, Controller

mouse = Controller()

# Read pointer position
print('The current pointer position is {0}'.format(
    mouse.position))

# Set pointer position
mouse.position = (10, 20)
print('Now we have moved it to {0}'.format(
    mouse.position))

# Move pointer relative to current position
mouse.move(5, -5)

# Press and release
mouse.press(Button.left)
mouse.release(Button.left)

# Double click; this is different from pressing and releasing
# twice on Mac OSX
mouse.click(Button.left, 2)

# Scroll two steps down
mouse.scroll(0, 2)

Use CoreGraphics from Quartz library, for example:

from Quartz.CoreGraphics import CGEventCreate
from Quartz.CoreGraphics import CGEventGetLocation
ourEvent = CGEventCreate(None);
currentpos = CGEventGetLocation(ourEvent);
mousemove(currentpos.x,currentpos.y)

^{Source: Tony comment at Geekorgy page.}

Here is the complete example using Quartz library:

#!/usr/bin/python
import sys
from AppKit import NSEvent
import Quartz

class Mouse():
    down = [Quartz.kCGEventLeftMouseDown, Quartz.kCGEventRightMouseDown, Quartz.kCGEventOtherMouseDown]
    up = [Quartz.kCGEventLeftMouseUp, Quartz.kCGEventRightMouseUp, Quartz.kCGEventOtherMouseUp]
    [LEFT, RIGHT, OTHER] = [0, 1, 2]

    def position(self):
        point = Quartz.CGEventGetLocation( Quartz.CGEventCreate(None) )
        return point.x, point.y

    def location(self):
        loc = NSEvent.mouseLocation()
        return loc.x, Quartz.CGDisplayPixelsHigh(0) - loc.y

    def move(self, x, y):
        moveEvent = Quartz.CGEventCreateMouseEvent(None, Quartz.kCGEventMouseMoved, (x, y), 0)
        Quartz.CGEventPost(Quartz.kCGHIDEventTap, moveEvent)

    def press(self, x, y, button=1):
        event = Quartz.CGEventCreateMouseEvent(None, Mouse.down[button], (x, y), button - 1)
        Quartz.CGEventPost(Quartz.kCGHIDEventTap, event)

    def release(self, x, y, button=1):
        event = Quartz.CGEventCreateMouseEvent(None, Mouse.up[button], (x, y), button - 1)
        Quartz.CGEventPost(Quartz.kCGHIDEventTap, event)

    def click(self, button=LEFT):
        x, y = self.position()
        self.press(x, y, button)
        self.release(x, y, button)

    def click_pos(self, x, y, button=LEFT):
        self.move(x, y)
        self.click(button)

    def to_relative(self, x, y):
        curr_pos = Quartz.CGEventGetLocation( Quartz.CGEventCreate(None) )
        x += current_position.x;
        y += current_position.y;
        return [x, y]

    def move_rel(self, x, y):
        [x, y] = to_relative(x, y)
        moveEvent = Quartz.CGEventCreateMouseEvent(None, Quartz.kCGEventMouseMoved, Quartz.CGPointMake(x, y), 0)
        Quartz.CGEventPost(Quartz.kCGHIDEventTap, moveEvent)

^{Above code is based on these original files: Mouse.pymouseUtils.py.}

Here is the demo code using above class:

# DEMO
if __name__ == '__main__':
    mouse = Mouse()
    if sys.platform == "darwin":
        print("Current mouse position: %d:%d" % mouse.position())
        print("Moving to 100:100...");
        mouse.move(100, 100)
        print("Clicking 200:200 position with using the right button...");
        mouse.click_pos(200, 200, mouse.RIGHT)
    elif sys.platform == "win32":
        print("Error: Platform not supported!")

You can combine both code blocks into one file, give execution permission and run it as a shell script.

The easiest way is using PyAutoGUI.

Installation:

pip install pyautogui

Examples:

• To get mouse position:

  >>> pyautogui.position()
  (187, 567)
  

• To move the mouse to a specific position:

  >>> pyautogui.moveTo(100,200)
  

• To trigger a mouse click:

  >>> pyautogui.click()
  

More details: PyAutoGUI