How Can I Control The Keyboard And Mouse With Python?

How can I control the mouse and keyboard in Python? The idea is to do the same as the Robot() class in Java. Be able to say: move the mouse from here to here, click there, write th

Solution 1:

I use dogtail (https://fedorahosted.org/dogtail/) to do such things, using this I have created an automated testing framework for my Linux(Ubuntu) app. That framework clicks buttons and types into text fields.

see the gedit example, https://fedorahosted.org/dogtail/browser/examples/gedit-test-utf8-procedural-api.py

So just use dogtail e.g

dogtail.rawinput.click(100, 100)

Solution 2:

I can advise you PyAutoGUI, it allows to full control Mouse and Keyboard and get Screenshots and even you can locate images within the screen (like: where is the button?), very useful to automate clicks dynamically. It works for Windows, macOS, and Linux.

For example:

>>>import pyautogui>>>screenWidth, screenHeight = pyautogui.size()>>>pyautogui.moveTo(screenWidth / 2, screenHeight / 2)

Check out the Introduction page.

Solution 3:

This totally works... on a Mac at least. This is for a click AND drag, etc.. but can be retrofitted accordingly.

#!/usr/bin/pythonimport sys
import time
from Quartz.CoreGraphics import * # imports all of the top-level symbols in the moduledefmouseEvent(type, posx, posy):
    theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)
    CGEventPost(kCGHIDEventTap, theEvent)
defmousemove(posx,posy):
    mouseEvent(kCGEventMouseMoved, posx,posy);
defmouseclickdn(posx,posy):
    mouseEvent(kCGEventLeftMouseDown, posx,posy);
defmouseclickup(posx,posy):
    mouseEvent(kCGEventLeftMouseUp, posx,posy);
defmousedrag(posx,posy):
    mouseEvent(kCGEventLeftMouseDragged, posx,posy);

ourEvent = CGEventCreate(None);
currentpos=CGEventGetLocation(ourEvent); # Save current mouse position
mouseclickdn(60, 100);
mousedrag(60, 300);
mouseclickup(60, 300);
time.sleep(1);
mousemove(int(currentpos.x),int(currentpos.y)); # Restore mouse position

Solution 4:

Here is an interessting Thread from Python Forum for you: Python Forum

Edit: There was also an interessting question on stackoverflow regarding mouse control...maybe it is a good starting point.. Mouse Control with Python

One of the Answers is refering to an Linux example...which heads you to an nice blog entry.

Solution 5:

for the mouse, I've found pymouse which seems to work (I haven't fully tried it, a small hack needed for the click, cf the issues)

for the keyboard, I'm not sure Xlib can do the job. I'm still looking on how to write something but you can catch key event as explained here or in C here using Xlib (but I don't know C).

here is an example working on gnome only (not good enough yet)

In pymouse, they have a nice way to make it work on the 3 different platform but needs to make 3 code...