Using xte to script your workflow

In the classic world of desktop automation, “macros” allow you to repeat a task easily. In general, xte is the best way of scripting this up on Linux.

First, you need to install it:

apt-get install xautomation
yum install xautomation

To use xte, you need to send it information via a ‘pipe’. The man page covers the key codes and commands in detail, but I’ll step through some basic examples below.

Examples

I’ve put together a couple of interesting examples to show some uses for this tool.

Example 1: Do a Google search

The example below uses xte to type “Hello world” into a text box.

Type the command, press enter, then quickly click on the text box:

sleep 1 && echo "Hello world" | xte

Example 2: Open a browser and search Wikipedia

To write a script which combines a few commands for xte, you could put it in a bash script. Remember to put a pause between commands so that the windowing system can catch up.

The script below will use Gnome Shells “overview mode” to launch Chromium, then open a new tab, and search Wikipedia for “cars”:

#!/bin/sh
xte << EOF
key Super_L
usleep 100000
str chromium
usleep 100000
key Return
sleep 1
keydown Control_L
key t
keyup Control_L
sleep 1
str https://en.wikipedia.org
usleep 100000
key Return
sleep 3
key Tab
usleep 100000
str cars
usleep 100000
key Return
EOF

Example 3: Draw a spiral in GIMP

When you script an operation, you can interact with any program using a set of rules. Below is a PHP script called spiral.php, which draws and labels a spiral in GIMP, switching the foreground & background colours at each step.

This requires an open GIMP window in the correct part of the screen:

The interaction in this case is quite simple for a computer, but would be tedious to do manually:

N: Open the Line tool
T: Open the Text tool
X: Swap foreground & background colours
Click & Drag: Draw a line

#!/usr/bin/env php
sleep 2
key N
usleep 100000
<?php
// Where the canvas is on-screen
$top_x = 150;
$top_y = 150;
$canvas_width = 400;

// Spiral properties
$pi = 3.14159265358979;
$centre_x = $top_x + $canvas_width / 2;
$centre_y = $top_y + $canvas_width / 2 + 100;
$spins = 5;

// Centre the mouse
echo "mousemove $centre_x $centre_y\n";
echo "usleep 100000\n";

// Draw a spiral
for($t = 0; $t < 1; $t += 0.005) {
	$angle = $spins * $t * 2 * $pi;
	$radius = $canvas_width / 2 * $t;
	$x = (int) ($centre_x + $radius * cos($angle));
	$y = (int) ($centre_y + $radius * sin($angle));
	echo "mousedown 1\n";
	echo "mousemove $x $y\n";
	echo "usleep 100000\n";
	echo "mouseup 1\n";
	echo "key X\n";
}

// Label the spiral
echo "mousemove $centre_x $top_y\n";
echo "key T\n";
echo "usleep 100000\n";
echo "mouseclick 1\n";
echo "usleep 100000\n";
echo "str Spiral\n";

After cropping, the spiral image on its own is:

Example 4: Forward emails

Google’s Gmail web interface has keyboard shortcuts for quick navigation. This example uses:

f: Forward an email
Tab: Move between fields
Ctrl+Enter: Send
j: Next email

Using these shortcuts, this script, forward.txt, will forward an email too bob@email.com and fred@example.com, then navigate to the next email:

sleep 1
key f
sleep 1
str bob@example.com
sleep 1
str fred@example.com
sleep 0.1
key Return
sleep 3
key Return
sleep 1
key Tab
sleep 0.1
keydown Control_L
key Return
keyup Control_L
sleep 3
key j
sleep 2

To send one email through xte, you could run this, and then click over to an open email in Gmail:

cat forward.txt | xte

To send the next 106 emails (e.g. in a label or search), you could instead type:

#!/bin/bash
for i in {1..106}; do echo $i; (cat forward.txt | xte); done

This is not fool-proof, so you would need to adjust the timing if your Internet connection is laggy.

When to use xte

Usually, a task which is supposed to be automated will have an API. For example, GIMP provides a python plugin interface, Gmail can be accessed via IMAP, and Google and Wikipedia searches can be done directly through HTTP. This is always the best way to do things.

However, the automation junkie should have xte in their toolkit, for situations where proper automation is not practical, such as when:

you don’t want to learn an API, to only use it for one day.
you’re doing a repetitive task in a program or website which is feature-poor.
you need to test some feature repeatedly under different circumstances.
you find a game which requires you to click fast.