Getting a USB receipt printer working on Linux

In this post, I’ll step through how to get a thermal receipt printer with USB interface appearing on Linux. The aim of this is to be able to point a driver such as escpos-php at the device. The printer used here is an Epson TM-T20, which is very common in point-of-sale environments.

I have previously written quite a bit about how to use thermal receipt printer protocols, but the previous printer I covered had only a network interface, not USB like this one:

2015-03-printer-back
2015-03-printer-top

The directions below are for Debian, but could be adapted for any other Linux.

Find the device file

Plug in your printer, and check that usblp sees it:

dmesg
[12724.994550] usb 8-4: new full-speed USB device number 5 using ohci-pci
[12725.168956] usb 8-4: New USB device found, idVendor=04b8, idProduct=0e03
[12725.168963] usb 8-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[12725.168968] usb 8-4: Product: TM-T20
[12725.168971] usb 8-4: Manufacturer: EPSON
[12725.168975] usb 8-4: SerialNumber: ....
[12725.175114] usblp 8-4:1.0: usblp1: USB Bidirectional printer dev 5 if 0 alt 0 proto 2 vid 0x04B8 pid 0x0E03

This kernel module makes your printer visible as a device file, so that it can be accessed in the old-fashioned way. Find the new device file under /dev/usb:

ls /dev/usb

In my case, this was /dev/usb/lp1. The next step is to see if you can write to it:

echo "Hello" >> /dev/usb/lp1

Chances are, you will get a permission denied error at this point, so find out what group the printer is in:

stat /dev/usb/lp1

Which will show output something like:

File: ‘/dev/usb/lp1’
  Size: 0         	Blocks: 0          IO Block: 4096   character special file
Device: 5h/5d	Inode: 220997      Links: 1     Device type: b4,1
Access: (0660/crw-rw----)  Uid: (    0/    root)   Gid: (    7/      lp)
...

This file is owned by group lp (“line printer”). If your username was bob, you would add yourself to this group using:

sudo usermod -a -G lp bob

If you plan to build a web-based point-of-sale system with this, then also add the www-data user to that group.

Now log out and back in, and the previous test should now be working:

echo "Hello" >> /dev/usb/lp1

Troubleshooting: Check usblp

If these steps don’t work, then your computer ether doesn’t have, or isn’t using usblp You’ll need to check a few things:

  • Install a different linux-image if the driver is not on your computer at all.
  • modprobe or insmod usblp
  • blacklist a vendor driver which has claimed the interface.
    • run lsusb -v and usb-devices (look for driver=)

Printing something useful

As a duplicated section from my earlier post, the printer uses ESC/POS, which means it accepts plaintext with some special commands for formatting.

A simple receipt-generator, foo.php, might look like this:

<?php
/* ASCII constants */
const ESC = "\x1b";
const GS="\x1d";
const NUL="\x00";

/* Output an example receipt */
echo ESC."@"; // Reset to defaults
echo ESC."E".chr(1); // Bold
echo "FOO CORP Ltd.\n"; // Company
echo ESC."E".chr(0); // Not Bold
echo ESC."d".chr(1); // Blank line
echo "Receipt for whatever\n"; // Print text
echo ESC."d".chr(4); // 4 Blank lines

/* Bar-code at the end */
echo ESC."a".chr(1); // Centered printing
echo GS."k".chr(4)."987654321".NUL; // Print barcode
echo ESC."d".chr(1); // Blank line
echo "987654321\n"; // Print number
echo GS."V\x41".chr(3); // Cut
exit(0);

And you would send it to the printer like this:

php foo.php > /dev/usb/lp1

Scaling this up

The codes are quite tricky to work with manually, which is why I put together the escpos-php driver. You can find it at:

The above example would be written using escpos-php as:

<?php
require __DIR__ . '/autoload.php';
use Mike42\Escpos\Printer;
use Mike42\Escpos\PrintConnectors\FilePrintConnector;
$connector = new FilePrintConnector("/dev/usb/lp1");
$printer = new Printer($connector);

/* Print some bold text */
$printer -> setEmphasis(true);
$printer -> text("FOO CORP Ltd.\n");
$printer -> setEmphasis(false);
$printer -> feed();
$printer -> text("Receipt for whatever\n");
$printer -> feed(4);

/* Bar-code at the end */
$printer -> setJustification(Printer::JUSTIFY_CENTER);
$printer -> barcode("987654321");
$printer -> cut();
?>

This would be sent to the printer by loading it from the web, or running the script on the command-line:

php foo2.php

Two ways to back up your Google Apps account

If you use Gmail or hosted Google Apps, you might be interested in taking a backup of your data, such as emails, Drive documents, and calendar entries. Thankfully, you can usually export copy of your account data using Google Takeout.

If your hosted Apps account has Takeout disabled, then you can do a backup, it simply has a few extra steps.

Option 1: Google Takeout

This method is nice and simple. Simply go to the Data tools – Download your data page, and select which services you want to export:

2015-01-google-takeout

It can be a bit eye-opening to see the amount of data Google has on you (Files, conversations, location history, etc). At this point, click through to “Prepare Download”. Depending on the size of your account, this may take as a coffee break, a few hours, or even an entire day.

2015-02-google-takeout-prepare

If you check the box for it, you’ll get an email like this when your Download completes:

2015-02-google-archive

And this lets you fetch a single file:

2015-01-download

The .zip file contains a series of folders, one for each service. The defaults seem to be:

Mail
A unix mbox file
Calendar
One iCal file for each calendar
Contacts
One vCard file for each group.
Drive
Exports as PDF, docx, xlsx

Option 2: Export data from each service

Sometimes, Google Takeout isn’t an option.

2015-01-google-takeout-disabled

Luckily, most Google services have some sort of data export built in. This means, if you have a new contact manager, or want to include your Drive in your PC backup, it’s still possible.

The export formats in these examples should match the Google Takeout defaults. Tab through each service t see how to export it:

If you are not a power user, then I would suggest setting up a copy of Mozilla Thunderbird via IMAP, and regularly using it for your email. This is a simple way to keep a clone of your inbox on your desktop computer, so that it can be included in backups.

If you are more tech-savvy, then the rest of this section will focus on helping you generate an mbox file containing a full backup of your email, the same format as Takeout uses. The best tool for that is a Linux program called getmail.

On Debian or Ubuntu Linux, issue this command to install getmail:

sudo apt-get install getmail4

For other package managers, see these directions.

First, you need to enable IMAP for your account, see Google’s article: Get started with IMAP and POP3, for the steps.

Now create a file at ~/.getmail/getmailrc, and configure it to read your email account via IMAP/SSL.

[retriever]
type = SimpleIMAPSSLRetriever
server = imap.gmail.com
port = 993
username = bob@mail.example.com
password = ....

[destination]
type = Mboxrd
path = ~/inbox

[options]
verbose = 1
getmail

After some time, you will end up with a large mbox file at ~/inbox, containing all of your mail.

If, for some reason, you need to use POP3 instead, then see this article on Gmail backup

Go to your contacts, and find a group. Check the box next to each name, and then find the Export button:

2015-02-contacts-export

Select the vCard format here, as it’s the same format which Takeout would have used:

2015-02-contacts-export-group-vcard

Google provides a share-able iCal link, which you can download once, but it is only available if your calendar is public.

So, if your calendar isn’t too sensitive, click “Share” and make the calendar public:2015-02-calendar-01-share

2015-02-calendar-02-public

Go to “Calendar Settings”, find the iCal link. It may take a few minutes for the link to start working, but once it does, download it, and then turn off public sharing.

There is a small risk that somebody else loads your calendar while its public, so if this concerns you, then save the events individually.

Exporting from Google Drive is nice and simple. Select all of your files (Shift+Click):

2015-02-drive-select

And then find the Download button:

2015-02-drive-download

If you apply this to your whole drive, it may take a while, so you may wish to download it in parts if your Internet can be unreliable.

Know how to export a different service? Send it in and I’ll add it to the list.

How do I use these files?

Google Drive’s files are exported in familiar formats. If you haven’t used an mbox, vCard or ics file before, then you will need to find a program which can read these for you.

Google’s support answer “Download your data: Per-service information” contains a list of files types which you’ll run into during this process, and suggests programs which can import them.

Howto: Tethered photo capture on Linux

Tethered capture diagram

Have you ever wondered how professionals get photos to pop up on their computer as they snap them? Most higher-end cameras have mini USB connection, and software is available to retrieve images as they are taken.

Rather than use a GUI app, in this post I’ll use a command-line program called gphoto2 to drop the images into a folder. With large thumbnails set in your file browser, a desktop program would be redundant.

First, you need to install the program. Depending on your system, one of the following commands should do the trick:

apt-get install gphoto2
yum install gphoto2

Now, plug in the camera. The command to do a “tethered capture” is:

gphoto2 --capture-tethered

Unfortunately, in most desktop environments, your file manager will mount the camera automatically. If this is the case, then the command will give you an error:

mike@mikebox:~$ gphoto2 --capture-tethered
Waiting for events from camera. Press Ctrl-C to abort.                         

*** Error ***              
An error occurred in the io-library ('Could not claim the USB device'): Could not claim interface 0 (Device or resource busy). Make sure no other program (gvfs-gphoto2-volume-monitor) or kernel module (such as sdc2xx, stv680, spca50x) is using the device and you have read/write access to the device.
*** Error (-53: 'Could not claim the USB device') ***       

For debugging messages, please use the --debug option.
Debugging messages may help finding a solution to your problem.
If you intend to send any error or debug messages to the gphoto
developer mailing list , please run
gphoto2 as follows:

    env LANG=C gphoto2 --debug --debug-logfile=my-logfile.txt --capture-tethered

Please make sure there is sufficient quoting around the arguments.

Simply find the camera and unmount it using the eject button:

Eject the camera storage

Now when you type the command, it will block until a photo is taken, and then show you the name of the photo:

mike@mikebox:~$ gphoto2 --capture-tethered
Waiting for events from camera. Press Ctrl-C to abort.                         
UNKNOWN PTP Property 5007 changed
...
Downloading 'DSC_0236.JPG' from folder '/store_00010001/DCIM/100NCD90'...
Saving file as DSC_0236.JPG                                                    
Deleting 'DSC_0236.JPG' from folder '/store_00010001/DCIM/100NCD90'...

Each of the photos is loaded into the working directory after you release the shutter, so you simply close gphoto2 when you’re done — no manual downloading or SD card required.

Extending this method

Ok, so now that I’ve covered this basic use case, the real reason I suggest gphoto2 is that it will let you script just about anything to do with your camera.

Just typing gphoto2 --help shows that it can let you trigger a photo or video on a timer, download and delete folders from the camera, or hook up programs via a pipe for processing the files in realtime.

Endless possibilities.

Locking down your VOIP setup with a SIP Threat Manager

If you run a Voice over IP network which is available from the Internet, then it’s quite important to lock it down properly, so that it isn’t hijacked for relaying spam calls at your expense.

This article will cover the steps you need to deploy the SIP Threat Manager from Allo, which you can think of as a security-focused SIP proxy & firewall.

Topology

For example, you might have SIP clients both on-site with your server, and also allow people with their own devices to connect directly from home. One of the simplest ways to harden this setup is to add a specialised SIP router between your server and the Internet, to filter connections and log security-relevant events:

VOIP network with STM

VOIP network with STM

This post will show you how to migrate your VOIP network to this more secure topology using Allo’s SIP Threat Manager..

The Allo STM Box

Allo STM front
Allo STM back

The STM itself is only small box, with two 100 Mbit/s Ethernet ports, and two USB ports. It is USB-powered, so one of these ports is for powering the box, and the other is for connecting external storage for log files. It is not power-hungry, and I was able to run it from a laptop USB port without any issues. Other than this, it’s externally a typical embedded network device: it has indicator lights, a power switch, a factory reset button, and a console port.

Allo STM internal

Internally, it runs a MIPS processor, which appears to host Snort on embedded Linux.

Initial Setup

Although the box advertises that it will work out-of-the-box, I found that it was easier to configure the box to match my network, than to re-work my network around the box. This section will simply show you how to get logged in and change the box’s IP address.

First up, I took a look at the console, which is accessible at a baudrate of 38400. It’s far from the most functional CLI around, so I only made use of the factoryreset function to get a clean slate. It showed a few of the open source packages running, such the dropbear sshd, lighttpd and crond among other familiar programs:

Please press Enter to activate this console. 
Starting pid 942, console /dev/ttyS2: '/usr/bin/maincli'
Shield STM Appliance Appliance
shield> factoryreset

Will output 1024 bit rsa secret key to '/etc/dropbear/dropbear_rsa_host_key'
Generating key, this may take a while...
Public key portion is:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgwCKBcVlWK+UiiELbg2CNfOt9rNmj51dmyz7d10MgRfAk9XU9x+kmlMueCFEBMTchsaywigLw0yFqeMZ
Fingerprint: md5 50:5b:c2:64:d4:87:f8:86:ab:c6:e1:59:e4:16:c2:cf
Generating a 1024 bit RSA private key
...++++++
...................++++++
writing new private key to '/etc/lighttpd/webserver.pem'
-----
ip: RTNETLINK answers: No such process
Jan  1 00:40:41 crond[875]: crond 2.3.2 dillon, started, log level 8

mount: mounting /dev/sda1 on /cf/disk failed

The CLI command show ip confirmed that the default IP of the box is 192.168.100.1, netmask 255.255.255.0. From a Linux laptop, you can change IP to something nearby and confirm that you can see the device with these commands:

# ifconfig wlan0 192.168.100.2 netmask 255.255.255.0
# arp-scan -l
Interface: wlan0, datalink type: EN10MB (Ethernet)
Starting arp-scan 1.8.1 with 256 hosts (http://www.nta-monitor.com/tools/arp-scan/)
192.168.100.1	00:17:f7:00:9b:0a	CEM Solutions Pvt Ltd

The STM is then accessible via the web address https://192.168.100.1, with the default credentials admin / admin.

STM login

The IP setup is located under Device → General Setup. Change this to DHCP or a spare address on your network:

STM IP config

Configuration

Now that you can access the STM from any device, your first task is to change the admin password. The button for this is in the top-right:

2015-01-stm-screen21

The STM only allows one session at a time- whilst it’s a good idea not to log in twice, this was a surprising limitation. At the STM does not act as its own SIP endpoint, my server was already able to contact the Internet through it at this point.

I quickly screen captured the available settings so that you can click through them. Some of these are SIP-specific, and others of which are general firewall features. One of the more interesting features which you can’t set up with iptables is location-based IP filtering. This could, for example, block problematic SIP calls coming from fraud hotspots in areas where your organisation doesn’t operate.

So once the network is set up on the STM, no changes need to be applied to your SIP server, other than its gateway or IP.

Notes

Whilst this box works as it’s supposed to, I found it to have an un-polished user experience.

  • The network interface labels on the box had a label over them with the opposite information.
  • The box is closed on port 80: It doesn’t reply to HTTP requests, even to redirect them to HTTPS.
  • The command prompt wasn’t as useful as other network devices.
  • I couldn’t get SSH login or NTP to work, although I didn’t investigate these in great detail.
  • The LAN interface (but not the WAN interface) did not light up when connected to a Gigabit POE network, but did work on a 100 megabit network.

However, there are some positives: The 100 Mbit/s interfaces are more than sufficient for voice traffic, the configuration was simple, and USB is a good choice of power supply for equipment which can be connected directly to a server.

Do you really need another box?

This depends on your setup. If your VOIP server doesn’t speak to the Internet, then this box wont fit into your topology.

If it only sees the outside world via an ISP-run SIP trunk, then this type of security is probably not necessary either. Security measures you would use instead are:

  • Use firewall rules to restrict connections so that only the SIP trunk can speak to your VOIP server.
  • Configure your VOIP server to “stay on the line” for calls (directmedia=no in Asterisk) so that the phones do not speak directly to the trunk, and disallow registration from the Internet.

If your VOIP server accepts connections from the public internet throgh SIP, then some sort of separate, SIP-aware firewall or proxy is highly advisable.

Acknowledgement

Thanks to Allo (allo.com) for sending in the box which is used for this example setup.

Using xte to script your workflow

In the classic world of desktop automation, “macros” allow you to repeat a task easily. In general, xte is the best way of scripting this up on Linux.

First, you need to install it:

apt-get install xautomation
yum install xautomation

To use xte, you need to send it information via a ‘pipe’. The man page covers the key codes and commands in detail, but I’ll step through some basic examples below.

Example 1: Do a Google search

The example below uses xte to type “Hello world” into a text box.

Minimal xte example - Typing text in a box

Type the command, press enter, then quickly click on the text box:

sleep 1 && echo "Hello world" | xte

Example 2: Open a browser and search Wikipedia

To write a script which combines a few commands for xte, you could put it in a bash script. Remember to put a pause between commands so that the windowing system can catch up.

The script below will use Gnome Shells “overview mode” to launch Chromium, then open a new tab, and search Wikipedia for “cars”:

#!/bin/sh
xte << EOF
key Super_L
usleep 100000
str chromium
usleep 100000
key Return
sleep 1
keydown Control_L
key t
keyup Control_L
sleep 1
str http://en.wikipedia.org
usleep 100000
key Return
sleep 3
key Tab
usleep 100000
str cars
usleep 100000
key Return
EOF

Example 3: Draw a spiral in GIMP

When you script an operation, you can interact with any program using a set of rules. Below is a PHP script called spiral.php, which draws and labels a spiral in GIMP, switching the foreground & background colours at each step.

This requires an open GIMP window in the correct part of the screen:

xte example - Drawing a spiral in GIMP automatically

The interaction in this case is quite simple for a computer, but would be tedious to do manually:

N
Open the Line tool
T
Open the Text tool
X
Swap foreground & background colours
Click & Drag
Draw a line
#!/usr/bin/env php
sleep 2
key N
usleep 100000
<?php
// Where the canvas is on-screen
$top_x = 150;
$top_y = 150;
$canvas_width = 400;

// Spiral properties
$pi = 3.14159265358979;
$centre_x = $top_x + $canvas_width / 2;
$centre_y = $top_y + $canvas_width / 2 + 100;
$spins = 5;

// Centre the mouse
echo "mousemove $centre_x $centre_y\n";
echo "usleep 100000\n";

// Draw a spiral
for($t = 0; $t < 1; $t += 0.005) {
	$angle = $spins * $t * 2 * $pi;
	$radius = $canvas_width / 2 * $t;
	$x = (int) ($centre_x + $radius * cos($angle));
	$y = (int) ($centre_y + $radius * sin($angle));
	echo "mousedown 1\n";
	echo "mousemove $x $y\n";
	echo "usleep 100000\n";
	echo "mouseup 1\n";
	echo "key X\n";
}

// Label the spiral
echo "mousemove $centre_x $top_y\n";
echo "key T\n";
echo "usleep 100000\n";
echo "mouseclick 1\n";
echo "usleep 100000\n";
echo "str Spiral\n";
?>

After cropping, the spiral image on its own is:

A spiral drawn automatically in GIMP via xte

Example 4: Forward emails

Google’s Gmail has keyboard shortcuts for quick navigation. This example uses:

f
Forward an email
Tab
Move between fields
Ctrl+Enter
Send
j
Next email

Using these shortcuts, this script, forward.txt, will forward an email too bob@email.com and fred@example.com, then navigate to the next email:

sleep 1
key f
sleep 1
str bob@example.com
sleep 1
str fred@example.com
sleep 0.1
key Return
sleep 3
key Return
sleep 1
key Tab
sleep 0.1
keydown Control_L
key Return
keyup Control_L
sleep 3
key j
sleep 2

To send one email through xte, you could run this, and then click over to an open email in Gmail:

cat forward.txt | xte

To send the next 106 emails (e.g. in a label or search), you could instead type:

#!/bin/bash
for i in {1..106}; do echo $i; (cat forward.txt | xte); done

This is not fool-proof, so you would need to adjust the timing if your Internet connection is laggy.

When to use xte

Usually, a task which is supposed to be automated will have an API. For example, GIMP provides a python plugin interface, Gmail can be accessed via IMAP, and Google and Wikipedia searches can be done directly through HTTP. This is always the best way to do things.

However, the automation junkie should have xte in their toolkit for as an inventive time-saver, in situations where proper automation is not practical, such as when:

  • you don’t want learn an API, to only use it for one day.
  • you doing a repetitive task in a program or website which is feature-poor.
  • you need to test some feature repeatedly under different circumstances.
  • you find a game which requires you to click fast.

Good luck!

Productivity: De-activate social networking on a schedule with cron

Sometimes you need to block out distractions for a set amount of time. One of these is social networking, and it’s fairly early to temporarily break it (all for productivity of course!).

Mac or Linux users can break example.com like this:

echo "127.0.0.1 example.com www.example.com" >> /etc/hosts

This tells the computer that it hosts example.com, so it wont load it from the internet.

And to compliment this, we have sed, which can delete lines from a file in-place based on a pattern:

sed -i '/example.com/d' /etc/hosts

Replace example.com above with your social network of choice, and have a shot. It will take a few minutes to have an effect, because of open connections and your computer’s DNS cache.

Scheduling it in

Cron is the go-to solution for scheduling any command on Unix. We’ll run this as root, as normal users don’t have permission to edit /etc/hosts:

su
crontab -e

If prompted, select nano as an editor.

If, for example, 6pm — 8pm weekdays is a distraction-free time for you, you would schedule the first command for 18:00 on days 1-5, and the second for 20:00 on days 1-5:

# m h  dom mon dow   command
0 18 * * 1-5 echo "127.0.0.1 example.com www.example.com" >> /etc/hosts
0 20 * * 1-5 sed -i '/example.com/d' /etc/hosts

If your use case calls for something more advanced, consider learning how to use squid to manage web traffic.

When you break something..

If you accidentally delete your /etc/hosts while experimenting, you can fetch its contents from /var/lib/dpkg/info/netbase.postinst (source).

Crash course: Run Windows on desktop Linux

Sometimes, you need to use a tricky windows-only proprietary program on a GNU/Linux desktop. If you have a Windows install disk and licence at your disposal, then this post will show you how to get a Windows environment running without dual-booting.

The host here is a Debian box, and the guest is running Windows 7. The instructions will work with slight modifications for any mix of GNU/Linux and Windows

On the desktop, some things are not as important as the server world. Some things are excluded for simplicity: network bridging, para-virtualised disks, migration between hosts, and disk replication.

Software setup

Everything required from the host machine can be pulled in via Debian’s qemu-kvm package.

sudo apt-get install qemu-kvm

Install

Prepare a disk image for Windows. The qcow2 format is suggested for the desktop as it will not expand the file to the full size until the guest uses the space:

qemu-img create -f qcow2 windows.img 30G

Launch the Windows installer in KVM with a command that looks something like this:

kvm -hda windows.img --cdrom windows-install-disc.iso -vga std -localtime -net nic,model=ne2k_pci -m 2048

Note the -m option is the number of megabytes of RAM to allocate. You can set it a little lower if you don’t have much to spare, but if it’s too low you’ll get this screen:

2014-06-04-capture-noboot

If you have a physical disk but no .iso of it, then using the disk drive via --cdrom /dev/cdrom will work.

Install

If you have GNU/Linux, chances are you have installed an OS before. In case you haven’t seen the Windows 7 installer, the steps are below:

Select language, accept the licence agreement, choose the target disk, and let the files copy:

2014-06-04-capture-005
2014-06-04-capture-023
2014-06-04-capture-024
2014-06-04-capture-028
2014-06-04-capture-033
2014-06-04-capture-035
2014-06-04-capture-081
2014-06-04-capture-149

After reboot, enter the user details, licence key, update settings and timezone:

2014-06-04-capture-160
2014-06-04-capture-178
2014-06-04-capture-190
2014-06-04-capture-191
2014-06-04-capture-199
2014-06-04-capture-205
2014-06-04-capture-214
2014-06-04-capture-250
2014-06-04-capture-254
2014-06-04-capture-261

After another reboot, Windows is installed in the virtual machine:

2014-06-04-capture-263
2014-06-04-capture-265
2014-06-04-capture-277

Post-install tasks

The guest you have now will only run at standard VGA resolutions, and will probably not be networked. This section will show you how to fix that.

Network drivers

You will notice that we are launching the guest with -net nic,model=virtio. This means that we are using a virtual network card, rather than simulating a real one. You need to fetch a disk image with the latest binary drivers, which are best tracked down on linux-kvm.org via google.

Once you have the disk image in the same folder as your virtual machine, shut down and launch it with a CD:

kvm -hda windows.img --cdrom virtio-win-0.1-74.iso -vga std -localtime -net nic,model=ne2k_pci -m 2048

Under "My Computer" track down the "Device Manager", find your network card, and tell Windows to update the drivers. You can then point it to the CDROM’s "Win7" directory (or other one, if you are installing a different guest). After the network adapter is recognised, you will be connected automatically.

Note that you are using "user-mode" networking, which means you are on a simulated subnet, and can only communicate via TCP and UDP (ie, ping will not work). This can be a little slow, but will work on a laptop host whether plugged in or running on WiFi.

Remote desktop

You may also be annoyed by the screen resolution and mouse sensitivity having strange settings. The best way around this is not to fiddle with settings and drivers, but to enable remote desktop and log in via the network. This lets you use an arbitrary screen size, and match mouse speed to the host.

This is set up to run locally, so it is neither laggy nor a security issue, and makes it possible to leverage all RDP features.

First, in the guest, enable remote desktop using these Microsoft instructions.

Then shut down and boot up with the extra -redir tcp:3389::3389 option:

kvm -hda windows.img -vga std -localtime -net nic,model=ne2k_pci -m 2048 -redir tcp:3389::3389

On the host, wait for the guest to boot, then use rdesktop to log in:

rdestkop localhost

One this works, you can shut down and boot with the extra -nographic option to leave remote desktop as the only way to interact with the guest:

kvm -hda windows.img -vga std -localtime -net nic,model=ne2k_pci -m 2048 -nographic -redir tcp:3389::3389

The rdesktop tool supports sound, file and printer redirection. It can also run fullscreen when launched with -f. All the details are in man rdesktop

If you end up using the guest operating system more, it is worth investigating USB-redirection for any peripherals (printers or mobile phones), running a virtual sound card, or running SAMBA on the host to share files.

Rolling your own cloud storage with Unison

Cloud storage is a very cool way to back up files, but it has two major drawbacks:

  1. If you back up everything, it will cost you a bit for the space.
  2. You need to trust that the company wont lose, tamper with, or leak your files.

I’ve recently discovered a little program called Unison, which provides a bandwidth-efficient way to synchronise folders on two computers, solving both of those problems. It has clients for just about every platform, it’s open source, and it’s been around for over a decade.

I’ve set up a copy of my laptop’s home directory on my desktop computer, so if it ever explodes, I can just sync it back:

With most ISP’s offering static addresses, you could even put a NAS on the internet to sync with, which would basically be a home DropBox.

Laptop and desktop example

Unison is in most Linux distributions. On Debian or Ubuntu, you can install it with:

apt-get install unison

mikebook has a profile called mikebox.prf, located in /home/mike/.unison/, which tells it to keep a copy of its home directory on mikebox.

root = /home/mike
root = ssh://mike@mikebox//home/mike/Remote/mikebook/
ignore = Name .*
ignore = Name *.iso
ignore = Name *.img
ignore = Name unison.log
ignore = BelowPath Downloads
ignore = BelowPath workspace

Everything can be brought up to speed with:

unison -batch mikebox

If mikeboox is running out of space, then I can drag files out of Remote/mikebook to somewhere else, and they simply vanish from the laptop next time it is synchronised.

Notes

  • Remote paths require that extra / in the filename.
  • Hidden folders seem to be skipped, so keep a copy of your .prf file in case you blow up your computer.

A tour of ReactOS 0.3.15

ReactOS is a project which aims to create an open source operating system which is binary-compatible with Windows. Although it is still cautiously labelled “alpha”, its basic use is about as reliable as Windows once was.

This post runs through the steps to install ReactOS 0.3.15 as a KVM guest on Linux.

Preparation

Before attempting anything, check that you a CPU supports Intel VT or AMD-V. This command will return the number of CPU cores with svm or vmx flags:

cat /proc/cpuinfo | grep -E 'svm|vmx' | wc -l

Now download the ReactOS 0.3.15 disk from reactos.org, extract it to get the .iso, and fetch some packages if you don’t have them installed:

apt-get install libvirt-bin kvm qemu-utils

Prepare a disk image to install to. If your hardware is slower, then a raw image is a better idea than the qcow2:

qemu-img --help
qemu-img create -f qcow2 reactos.img 4G

The working directory now has:

mike@mikebox:~/vm/reactos$ ls -Ahl
total 77M
-rw-r--r-- 1 mike mike  77M May 19  2013 ReactOS-BootCD.iso
-rw-r--r-- 1 mike mike 193K Jan 30 21:05 reactos.img

Installation and first boot

The kvm command will pop up a window with the guest operating system. To boot from the install disk, run:

kvm -hda reactos.img --cdrom ReactOS-BootCD.iso -vga std -localtime -net nic,model=ne2k_pci -net user

The meaning of each of these options is:

-hda reactos.img
Sets the HDD image file.
--cdrom ReactOS-BootCD.iso
Sets the CDROM image file. Because reactos.img is blank, this will boot.
-vga std
Sets the VGA card.
-localtime
Emulates a system clock in local time, rather than UTC.
-net nic,model=ne2k_pci
Sets the network card to something ReactOS will recognise.
-net user
Enables user-mode networking. Your computer will emulate a network and pass on TCP and UDP connections. This is the easiest mode to use, but ICMP packets (such as pings) will not work, and the VM will not be accessible from other computers.

Installation was fast, error-free, and did not require a network connection. The first screen capture below was taken at 16:04:49, and the desktop was captured at 16:06:07 (1 minute 18 seconds later). Most of that time would have been wasted waiting for user input.

ReactOS installer language select
ReactOS install or repair
ReactOS installer disclaimer
Confirm install settings
Select partition
Formatting options for new partition
Formatting confirmation dialog
ReactOS install directory
ReactOS installer copying files
Bootloader options (freeloader)

After copying files, the installer reboots to a more user-friendly mode (similar to the Windows installer):

ReactOS boot menu
ReactOS loading NTOSKRNL.EXE
ReactOS boot screen
ReactOS Install - Installing devices
ReactOS Install - Welcome to the ReactOS Setup Wizard
ReactOS Install - Acknowledgements
ReactOS Install - Personalize your Software
ReactOS Install - Computer name and Administrative Password
ReactOS Install - Regional settings
ReactOS Install - Date and Time
ReactOS Install - Registering Components
ReactOS Install - Completing the ReactOS Setup Wizard
ReactOS desktop after installation

The installed system

After installation, the --cdrom option can be dropped:

kvm -hda reactos.img -vga std -localtime -net nic,model=ne2k_pci -net user

The first thing I did was correct the colour depth, and then attempt to install VLC. This did not turn out well (the console screen is QEMU-monitor):

Reactos display properties
VLC installation in ReactOS Applications Manager
Ctrl+Alt+Del from QEMU monitor
BSOD during VLC installation

I used command prompt to verify that networking was fine (note the lack of ICMP in user-mode networking):

Ping from command prompt with user-mode networking as KVM guest
Testing network on ReactOS

The Firefox 22 install worked, but it went awry after that. Several reboots later I gave up:

Firefox installer on ReactOS
Firefox frozen on startup
BSOD while running firefox on ReactOS

The built-in programs were much more usable:

ReactOS paint
ReactOS paint Save As dialog
ReactOS Explorer
ReactOS Start Menu showing Administrative Tools
ReactOS Device Manager

PuTTY installed flawlessly, and I was able to SSH to the host computer:

PuTTY installation on ReactOS
PuTTY readme in ReactOS notepad
Main PuTTY window after installation
Using command prompt to find host computer address:
PuTTY with host computer address
PuTTY connected to host computer, showing 'uname -a' output

An example of a frozen program causing graphics glitches (Windows up to XP does this as well):


Frozen Application Manager in ReactOS

And an obligatory screenshot of the “Properties for System” dialog, showing the build as 20130518-r59037:


Reactos 'Properties of System' dialog 20130518-r59037

Conclusion

ReactOS is a cool idea and project, but the OS is still very glitchy. The built-in apps are stable and familiar-looking, but you would require a lot of patience (and a lot of rebooting) to use a ReactOS system for more than a few minutes.

Being open source is a big plus, as there is no need to activate the installation or enter software keys. GNU/Linux users will already be accustomed to this.

Scripted screen captures

This is a script I put together for capturing a window’s contents as it changes, because “Print Screen and crop” gets old very quickly!

It saves me a lot of time when working in virtual machines or creating user docs, as it means that every step (and error message) is captured.

The commands used are in the x11-utils and netpbm and x11-apps packages on Debian.

sudo apt-get install x11-utils netpbm x11-apps

capture.sh

#!/bin/bash
echo "Click a window to start capturing it."
window=`xwininfo -int | grep 'Window id:' | cut -d' ' -f4`
echo -n "Capturing window $window"
prev=""
i=1
empty=`echo -n "" | md5sum | cut -d' ' -f1`
captured="captured.txt"
echo -n "" > $captured
while [ "$prev" != "$empty" ]; do
	md5sum=`xwd -id $window 2> /dev/null | xwdtopnm 2> /dev/null | md5sum | cut -d' ' -f1`
	if grep -Fxq "$md5sum" "$captured"
	then
		echo -n "."
	else
		if [ "$md5sum" != "$empty" ]
		then
			echo $md5sum >> $captured
			echo ""
			echo -n $md5sum
			file=`date --iso`-capture-`printf "%03d" $i`.png
			xwd -id $window | xwdtopnm 2> /dev/null | pnmtopng 2> /dev/null > $file
			i=$[i+1]
		fi
	fi
	sleep 1
	prev=$md5sum
done
echo ""
echo "Empty screen capture received. Quitting. (did you close the window?)"

Example

If you wanted to document a “Malformed Expression” error in Gnome Calculator, you can run capture.sh and then demonstrate it:

$ ./capture.sh
Click a window to start capturing it.
Capturing window 31457283
3bbe32ef05f49ae65922fcfedc842828
c0cef7d3108263fbb1beaa7b52492e6a
fd6df04c4ad844bb4a2f27be29dffb29
6ba0fb7ee1ca85640998013c6a258520.
bb061cc56fa3822f9764c0f6af2156df
5f788568c7f34cda55a1680ac72e1cf0
dca714e6691a3a239914106996905047
0bf916ba3f96f3def57191174e55dea0.
2a4a2589662901d9a55d1e170ffbd322....
Empty screen capture received. Quitting. (did you close the window?)

The lines are checksums of PNM data, and the dots are times when no screenshot was saved. This guarantees that each file in the output is unique:


Example of captured files

The captured.txt file simply contains the list of checksums, and is useless after the script terminates.