Tag Archives: linux

OpenWrt setup on Netgear WNR2200

I recently wanted to connect some devices for a temporary setup, where a wireless LTE modem would provide Internet access. Unfortunately, one of the devices was not close enough to pick up the signal with its USB WiFi dongle.


Because the modem does not have a LAN port, the usual “run a cable” solution was out. There’s a few other options, from range extenders, to getting better modem, or just upgrading to a “real” USB WiFi dongle. Before purchasing new hardware, I decided to try re-purposing an old Netgear WNR2200 as a wireless client and 4 port switch.


In this setup, the LTE modem does the heavy lifting, with all of the wireless clients using it for LAN and Internet access. In the next room, the Netgear router is placed close enough to pick up the signal, and an Ethernet cable runs to the PC, beyond the reach of WiFi.

Deciding to re-flash

Replacing firmware is worth investigating when the hardware is capable, but you aren’t given the option to configure it the way you want.

The Netgear WNR2200 is a low end wireless router, and the vendor firmware does not support joining a WiFi network as a client.


It also pays to update your research. OpenWrt added support for this router a few days after I bought it, but I hadn’t looked it up again.

Uploading firmware

My main resource was this page on the OpenWRT Wiki. Firmware is organised by wireless chipset, then by router model.

The file I used to update my router was named openwrt-15.05.1-ar71xx-generic-wnr2200-squashfs-factory.img.

This is simply uploaded on the Adminisration → Firmware Upgrade screen:



First impressions

The first thing I noticed was that I lost WiFi, and that the page I had bookmarked for logging in was no longer valid!


This makes sense, of course. The configuration will not be carried across from the vendor firmware, and a different web administration tool is being used.

The Linux userspace is very rich compared with vendor firmware. It has things like dmesg, SSH, ifconfig, ping, and even a networked package manager.

Configuration checklist

I performed all configuration through the web in this setup. The “LuCi” interface allows setting the WiFi chip into “Client” mode, and then searching and joining a network. Once this was done, I assigned it as the “WAN” interface, so that it occupied a single IP address on the WiFi network, and providing a NAT and wired, four port switch.

There are more advanced, bridged setups that are possible. You should investigate this if you want one network, so that things like printer auto-discovery and internal SSH work consistently. I was only interested in sharing the Internet connection, which is why the setup was so simple.

What didn’t work

USB, but I didn’t spend long on this either. I was considering using USB to connect the modem to the Netgear router. The Wiki suggests that this is now possible, but after installing some packages for “USB tethering” and rebooting, I had no luck. Typing lsusb, only the “root hub” was listed, and the device was not getting any power.

This was necessary for the setup, so I just abandoned it. The vendor firmware couldn’t use the USB port for networking either, so no real loss.

libvirt: Migrate a VM from qemu:///session to qemu:///system

In recent versions of the libvirt virtualisation libraries, you to create and manage virtual machines as a regular user, using the qemu:///session connection.

This is great, but the networking is quite limited. I found that machines defined in Gnome Boxes could not speak to each-other, and that libvirt commands for networking were unavailable.

For this reason, I’ve written this quick guide for booting up an existing same VM image under the qemu:///system instance, which is faster than re-installing the machine. Unlike most sorts of migrations, this leaves the disk image at the same location on the same host machine.

There’s many different ways to do VM’s in Linux. This setup will be useful only if you use libvirt/kvm using qcow2 images on Debian. As always, consider doing a backup before trying new things.


First, find your virtual machine in virsh, and dump its configuration to a text file in your home directory, as a regular user.

$ virsh list --all
 Id    Name                           State
 -     foo-machine                    shut off
$ virsh dumpxml > foo-machine.xml

Now remove the VM definition from your user:

$ virsh undefine foo-machine
Domain foo-machine has been undefined

Import the definitions into virsh as the root user:

$ sudo virsh define foo-machine.xml 
Domain foo-machin defined from foo-machine.xml

Attempt to start the new VM definition. Depending on where the disk image is, expect an error.

$ sudo virsh start foo-machine

Disk images

The disk image needs to be accessible to the libvirt-qemu user. There’s two basic ways to achieve this: Re-permission the directories above it, or move it.

I chose to just re-permission it, since it’s not an issue to have world-readable directories on this particular box:

$ cat foo-machine.xml | grep source
      <source file='/home/example/.local/share/gnome-boxes/images/foo-machine'/>

This one-liner outputs the commands to run to make a directory work-navigable:

$ dir=`pwd`; while [ "$dir" != "/" ]; do echo "chmod o+x,g+x \"$dir\""; dir=`dirname $dir`; done
chmod o+x,g+x "/home/example/.local/share/gnome-boxes/images"
chmod o+x,g+x "/home/example/.local/share/gnome-boxes"
chmod o+x,g+x "/home/example/.local/share"
chmod o+x,g+x "/home/example/.local"
chmod o+x,g+x "/home/example"
chmod o+x,g+x "/home"

And the user account needs to be able to write as well:

$ sudo chown libvirt-qemu /home/example/.local/share/gnome-boxes/images/foo-machine

Once you have the permissions right, the VM should start, using the same command as before:

$ sudo virsh start foo-machine

More importantly, you can now hook up virt-manager and view your machine on qemu:///system, allowing you to configure the VM with any network settings you need.

How to set up Docker containers in Travis CI

This post outlines a method for using Docker for testing on Travis CI. It may be useful to you if you are a web application developer who uses GitHub.

I use this setup in my web-based word puzzle generator, so that every change is spun up and tested with a web browser before it is merged.

I got the idea for writing this from a few lines in the docker-compose documentation, which suggested that Docker is an easy way to perform automated testing over a running application:

$ docker-compose up -d
$ ./run_tests
$ docker-compose down

This snippet was missing some setup and an example app, but these three lines do all the heavy lifting.

The Docker setup

In order to focus on the Docker setup, I made a server which simply responds to TCP requests on port 5000 with the text “Hello World”.

This file is called server.sh, and sits in a directory called foo_server:

while true; do
  # Send 'Hello World' to anybody who connects on port 5000
  echo "Hello World" | nc -l 5000

Alongside it, I added a Dockerfile to instruct docker to execute this tiny ‘application’ in a container, after installing the dependencies. This machine is built from the Docker-official Debian image:

FROM debian
ADD . /usr/share/test-server
WORKDIR /usr/share/test-server
RUN apt-get update && apt-get install --assume-yes netcat-openbsd
CMD ./server.sh

Lastly, a .dockerignore file is used to avoid loading the Dockerfile to the container:

# Ignore docker files

In the directory above, a simple test script, test.sh is used to see that the server is returning the expected output:

set -e
expected="Hello World"
actual=`nc -v localhost 5000`
echo "Expecting: $expected"
echo "Server says: $actual"
if [ "$expected" != "$actual" ]; then
  echo "Test failed"
  exit 1
  echo "Test passed"
  exit 0

Alongside the test file, a file called docker-compose.yml instructs Docker to create a container out of the foo_server example, and forward port to it.

version: '2'
    build: foo_server
     - "5000:5000"
    container_name: foo_1

To try it out for yourself, you need a relatively recent version of Docker and docker-compose. The versions provided in Debian were not new enough to execute the examples, but the Docker project provides repos containing newer builds for Debian & Ubuntu. For my distro, the install was:

curl -sSL "https://get.docker.com/gpg" | sudo -E apt-key add -
echo "deb https://apt.dockerproject.org/repo debian-stretch main" | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get install docker-engine
sudo pip install docker-compose

The versions this got me were docker 1.11.2, and docker-compose 1.7.1. Straight after the install, I could deploy & test an example locally:

$ docker-compose up --build -d
$ ./test.sh
$ docker-compose down

The CI setup

I’ll assume that if you’re reading this, you are familiar with the basics of Travis CI. The large block of code below is the .travis.yml file to set up the test machine, then execute the tests against a container.

# Use Ubuntu 'trusty' distribution
sudo: required
dist: trusty

  # Update docker-engine using Ubuntu 'trusty' apt repo
  - >
    curl -sSL "https://get.docker.com/gpg" |
     sudo -E apt-key add -
  - >
    echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" |
     sudo tee -a /etc/apt/sources.list
  - sudo apt-get update
  - >
    sudo apt-get -o Dpkg::Options::="--force-confdef" \
     -o Dpkg::Options::="--force-confold" --assume-yes install docker-engine
  - docker version
  # Update docker-compose via pip
  - sudo pip install docker-compose
  - docker-compose version

  - docker-compose up --build -d

  - ./test.sh

  - docker-compose down

Note: This uses Travis CI’s trusty distribution, which at the time of writing is the newest stable build platform available on Travis CI. This shipped an outdated version of Docker, which had to be installed over. Because the existing Docker was configured, I had to override a debconf prompt, which is why the apt addon syntax was not used to set up dependencies.


The build result for each commit is displayed in Travis CI:


Under this, the output of the passing test script is shown, showing what has been set up:


Using this setup in practice

Moving this from a demo setup to a real setup would be fairly simple:

  1. Replace the installation with a real software stack
  2. Replace the server run with a command to serve the application (such as a Apache HTTP, Tomcat or Node)
  3. Replace the tests with real tests (such as Cucumber or Selenium).

The example in the pre-amble installs a LAMP stack and tests it with Selenium in its CI build.

If your application is a bit larger, your only extra complexity will come from running multiple containers with docker-compose.

Get the code

All of these scripts in a working CI example are available on GitHub:

How to compile a C++11 app on Travis CI

I have recently been adding Travis CI builds to code that I host on GitHub, so that I don’t need to host my own build infrastructure.

To users, this just means that there is a green badge at the top of the README, but not much else:


To build a simple C++ project, I added in this .travis.yml file:

langauge: cpp
sudo: false

      - libusb-1.0-0-dev

  - make

Unfortunately, on this infrastructure, the default build tools are currently ancient, and installed on Ubuntu Precise (12.04):

$ make
g++ src/missile.cpp examples/basic-sync/basic-sync.cpp -o bin/basic-sync -lpthread -lusb-1.0 -std=c++11 -Wall
cc1plus: error: unrecognized command line option ‘-std=c++11’
cc1plus: error: unrecognized command line option ‘-std=c++11’

Option 1: Update the toolchain

There is some structures you can use to install an extra repository and some named packages, instead of using apt-get directly.

langauge: cpp
sudo: false

    - ubuntu-toolchain-r-test
    - gcc-4.8
    - g++-4.8
    - libusb-1.0-0-dev

  - make

Because the old version was still installed, I had to refer to the exact version in the Makefile, as in:

g++-4.8 src/missile.cpp examples/basic-sync/basic-sync.cpp -o bin/basic-sync -lpthread -lusb-1.0 -std=c++11 -Wall

Option 2: Update the platform

You can also change to a more recent Ubuntu distribution. Presumably Ubuntu Precise is only the default because existing builds use it.

If you need to build C++11 apps on Travis CI, then builds will work under Ubuntu Trusty (14.04), which happens to be the newest distribution currently available:

langauge: cpp
sudo: required
dist: trusty

      - libusb-1.0-0-dev

  - make

Convert a PC to a HTPC with Debian and Kodi

I recently converted an old workstation to run as a home-theatre PC (HTPC). I’ve noted down the setup here for others who are making an installation like this. Some steps depend on using a radeon chipset, and will need to be adjusted for your computer.


First up, Desktop ‘towers’ are not a good form-factor for sitting in TV cabinets. If your PC is this sort of size, then source a small form-factor case and power supply, and load the computer’s components into it:


I also used a Logitech k400r keyboard and mouse for wireless input.

Install Debian and apps

Write the latest copy of Debian Stable to a CD or flash drive (this is version 8.3 at time of writing), and install it on the computer. Check “Debian Desktop environment” / GNOME during setup.

After installation, open a terminal, and type “su” to get root privileges.


Edit the software sources to include ‘contrib’ and ‘non-free’, as well as ‘jessie-backports’.

nano /etc/apt/sources.list
deb http://ftp.us.debian.org/debian/ jessie main contrib non-free
deb-src http://ftp.us.debian.org/debian/ jessie main contrib non-free

deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free

# jessie-updates, previously known as 'volatile'
deb http://ftp.us.debian.org/debian/ jessie-updates main contrib non-free
deb-src http://ftp.us.debian.org/debian/ jessie-updates main contrib non-free

# jessie-backports
deb http://ftp.us.debian.org/debian/ jessie-backports main contrib non-free
deb-src http://ftp.us.debian.org/debian/ jessie-backports main contrib non-free

Update sources and install Kodi:

apt-get install --install-suggests kodi

Also install the firmware packages that you may need.

apt-get install firmware-linux-free firmware-amd-graphics



sudo allows you to run commands as root from your regular user account. Install the package and add yourself to the sudo group:

apt-get install sudo
usermod -a -G sudo mike

To apply the change, log out and back in again. The rest of this guide assumes you are logged in as yourself, and will use sudo where necessary.

Auto-start Kodi

Open the tweak tool, and locate the list of startup programs.


Add Kodi to the list, log out, log in, and Kodi will launch automatically.


For a PC attached to a TV, user permissions are not so importnat, so set the user to log in automatically.

sudo nano /etc/gdm3/daemon.conf

Un-comment this block and enter your username:

# Enabling automatic login
#  AutomaticLoginEnable = true
#  AutomaticLogin = user1

Plymouth start-up screen

Install plymouth and configure grub to change the Debian boot sequence (a menu with timeout, followed by lots of text) into a graphical splash screen. This takes a bit of configuration.

sudo apt-get install plymouth

Set it up according to these instructions:

sudo nano /etc/initramfs-tools/modules

Set drm correctly for your chipset:

radeon modeset=1

Configure grub:

sudo nano /etc/default/grub

Update grub, set the theme in Plymouth:

sudo update-grub2
sudo /usr/sbin/plymouth-set-default-theme --list
sudo /usr/sbin/plymouth-set-default-theme joy

Run update-initramfs to apply the changes

sudo update-initramfs -u


Samba will let you share folders over your network. A basic folder with guest read/write is simple to set up:

sudo apt-get install nautilus-share samba libpam-smbpass winbind
sudo usermod -a -G sambashare mike

Log out, and back in to apply the group change, and then share the Public folder over the network by right-clicking on it and opening the “Sharing Options”:


Gnome will warn that the folder as shared if you open it:


Test the setup by typing smb://localhost into the address bar:


Overscan correction

In my case, I was able to set the TV to treat the input as a “PC” input. If that doesn’t work for you, then use xrandr in a login script:

Find the name of your input:

xrandr --query

Set underscan (get the horizontal and vertical values by trial and error):

xrandr --output HDMI-0 --set underscan on
xrandr --output HDMI-0 --set "underscan hborder" 32 --set "underscan vborder" 16

Kodi plugins

Add these as needed. The Australian catchup TV plugins repository from GitHub worked well.

Kodi RSS

The RSS feed shows Kodi updates by default, and is part of your user profile.


Edit the configuration file, and adjust the paths to your news sources of choice.


Boot speed

Readahead is the tool of choice for boot speed optimisation. Install it, and reboot.

sudo apt-get install readahead
sudo touch  /.readahead_collect
sudo reboot

Desktop Apps

If you quit Kodi, you are dropped back to the GNOME desktop. These apps are simply to improve the desktop user experience.

Google Chrome

Download the .deb file for Chrome from Google, install with dpkg, and then clean up dependencies:

dpkg -i google-chrome-stable_current_amd64.deb 
apt-get -f install


Download and extract the Firefox for Linux tarball from Mozilla.

Move it to /usr/share, and change the owner to match other applications there.

mv firefox /usr/share/
cd /usr/share/
ls -Ahl
chown root:root firefox
chown -R root:root firefox

Find the main menu editor, and add Firefox to the menu.


Firefox Web Browser



Test the new icon by searching:


Auto-clear browser profiles

Because you don’t need a password to log in to the user account, you can add this as a bit of insurance so that your box wont remember any passwords or sessions.

crontab -e

This job removes the Firefox and Chrome user profiles each boot.

@reboot rm --preserve-root -Rf --~/.config/google-chrome ~/.cache/google-chrome ~/.mozilla/firefox ~/.cache/mozilla/firefox


For file format support, best to have another media player:

sudo apt-get install vlc


You should now have a PC which boots into Kodi for media and TV, and lets you quit into a desktop to browse the web or run regular desktop apps.



On the 1GB RAM/ dual core workstation, the shortened the boot to around 45 seconds from BIOS handing over control, to Kodi being ready.

Fix merge conflicts in git with Meld

When you’re writing code collaboratively, there’s plenty of situations when you need to combine two sets of changes.

This could happen, for example, if Bob and yourself both fix different bugs by making edits to the same file.


This post assumes that your source code is tracked in git.

First up, install meld. The Meld homepage has instructions for other platforms, but on Debian/Ubuntu, it’s just:

sudo apt-get install meld

Now tell git to use it as a tool:

git config --global merge.tool meld

Once you have a merge conflict, you can then fire up Meld like this:

git mergetool

For each file, you will get a 3-way diff. Click the arrows on the sides to move the code you want into the middle:


Once you’ve saved the file and closed Meld, you will be prompted on the command-line. You just tell it whether you’ve successfully merged the file, until it stops giving you new files to merge.

After this, commit the changes:

git commit


Simpler usage

If you don’t use git, you can simply call Meld from the command-line as well. This shows you differences between files in a similar window, and lets you move blocks of code around as well:

meld foo.c bar.c

How to run Tetris on your Raspberry Pi

This is a simple walkthrough on how to install my Tetris clone, Blocks, on a Raspberry Pi.

On most computers running Debian (or Raspbian in the case of the Raspberry Pi), it’s as simple as clone, compile, run:

sudo apt-get install libncurses5-dev doxygen
git clone https://github.com/mike42/blocks
cd blocks

If you have any issues running this, then you need to fetch a newer version of GCC, as this needs C++11 support to compule (see last section for instructuins).

But if all goes to plan, you will get something like this in your terminal:


Use the keyboard to control the game:

Right, down, left

Get a screen

Basically any project with graphics can benefit from one of these. Simply add on a TFT shield, such as PiTFT to create a tiny console:


Of course, this is still keyboard-controlled, but with some hacking, I’m sure you could map touch events to keyboard actions.

Troubleshooting: Update GCC

The Raspbian spftware image which many Raspberry Pi’s have is slightly too old to compile Blocks, which requires C++11 support.

Luckily, it’s very easy to upgrade from wheezy to jessie to add it. You know you need to do this if you get this error compiling:

$ git clone https://github.com/mike42/blocks
$ make
mkdir -p bin
g++ src/main.cpp src/blocks_game.cpp src/blocks_shape.cpp -o bin/blocks -lcurses -lrt -std=c++11 -Wall
cc1plus: error: unrecognized command line option ‘-std=c++11’
cc1plus: error: unrecognized command line option ‘-std=c++11’
cc1plus: error: unrecognized command line option ‘-std=c++11’
Makefile:2: recipe for target 'default' failed
make: *** [default] Error 1

Generally this means you don’t have GCC 4.8, which is not available in wheezy edition of Raspian.

$ g++ --version
g++ (Debian 4.6.3-14+rpi1) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO

So to summarise this thread, you need to:

nano /etc/apt/sources.list

Find this line:

deb http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi

And change the word “wheezy” to “jessie”:

deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi

You can then update everything with:

sudo apt-get update && sudo apt-get dist-upgrade

You are now running the newer jessie release, which gives you access to the GCC 4.8 package we need:

apt-get install g++-4.8

So we can pick up where we left off, and compile the game:


How to empty your local user account

If you’re not going to use a user account on your computer again, but can’t delete it for some reason, then emptying it is the next best thing to do.

Note: Save anything you want to keep before you start deleting things. These are destructive commands which delete all of the files and settings in the current user’s profile. If you are at all unsure, consider using a file browser to clear out the profile instead.


del /A / F /Q /S .

Linux or Mac:

cd ~
rm -Rf .

This will make sure that the disused account no-longer wastes any disk space.

Getting a USB receipt printer working on Linux

In this post, I’ll step through how to get a thermal receipt printer with USB interface appearing on Linux. The aim of this is to be able to point a driver such as escpos-php at the device. The printer used here is an Epson TM-T20, which is very common in point-of-sale environments.

I have previously written quite a bit about how to use thermal receipt printer protocols, but the previous printer I covered had only a network interface, not USB like this one:


The directions below are for Debian, but could be adapted for any other Linux.

Find the device file

Plug in your printer, and check that usblp sees it:

[12724.994550] usb 8-4: new full-speed USB device number 5 using ohci-pci
[12725.168956] usb 8-4: New USB device found, idVendor=04b8, idProduct=0e03
[12725.168963] usb 8-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[12725.168968] usb 8-4: Product: TM-T20
[12725.168971] usb 8-4: Manufacturer: EPSON
[12725.168975] usb 8-4: SerialNumber: ....
[12725.175114] usblp 8-4:1.0: usblp1: USB Bidirectional printer dev 5 if 0 alt 0 proto 2 vid 0x04B8 pid 0x0E03

This kernel module makes your printer visible as a device file, so that it can be accessed in the old-fashioned way. Find the new device file under /dev/usb:

ls /dev/usb

In my case, this was /dev/usb/lp1. The next step is to see if you can write to it:

echo "Hello" >> /dev/usb/lp1

Chances are, you will get a permission denied error at this point, so find out what group the printer is in:

stat /dev/usb/lp1

Which will show output something like:

File: ‘/dev/usb/lp1’
  Size: 0         	Blocks: 0          IO Block: 4096   character special file
Device: 5h/5d	Inode: 220997      Links: 1     Device type: b4,1
Access: (0660/crw-rw----)  Uid: (    0/    root)   Gid: (    7/      lp)

This file is owned by group lp (“line printer”). If your username was bob, you would add yourself to this group using:

sudo usermod -a -G lp bob

If you plan to build a web-based point-of-sale system with this, then also add the www-data user to that group.

Now log out and back in, and the previous test should now be working:

echo "Hello" >> /dev/usb/lp1

Troubleshooting: Check usblp

If these steps don’t work, then your computer ether doesn’t have, or isn’t using usblp You’ll need to check a few things:

  • Install a different linux-image if the driver is not on your computer at all.
  • modprobe or insmod usblp
  • blacklist a vendor driver which has claimed the interface.
    • run lsusb -v and usb-devices (look for driver=)

Printing something useful

As a duplicated section from my earlier post, the printer uses ESC/POS, which means it accepts plaintext with some special commands for formatting.

A simple receipt-generator, foo.php, might look like this:

/* ASCII constants */
const ESC = "\x1b";
const GS="\x1d";
const NUL="\x00";

/* Output an example receipt */
echo ESC."@"; // Reset to defaults
echo ESC."E".chr(1); // Bold
echo "FOO CORP Ltd.\n"; // Company
echo ESC."E".chr(0); // Not Bold
echo ESC."d".chr(1); // Blank line
echo "Receipt for whatever\n"; // Print text
echo ESC."d".chr(4); // 4 Blank lines

/* Bar-code at the end */
echo ESC."a".chr(1); // Centered printing
echo GS."k".chr(4)."987654321".NUL; // Print barcode
echo ESC."d".chr(1); // Blank line
echo "987654321\n"; // Print number
echo GS."V\x41".chr(3); // Cut

And you would send it to the printer like this:

php foo.php > /dev/usb/lp1

Scaling this up

The codes are quite tricky to work with manually, which is why I put together the escpos-php driver. You can find it at:

The above example would be written using escpos-php as:

require __DIR__ . '/autoload.php';
use Mike42\Escpos\Printer;
use Mike42\Escpos\PrintConnectors\FilePrintConnector;
$connector = new FilePrintConnector("/dev/usb/lp1");
$printer = new Printer($connector);

/* Print some bold text */
$printer -> setEmphasis(true);
$printer -> text("FOO CORP Ltd.\n");
$printer -> setEmphasis(false);
$printer -> feed();
$printer -> text("Receipt for whatever\n");
$printer -> feed(4);

/* Bar-code at the end */
$printer -> setJustification(Printer::JUSTIFY_CENTER);
$printer -> barcode("987654321");
$printer -> cut();

This would be sent to the printer by loading it from the web, or running the script on the command-line:

php foo2.php

Two ways to back up your Google Apps account

If you use Gmail or hosted Google Apps, you might be interested in taking a backup of your data, such as emails, Drive documents, and calendar entries. Thankfully, you can usually export copy of your account data using Google Takeout.

If your hosted Apps account has Takeout disabled, then you can do a backup, it simply has a few extra steps.

Option 1: Google Takeout

This method is nice and simple. Simply go to the Data tools – Download your data page, and select which services you want to export:


It can be a bit eye-opening to see the amount of data Google has on you (Files, conversations, location history, etc). At this point, click through to “Prepare Download”. Depending on the size of your account, this may take as a coffee break, a few hours, or even an entire day.


If you check the box for it, you’ll get an email like this when your Download completes:


And this lets you fetch a single file:


The .zip file contains a series of folders, one for each service. The defaults seem to be:

A unix mbox file
One iCal file for each calendar
One vCard file for each group.
Exports as PDF, docx, xlsx

Option 2: Export data from each service

Sometimes, Google Takeout isn’t an option.


Luckily, most Google services have some sort of data export built in. This means, if you have a new contact manager, or want to include your Drive in your PC backup, it’s still possible.

The export formats in these examples should match the Google Takeout defaults. Tab through each service t see how to export it:

If you are not a power user, then I would suggest setting up a copy of Mozilla Thunderbird via IMAP, and regularly using it for your email. This is a simple way to keep a clone of your inbox on your desktop computer, so that it can be included in backups.

If you are more tech-savvy, then the rest of this section will focus on helping you generate an mbox file containing a full backup of your email, the same format as Takeout uses. The best tool for that is a Linux program called getmail.

On Debian or Ubuntu Linux, issue this command to install getmail:

sudo apt-get install getmail4

For other package managers, see these directions.

First, you need to enable IMAP for your account, see Google’s article: Get started with IMAP and POP3, for the steps.

Now create a file at ~/.getmail/getmailrc, and configure it to read your email account via IMAP/SSL.

type = SimpleIMAPSSLRetriever
server = imap.gmail.com
port = 993
username = bob@mail.example.com
password = ....

type = Mboxrd
path = ~/inbox

verbose = 1

After some time, you will end up with a large mbox file at ~/inbox, containing all of your mail.

If, for some reason, you need to use POP3 instead, then see this article on Gmail backup

Go to your contacts, and find a group. Check the box next to each name, and then find the Export button:


Select the vCard format here, as it’s the same format which Takeout would have used:


Google provides a share-able iCal link, which you can download once, but it is only available if your calendar is public.

So, if your calendar isn’t too sensitive, click “Share” and make the calendar public:2015-02-calendar-01-share


Go to “Calendar Settings”, find the iCal link. It may take a few minutes for the link to start working, but once it does, download it, and then turn off public sharing.

There is a small risk that somebody else loads your calendar while its public, so if this concerns you, then save the events individually.

Exporting from Google Drive is nice and simple. Select all of your files (Shift+Click):


And then find the Download button:


If you apply this to your whole drive, it may take a while, so you may wish to download it in parts if your Internet can be unreliable.

Know how to export a different service? Send it in and I’ll add it to the list.

How do I use these files?

Google Drive’s files are exported in familiar formats. If you haven’t used an mbox, vCard or ics file before, then you will need to find a program which can read these for you.

Google’s support answer “Download your data: Per-service information” contains a list of files types which you’ll run into during this process, and suggests programs which can import them.