How to compile a C++11 app on Travis CI

I have recently been adding Travis CI builds to code that I host on GitHub, so that I don’t need to host my own build infrastructure.

To users, this just means that there is a green badge at the top of the README, but not much else:

passing.svg

To build a simple C++ project, I added in this .travis.yml file:

langauge: cpp
sudo: false

addons:
  apt:
    packages:
      - libusb-1.0-0-dev

script:
  - make

Unfortunately, on this infrastructure, the default build tools are currently ancient, and installed on Ubuntu Precise (12.04):

$ make
g++ src/missile.cpp examples/basic-sync/basic-sync.cpp -o bin/basic-sync -lpthread -lusb-1.0 -std=c++11 -Wall
cc1plus: error: unrecognized command line option ‘-std=c++11’
cc1plus: error: unrecognized command line option ‘-std=c++11’

Option 1: Update the toolchain

There is some structures you can use to install an extra repository and some named packages, instead of using apt-get directly.

langauge: cpp
sudo: false

addons:
  apt:
    sources:
    - ubuntu-toolchain-r-test
    packages:
    - gcc-4.8
    - g++-4.8
    - libusb-1.0-0-dev

script:
  - make

Because the old version was still installed, I had to refer to the exact version in the Makefile, as in:

g++-4.8 src/missile.cpp examples/basic-sync/basic-sync.cpp -o bin/basic-sync -lpthread -lusb-1.0 -std=c++11 -Wall

Option 2: Update the platform

You can also change to a more recent Ubuntu distribution. Presumably Ubuntu Precise is only the default because existing builds use it.

If you need to build C++11 apps on Travis CI, then builds will work under Ubuntu Trusty (14.04), which happens to be the newest distribution currently available:

langauge: cpp
sudo: required
dist: trusty

addons:
  apt:
    packages:
      - libusb-1.0-0-dev

script:
  - make

A Yes/No filtering tool for images

While collecting very large numbers of screen captures for writing documentation, I noticed that it takes far too long to filter out the junk.

To fix this, I coded up yn (source code via github). Given a list of images, it will display each one to the user, which they then include by pressing Y (yes) or exclude by pressing N (no) (hence the name yn).

The images which are selected are saved as a list, so that a script can continue by processing them in some way. This could be by copying them elsewhere, or generating a document with spaces to caption them. The example in the README.md does both of these.

The code to this early version is very simple, which makes it a good example of a simple OpenCV C++ app. I’ve stepped through it below.

The code

C++ does console input and output via the iostream library:

#include <iostream>

To use OpenCV with a GUI, you need these headers. You then need to add them to your include path, and link to OpenCV for the program to compile:

#include <cv.h>
#include <highgui.h>

This line simly tells the compiler that when we say cout (console out), we mean std::cout.

using namespace std;

The only OpenCV code is in the below function. The steps are:

  1. load an image from a file via imread(),
  2. display it in a window with imshow(),
  3. capture the next keypress with waitKey(), then
  4. delete the window with destroyWindow()
/**
 * Load a file, and wait for the user to press a key.
 *
 * If the pressed key is 'y', print the filename.
 */
bool yn(string fn) {
    cv::Mat img;
    char key = 0;

    img = cv::imread(fn);
    if(!img.data) {
        cerr << "Failed to load " << fn << endl;
    } else {
        cv::namedWindow(fn);
        cv::imshow(fn, img);
        key = cv::waitKey(0);
        cv::destroyWindow(fn);
        if(key == 'y') {
            cout << fn << endl; // 'y' pressed
        } else if(key == 0x1B) {
            return false; // ESC pressed
        }
    }
    return true;
}

So with that library usage out of the way, all we need to do is get the list of files to check, and stop popping up windows when the user has pressed the escape key.

/**
 * Get list of files from command-line arguments and display them in turn
 */
int main(int argc, char** argv) {
    int i;

    /* Command-line arguments given */
    for(i = 1; i < argc; i++) {
        if(yn(string(argv[i])) == false) {
            cerr << "Quitting.\n";
            break;
        }
    }

    return 0;
}

Next

The delay between images appearing can be removed by loading them in a separate thread, which I may do in a future version.

Recovering auto-saved files in MySQL Workbench

MySQL workbench is an open source tool for designing databases. As version 6.0.8, it is one of those programs where you need to save often, because the window you are working in will vanish every couple of hours.

Bug #1: Can’t recover files that weren’t saved

I was unlucky enough to have forgotten to save my work when it crashed today, and found this nasty flaw in the auto-recover feature:

Auto-save model interval: An open model that has not been saved will automatically be saved after this period. On loading a model file, MySQL Workbench will notify the user if the file was not previously saved correctly, due to a crash or power failure. MySQL Workbench can then attempt to recover the last auto-saved version. For automatic recovery to be available for a new file, it will have to have been saved at least once by the user.

Uh oh! The file hadn’t been saved yet, so it’s gone right? According to wb_model_file.cpp, this is not the case. The auto-save file is always written, but the recovery process wont be started until you try to use it again (which will never happen if you don’t have an old saved version):

/* Auto-saving
 *
 * Auto-saving works by saving the model document file (the XML) to the expanded document folder
 * from time to time, named as document-autosave.mwb.xml. The expanded document folder is
 * automatically deleted when it is closed normally.
 * When a document is opened, it will check if there already is a document folder for that file
 * and if so, the recovery function will kick in, using the autosave XML file.
 */

So under ~/.mysql/workbench/, I found a newmodel.mwbd folder. Workbench files are .zip files in disguise, so I compared it to a test file. It had all the same content, but with a document-autosave.xml, rather than a document.xml (see test file below):

Test archive

Cool, so I’d just rename the file, compress the whole lot and make it a .mwb? No such luck.

Bug #2: File Roller can’t compress ‘@’ files

Possibly because of the -@ command-line option in the zip command, File Roller refused to work with these files.

File Roller bug

Luckily, the document.mwb.xml file alone is enough for the file to be recognised and recovered from the auto-saved files:

File is recovered

The take-away from this? Save your work. In 2014, you still can’t count on auto-save to do this!

USB Missile Launcher

Back in February I coded up a userspace driver to control a USB missile launcher manufactured by DreamCheeky. The video below shows one of the example programs in action.

 

The code being executed in the video is from basic-sync.cpp, the simplest demonstration I could think of:

Missile *launcher = new Missile(launcherHandle);

launcher -> async = false;
launcher -> move(ML_DOWN, 1000);
launcher -> move(ML_UP, 1000);
launcher -> move(ML_LEFT, 1000);
launcher -> move(ML_RIGHT, 1000);
launcher -> fire();

delete launcher;

The USB driver uses libusb, and was coded in response to this trivial bug not being fixed in the Ubuntu repositories for over a year.

qtHiero: Open-source Egyptian hieroglyph editor

I’m just starting out with Qt4 and C++ and came up with this semi-useful little tool for marking up Egyptian hieroglyphs in MdC.

So far the only annoying Qt-quirk I’ve found is the lack of support for non-BMP unicode characters in the QChar type. Turns out you need to use a QString with two QChars, which is exactly the situation which QChar is supposed to solve (by being larger than 8 bits so that there is a 1-1 correspondence between written characters and QChars in a string).

The unfortunate hack I had to put in for fetching a hieroglyph from a codepoint looked like this:

/**
 * Return a QString from a unicode code-point
 **/
QString MainWindow :: unicode2qstr(uint32_t character) {
	if(0x10000 > character) {
		/* BMP character. */
		return QString(QChar(character));
	} else if (0x10000 <= character) {
		/* Non-BMP character, return surrogate pair */
		unsigned int code;
		QChar glyph[2];
		code = (character - 0x10000);
		glyph[0] = QChar(0xD800 | (code >> 10));
		glyph[1] = QChar(0xDC00 | (code & 0x3FF));
		return QString(glyph, 2);
	}
	/* character > 0x10FFF */
	return QString("");
}

The Qt developer tools get a 10/10 from me though. I say this mainly because glade runs like a slug at the best of times.

Making an XKCD-style password generator in C++

I’m learning C++ at the moment, and I don’t find long tutorials or studying the standard template library particularly fun.

Making this type of password-generator is not new, but it is a nice practical exercise to start out in any language.

1. Get a list of common English words

Googling “common English words” yielded this list, purporting to contain 5,000 words. Unfortunately it contains almost 1,000 duplicates and numerous non-words! Wiktionary has a much higher-quality list of words compiled from Project Gutenberg, but the markup looks a bit like this:

==== 1 - 1000 ====
===== 1 - 100 =====
[[the]] = 56271872
[[of]] = 33950064
[[and]] = 29944184
[[to]] = 25956096
[[in]] = 17420636
[[I]] = 11764797  

Noting the wikilinks surrounding each word, I put together this PHP script to extract the link destinations and called it get-wikilinks.php:

#!/usr/bin/php
<?php
/* Return list of wikilinked words from input text */
$text = explode("[[", file_get_contents("php://stdin"));
foreach($text as $link) {
	$rbrace = strpos($link, "]]");
	if(!$rbrace === false) {
		/* Also escape on [[foo|bar]] links */
		$pipe = strpos($link, "|");
		if(!$pipe === false && $pipe < $rbrace) {
			$rbrace = $pipe;
		}
		$word = trim(substr($link, 0, $rbrace))."n";
		if(strpos($word, "'") === false && !is_numeric(substr($word, 0, 1))) {
			/* Leave out words with apostrophes or starting with numbers */
			echo $word;
		}
	}
}

The output of this script is much more workable:

$ chmod +x get-wikilinks.php
$ cat wikt.txt | ./get-wikilinks.php
the
of
and
to
in
I

Using sort and uniq makes a top-notch list of common words, ready for an app to digest:

$ cat wikt.txt | ./get-wikilinks.php | sort | uniq > wordlist.txt

2. Write some C++

There are two problems being solved here:

  • Reading a file into memory
    • An ifstream is used to access the file, and getline() will return false when EOF has been reached
    • Each line is loaded into a vector (roughly the same type of container as an ArrayList in Java), which is resized dynamically and accessed like an array.
  • Choosing random numbers
    • These are seeded from a random_device, being more cross-platform than reading from a file like /dev/urandom.
    • Note that random is new to C++11.
pw.cpp
#include <fstream>
#include <vector>
#include <string>
#include <iostream>
#include <random>
#include <cstdlib>

using namespace std;

int main(int argc, char* argv[]) {
    const char* fname = "wordlist.txt";

    /* Parse command-line arguments */
    int max = 1;
    if(argc == 2) {
        max = atoi(argv[1]);
    }

    /* Open word list file */
    ifstream input;
    input.open(fname);
    if(input.fail()) {
        cerr << "ERROR: Failed to open " << fname << endl;
    }

    /* Read to end and load words */
    vector<string> wordList;
    string line;
    while(getline(input, line)) {
        wordList.push_back(line);
    }

    /* Seed from random device */
    random_device rd;
    default_random_engine gen;
    gen.seed(rd());
    uniform_int_distribution<int> dist(0, wordList.size() - 1);

    /* Output as many passwords as required */
    const int pwLen = 4;
    int wordId, i, j;
    for(i = 0; i < max; i++) {
        for(j = 0; j < pwLen; j++) {
            cout << wordList[dist(gen)] << ((j != pwLen - 1) ? " " : "");
        }
        cout << endl;
    }

    return 0;
}

3. Compile

Lots of projects in compiled languages have a Makefile, so that you can compile them without having to type all the compiler options manually.

Makefiles are a bit heavy to learn properly, but for a project this tiny, something simple is fine:

default:
	g++ pw.cpp -o pw -std=c++11

clean:
	rm -f pw

Now we can compile and run the generator:

make
./pw

The output looks like this for ./pw 30 ("generate 30 passwords"):