Building an emulator for my 65C816 computer

I’ve been working on adding text-based input and output to my 65C816 computer prototype, but it’s not yet working reliably.

To unblock software development, I decided to put together an emulator, so that I can test my code while the hardware is out of action.

Choosing a base project

There were three main options I considered for running 65C816 programs on a modern computer,

  1. Write a new emulator from scratch
  2. Extract the useful parts of an open source Super Nintendo or Apple IIgs emulator and adapt them
  3. Find a 65C816 CPU emulator, and extend it.

I found a good candidate for the third option in Lib65816 by Francesco Rigoni, which is a C++ library for emulating a 65C816 CPU. The code is GPLv3 licensed, and I could see where I would need to make changes to match the design of my computer.

Process

After reading the code, I could see some places where I would need to modify the library, so I copied it in to a new folder, together with its logging dependency and sample program, and refactored it into one project.

I started by extending RAM up to 16 banks (1 megabyte), and mapping in a ROM device which serves bytes from a file. I also updated the handling of interrupt vectors, so that the CPU would retrieve them from the system bus.

Adding 65C22 support

My first test program was an example of preemptive multi-tasking from my previous blog post. At this point it ran, but did not switch between tasks, since that requires extra hardware support.

I implemented a minimal support for the 65C22 VIA used in my computer design, just enough to log changes to the outputs, and to fire interrupts from a timer.

The Lib65816 library does not support NMI interrupts, which are used in this test program. I added edge detection, and connected the interrupt output of the 65C22 to the CPU NMI input in code.

This screen capture shows the test program during switching between two tasks, with a different VIA port being accessed before and after the switch.

This gave me some confidence that the CPU emulation was good enough to develop on, so I moved on to the next test program.

Adding serial support

The second test program is also from an earlier blog post, and tests printing and reading characters from an NXP SC16C752 UART.

I added a very minimal emulation of the UART chip I’m using, suppressed all of the logging, and converted output to use the curses library. This last step enabled character-by-character I/O, instead of the default line-buffered I/O.

This screen capture shows the program running under emulation. It simply prints “Hello world”, then echoes back the next 3 characters that the user types.

Wrap-up

I didn’t plan to write an emulator for my custom computer, but I am currently debugging some tricky hardware problems, and this detour will allow me to continue to make progress in other areas while I try to find a solution.

I plan to keep the emulator up to date with hardware changes, since it will make it possible to demonstrate the system without any hardware access. It’s also easier to debug issues now that I can choose two different execution environments.

The hardware design, software and emulator for this computer are online at mike42/65816-computer. I would like to again acknowledge Francesco Rigoni’s work on Lib65816. I’m very grateful that he chose to release this under the GPL, since it allows me to re-use the code for my project.

How to compile a C++11 app on Travis CI

I have recently been adding Travis CI builds to code that I host on GitHub, so that I don’t need to host my own build infrastructure.

To users, this just means that there is a green badge at the top of the README, but not much else:

passing.svg

To build a simple C++ project, I added in this .travis.yml file:

langauge: cpp
sudo: false

addons:
  apt:
    packages:
      - libusb-1.0-0-dev

script:
  - make

Unfortunately, on this infrastructure, the default build tools are currently ancient, and installed on Ubuntu Precise (12.04):

$ make
g++ src/missile.cpp examples/basic-sync/basic-sync.cpp -o bin/basic-sync -lpthread -lusb-1.0 -std=c++11 -Wall
cc1plus: error: unrecognized command line option ‘-std=c++11’
cc1plus: error: unrecognized command line option ‘-std=c++11’

Option 1: Update the toolchain

There is some structures you can use to install an extra repository and some named packages, instead of using apt-get directly.

langauge: cpp
sudo: false

addons:
  apt:
    sources:
    - ubuntu-toolchain-r-test
    packages:
    - gcc-4.8
    - g++-4.8
    - libusb-1.0-0-dev

script:
  - make

Because the old version was still installed, I had to refer to the exact version in the Makefile, as in:

g++-4.8 src/missile.cpp examples/basic-sync/basic-sync.cpp -o bin/basic-sync -lpthread -lusb-1.0 -std=c++11 -Wall

Option 2: Update the platform

You can also change to a more recent Ubuntu distribution. Presumably Ubuntu Precise is only the default because existing builds use it.

If you need to build C++11 apps on Travis CI, then builds will work under Ubuntu Trusty (14.04), which happens to be the newest distribution currently available:

langauge: cpp
sudo: required
dist: trusty

addons:
  apt:
    packages:
      - libusb-1.0-0-dev

script:
  - make

How to run Tetris on your Raspberry Pi

This is a simple walkthrough on how to install my Tetris clone, Blocks, on a Raspberry Pi.

On most computers running Debian (or Raspbian in the case of the Raspberry Pi), it’s as simple as clone, compile, run:

sudo apt-get install libncurses5-dev doxygen
git clone https://github.com/mike42/blocks
cd blocks
make
./bin/blocks

If you have any issues running this, then you need to fetch a newer version of GCC, as this needs C++11 support to compule (see last section for instructuins).

But if all goes to plan, you will get something like this in your terminal:

2015-04-tetris

Use the keyboard to control the game:

Move
Right, down, left
Rotate
Up
Drop
Spacebar
Quit
q

Get a screen

Basically any project with graphics can benefit from one of these. Simply add on a TFT shield, such as PiTFT to create a tiny console:

2015-04-tetris

Of course, this is still keyboard-controlled, but with some hacking, I’m sure you could map touch events to keyboard actions.

Troubleshooting: Update GCC

The Raspbian spftware image which many Raspberry Pi’s have is slightly too old to compile Blocks, which requires C++11 support.

Luckily, it’s very easy to upgrade from wheezy to jessie to add it. You know you need to do this if you get this error compiling:

$ git clone https://github.com/mike42/blocks
$ make
mkdir -p bin
g++ src/main.cpp src/blocks_game.cpp src/blocks_shape.cpp -o bin/blocks -lcurses -lrt -std=c++11 -Wall
cc1plus: error: unrecognized command line option ‘-std=c++11’
cc1plus: error: unrecognized command line option ‘-std=c++11’
cc1plus: error: unrecognized command line option ‘-std=c++11’
Makefile:2: recipe for target 'default' failed
make: *** [default] Error 1

Generally this means you don’t have GCC 4.8, which is not available in wheezy edition of Raspian.

$ g++ --version
g++ (Debian 4.6.3-14+rpi1) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

So to summarise this thread, you need to:

nano /etc/apt/sources.list

Find this line:

deb http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi

And change the word “wheezy” to “jessie”:

deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi

You can then update everything with:

sudo apt-get update && sudo apt-get dist-upgrade

You are now running the newer jessie release, which gives you access to the GCC 4.8 package we need:

apt-get install g++-4.8

So we can pick up where we left off, and compile the game:

make
./bin/blocks

A Yes/No filtering tool for images

While collecting very large numbers of screen captures for writing documentation, I noticed that it takes far too long to filter out the junk.

To fix this, I coded up yn (source code via github). Given a list of images, it will display each one to the user, which they then include by pressing Y (yes) or exclude by pressing N (no) (hence the name yn).

The images which are selected are saved as a list, so that a script can continue by processing them in some way. This could be by copying them elsewhere, or generating a document with spaces to caption them. The example in the README.md does both of these.

The code to this early version is very simple, which makes it a good example of a simple OpenCV C++ app. I’ve stepped through it below.

The code

C++ does console input and output via the iostream library:

#include <iostream>

To use OpenCV with a GUI, you need these headers. You then need to add them to your include path, and link to OpenCV for the program to compile:

#include <cv.h>
#include <highgui.h>

This line simly tells the compiler that when we say cout (console out), we mean std::cout.

using namespace std;

The only OpenCV code is in the below function. The steps are:

  1. load an image from a file via imread(),
  2. display it in a window with imshow(),
  3. capture the next keypress with waitKey(), then
  4. delete the window with destroyWindow()
/**
 * Load a file, and wait for the user to press a key.
 *
 * If the pressed key is 'y', print the filename.
 */
bool yn(string fn) {
    cv::Mat img;
    char key = 0;

    img = cv::imread(fn);
    if(!img.data) {
        cerr << "Failed to load " << fn << endl;
    } else {
        cv::namedWindow(fn);
        cv::imshow(fn, img);
        key = cv::waitKey(0);
        cv::destroyWindow(fn);
        if(key == 'y') {
            cout << fn << endl; // 'y' pressed
        } else if(key == 0x1B) {
            return false; // ESC pressed
        }
    }
    return true;
}

So with that library usage out of the way, all we need to do is get the list of files to check, and stop popping up windows when the user has pressed the escape key.

/**
 * Get list of files from command-line arguments and display them in turn
 */
int main(int argc, char** argv) {
    int i;

    /* Command-line arguments given */
    for(i = 1; i < argc; i++) {
        if(yn(string(argv[i])) == false) {
            cerr << "Quitting.\n";
            break;
        }
    }

    return 0;
}

Next

The delay between images appearing can be removed by loading them in a separate thread, which I may do in a future version.

Recovering auto-saved files in MySQL Workbench

MySQL workbench is an open source tool for designing databases. As version 6.0.8, it is one of those programs where you need to save often, because the window you are working in will vanish every couple of hours.

Bug #1: Can’t recover files that weren’t saved

I was unlucky enough to have forgotten to save my work when it crashed today, and found this nasty flaw in the auto-recover feature:

Auto-save model interval: An open model that has not been saved will automatically be saved after this period. On loading a model file, MySQL Workbench will notify the user if the file was not previously saved correctly, due to a crash or power failure. MySQL Workbench can then attempt to recover the last auto-saved version. For automatic recovery to be available for a new file, it will have to have been saved at least once by the user.

Uh oh! The file hadn’t been saved yet, so it’s gone right? According to wb_model_file.cpp, this is not the case. The auto-save file is always written, but the recovery process wont be started until you try to use it again (which will never happen if you don’t have an old saved version):

/* Auto-saving
 *
 * Auto-saving works by saving the model document file (the XML) to the expanded document folder
 * from time to time, named as document-autosave.mwb.xml. The expanded document folder is
 * automatically deleted when it is closed normally.
 * When a document is opened, it will check if there already is a document folder for that file
 * and if so, the recovery function will kick in, using the autosave XML file.
 */

So under ~/.mysql/workbench/, I found a newmodel.mwbd folder. Workbench files are .zip files in disguise, so I compared it to a test file. It had all the same content, but with a document-autosave.xml, rather than a document.xml (see test file below):

Test archive

Cool, so I’d just rename the file, compress the whole lot and make it a .mwb? No such luck.

Bug #2: File Roller can’t compress ‘@’ files

Possibly because of the -@ command-line option in the zip command, File Roller refused to work with these files.

File Roller bug

Luckily, the document.mwb.xml file alone is enough for the file to be recognised and recovered from the auto-saved files:

File is recovered

The take-away from this? Save your work. In 2014, you still can’t count on auto-save to do this!

USB Missile Launcher

Back in February I coded up a userspace driver to control a USB missile launcher manufactured by DreamCheeky. The video below shows one of the example programs in action.

 

The code being executed in the video is from basic-sync.cpp, the simplest demonstration I could think of:

Missile *launcher = new Missile(launcherHandle);

launcher -> async = false;
launcher -> move(ML_DOWN, 1000);
launcher -> move(ML_UP, 1000);
launcher -> move(ML_LEFT, 1000);
launcher -> move(ML_RIGHT, 1000);
launcher -> fire();

delete launcher;

The USB driver uses libusb, and was coded in response to this trivial bug not being fixed in the Ubuntu repositories for over a year.

qtHiero: Open-source Egyptian hieroglyph editor

I’m just starting out with Qt4 and C++ and came up with this semi-useful little tool for marking up Egyptian hieroglyphs in MdC.

So far the only annoying Qt-quirk I’ve found is the lack of support for non-BMP unicode characters in the QChar type. Turns out you need to use a QString with two QChars, which is exactly the situation which QChar is supposed to solve (by being larger than 8 bits so that there is a 1-1 correspondence between written characters and QChars in a string).

The unfortunate hack I had to put in for fetching a hieroglyph from a codepoint looked like this:

/**
 * Return a QString from a unicode code-point
 **/
QString MainWindow :: unicode2qstr(uint32_t character) {
	if(0x10000 > character) {
		/* BMP character. */
		return QString(QChar(character));
	} else if (0x10000 <= character) {
		/* Non-BMP character, return surrogate pair */
		unsigned int code;
		QChar glyph[2];
		code = (character - 0x10000);
		glyph[0] = QChar(0xD800 | (code >> 10));
		glyph[1] = QChar(0xDC00 | (code & 0x3FF));
		return QString(glyph, 2);
	}
	/* character > 0x10FFF */
	return QString("");
}

The Qt developer tools get a 10/10 from me though. I say this mainly because glade runs like a slug at the best of times.

Making an XKCD-style password generator in C++

I’m learning C++ at the moment, and I don’t find long tutorials or studying the standard template library particularly fun.

Making this type of password-generator is not new, but it is a nice practical exercise to start out in any language.

1. Get a list of common English words

Googling “common English words” yielded this list, purporting to contain 5,000 words. Unfortunately it contains almost 1,000 duplicates and numerous non-words! Wiktionary has a much higher-quality list of words compiled from Project Gutenberg, but the markup looks a bit like this:

==== 1 - 1000 ====
===== 1 - 100 =====
[[the]] = 56271872
[[of]] = 33950064
[[and]] = 29944184
[[to]] = 25956096
[[in]] = 17420636
[[I]] = 11764797  

Noting the wikilinks surrounding each word, I put together this PHP script to extract the link destinations and called it get-wikilinks.php:

#!/usr/bin/php
<?php
/* Return list of wikilinked words from input text */
$text = explode("[[", file_get_contents("php://stdin"));
foreach($text as $link) {
	$rbrace = strpos($link, "]]");
	if(!$rbrace === false) {
		/* Also escape on [[foo|bar]] links */
		$pipe = strpos($link, "|");
		if(!$pipe === false && $pipe < $rbrace) {
			$rbrace = $pipe;
		}
		$word = trim(substr($link, 0, $rbrace))."n";
		if(strpos($word, "'") === false && !is_numeric(substr($word, 0, 1))) {
			/* Leave out words with apostrophes or starting with numbers */
			echo $word;
		}
	}
}

The output of this script is much more workable:

$ chmod +x get-wikilinks.php
$ cat wikt.txt | ./get-wikilinks.php
the
of
and
to
in
I

Using sort and uniq makes a top-notch list of common words, ready for an app to digest:

$ cat wikt.txt | ./get-wikilinks.php | sort | uniq > wordlist.txt

2. Write some C++

There are two problems being solved here:

  • Reading a file into memory
    • An ifstream is used to access the file, and getline() will return false when EOF has been reached
    • Each line is loaded into a vector (roughly the same type of container as an ArrayList in Java), which is resized dynamically and accessed like an array.
  • Choosing random numbers
    • These are seeded from a random_device, being more cross-platform than reading from a file like /dev/urandom.
    • Note that random is new to C++11.
pw.cpp
#include <fstream>
#include <vector>
#include <string>
#include <iostream>
#include <random>
#include <cstdlib>

using namespace std;

int main(int argc, char* argv[]) {
    const char* fname = "wordlist.txt";

    /* Parse command-line arguments */
    int max = 1;
    if(argc == 2) {
        max = atoi(argv[1]);
    }

    /* Open word list file */
    ifstream input;
    input.open(fname);
    if(input.fail()) {
        cerr << "ERROR: Failed to open " << fname << endl;
    }

    /* Read to end and load words */
    vector<string> wordList;
    string line;
    while(getline(input, line)) {
        wordList.push_back(line);
    }

    /* Seed from random device */
    random_device rd;
    default_random_engine gen;
    gen.seed(rd());
    uniform_int_distribution<int> dist(0, wordList.size() - 1);

    /* Output as many passwords as required */
    const int pwLen = 4;
    int wordId, i, j;
    for(i = 0; i < max; i++) {
        for(j = 0; j < pwLen; j++) {
            cout << wordList[dist(gen)] << ((j != pwLen - 1) ? " " : "");
        }
        cout << endl;
    }

    return 0;
}

3. Compile

Lots of projects in compiled languages have a Makefile, so that you can compile them without having to type all the compiler options manually.

Makefiles are a bit heavy to learn properly, but for a project this tiny, something simple is fine:

default:
	g++ pw.cpp -o pw -std=c++11

clean:
	rm -f pw

Now we can compile and run the generator:

make
./pw

The output looks like this for ./pw 30 ("generate 30 passwords"):