On running deployments via GitHub

TL;DR; – GitHub can be pretty unreliable, depend on it at your peril.

GitHub was down for about 20 minutes today. I happened to be logged in so I’ll share a few screen captures.

The status page and twitter showed no activity for the first nine minutes of the outage, but were then updated with erroneous information.

Meanwhile, the website started displaying unicorns.

If you are logged into GitHub regularly, you might know that this is not a rare event. I don’t have any data on how often GitHub is actually broken, but based on that status page, I’m not sure that know either.

Why I don’t deploy via GitHub

Around May 2013, a website that I maintained started rendering incorrectly because of a bug in my code: I had made some MySQL fields TEXT type, which have a limited size, and part of the application had exceeded the limit, resulting in truncated pages.

This app is written in PHP, and my deployment workflow at the time involved pushing up a change to GitHub, then then triggering a git pull on the server, which ran an update script to bring everything up to speed.

In this case, I was on mobile internet, so I diagnosed the problem and prepared a hotfix on my laptop. When I tried to push it to GitHub, it was offline. I ended up logging in and running a few ALTER TABLE statements over SSH, which is a long shot from the robust deployment pipeline I had envisaged.

There are some emerging SaaS products that offer to deploy directly from GitHub. For example, I use Travis CI extensively for open source, and you can hook it up with deployment keys.

This stuff seems really cool, but I imagine that an error message containing rainbow unicorns would not be very funny if you wanted to fix something in a hurry. This particular app is still deployed with a git pull, but I’ve started to avoid mixing deployment with version control, so that I can run a build and deploy anywhere in case of emergency.

Leave a Reply

Your email address will not be published. Required fields are marked *