How to use HiDPI displays on Debian 9

I recently added a 4K monitor to my Debian box, and had to set a few things to make it display things at a good size. These high-density moniotors that are becoming common on laptops and desktops are known as “HiDPI” displays.

Currently I get the best results with:

  • Window scaling factor of 2
  • Font scaling 0.90 to make text slightly smaller

Note that “window scaling” is not “upscaling” (stretching an image). In this version of Gnome, it means “single/double/triple DPI”. The implementations are in the process of changing: Soon you should be able to set any scaling factor.

This post assumes a Gnome version around 3.26, which is what you would get as a default if you installed Debian 9 today.

Apply to one user

Under Settings → Devices → Displays, set the Scale to 200%.

Under Tweaks → Fonts, set the Scaling Factor to 0.90.

Next, add these variables to ~/bashrc to apply similar scaling to QT apps.

QT_AUTO_SCREEN_SCALE_FACTOR=0
QT_SCALE_FACTOR=2

Log out and back in to ensure that the settings have applied everywhere.

Apply to any user

If you have a shared system (eg. domain accounts), or want to style the login box as well, then you can set the same settings as below.

These steps are based on answers to the Ask Ubuntu question: Adjust text scaling factor for all users.

nano /usr/share/glib-2.0/schemas/org.gnome.desktop.interface.gschema.xml

Set the text-scaling-factor to 0.9, and the scaling-factor to 2.

<key name="text-scaling-factor" type="d">
  <range min="0.5" max="3.0"/>
  <default>0.9</default>
  <summary>Text scaling factor</summary>
  <description>
    Factor used to enlarge or reduce text display, without changing font si$
  </description>
</key>
<key name="scaling-factor" type="u">
  <default>2</default>
  <summary>Window scaling factor</summary>
  <description>
    Integer factor used to scale windows by. For use on high-dpi screens.
    0 means pick automatically based on monitor.
  </description>
</key>

Re-compile the schemas:

glib-compile-schemas /usr/share/glib-2.0/schemas

Next drop some similar environent variables for QT apps in /etc/profile.d/hidpi.sh to apply it to all users:

export QT_AUTO_SCREEN_SCALE_FACTOR=0
export QT_SCALE_FACTOR=2

After this, reboot. If the setting has applied, then the gdm3 login box will be scaled as well.

How to print Spanish characters on a thermal receipt printer

Today we’re going to send some Spanish text to an Epson TM-T20 thermal receipt printer. This post is intended for people who know how to send ASCII to a similar printer, but can’t get the diacritics from Spanish to displayu properly, characters like á, é, í, ó, ñ, ú or ü.

Many receipt printers understand a language called ESC/POS, which uses legacy code pages for non-ASCII text.

Spanish is very easy to encode manually, because it can be represented fully using just Code Page 437, which happens to be the default code page on most receipt printers.

To convert UTF-8 to CP437, I will use the iconv utility on the Linux command-line. API’s for encoding conversion are available for your favourite programming language:

echo "áéíóñúü" | iconv --from-code=UTF-8 --to-code=CP437

Or, as a more complete example, let’s use a pangram from here:

“Benjamín pidió una bebida de kiwi y fresa; Noé, sin vergüenza, la más exquisita champaña del menú”

echo "Benjamín pidió una bebida de kiwi y fresa; \
Noé, sin vergüenza, la más exquisita champaña del menú" | \
iconv --from-code=UTF-8 --to-code=CP437

Next, let’s send some text to the printer. I have a USB printer on Linux, so this is how I print:

echo "Hello World" > /dev/usb/lp0

And putting it all together:

echo "Benjamín pidió una bebida de kiwi y fresa; \
Noé, sin vergüenza, la más exquisita champaña del menú" | \
iconv --from-code=UTF-8 --to-code=CP437 > /dev/usb/lp0

Libraries for PHP and Python users

I’ve helped out writing the internationalization features for two printing libraries (escpos-php and python-escpos), so that programmers can work with text in UTF-8, and let the library translate the text to something that your printer understand.

The same example above, if you use the PHP library, would look like this:

<?php
require __DIR__ . '/vendor/autoload.php';
use Mike42\Escpos\PrintConnectors\FilePrintConnector;
use Mike42\Escpos\Printer;
$connector = new FilePrintConnector("/dev/usb/lp0");
$printer = new Printer($connector);
$printer -> text("Benjamín pidió una bebida de kiwi y fresa; Noé, sin vergüenza, la más exquisita champaña del menú\n");
$printer -> cut();
$printer -> close();

And in Python, it would look like this:

from escpos import *
my_printer = printer.File("/dev/usb/lp0")
my_printer.text("Benjamín pidió una bebida de kiwi y fresa; Noé, sin vergüenza, la más exquisita champaña del menú\n")
my_printer.cut()

Most of the time you need more than text: Formatting, adding barcodes, or cutting the paper will all need ESC/POS commands, and these libraries will help with that too.

Take a look at the project links for more detail.

How to install PHP Composer as a regular user

Composer is an essential utility for PHP programmers, and allows you to manage dependencies.

Dependencies

You can use your regular account to install composer, use it, and even update it. You do need to have a few packages installed first though:

sudo apt-get install git curl php-cli

Or on Fedora:

sudo dnf install git curl php-cli

Local install

Next, fetch the installer and deploy composer to your home directory

curl https://getcomposer.org/installer > composer-setup.php
mkdir -p ~/.local/bin
php composer-setup.php --install-dir=$HOME/.local/bin --filename=composer
rm composer-setup.php

Last, add ~/.local/bin to your $PATH:

echo 'PATH=$PATH:~/.local/bin' >> ~/.bashrc
source  ~/.bashrc
echo $PATH

You can now run composer:

$ composer --help
Usage:
  help [options] [--] []
...
$ composer self-update
You are already using composer version 1.5.6 (stable channel).

Make Composer available for all users

Just run this line if you decide that all users should have access to your copy of Composer:

sudo mv ~/.local/bin/composer /usr/local/bin/composer

If you look up a how to install Composer, you will find a tempting one-liner that uses curl to fetch a script from the Composer website, then executes it as root. I don’t think it’s good practice to install software like that, so I would encourage you to just run ‘sudo mv’ at the end.

How to boot Debian in 4 seconds

This blog post is a throwback to “Booting Debian in 14 seconds” from debian-administration.org, where the author went through some fairly advanced steps to get his low-spec Debian laptop to boot quickly. Debian was version 4.0 at the time, and I recall it taking around 40 seconds to boot on a default desktop install.

In a rare exception to Wirth’s law, waiting for a computer to boot is no longer “a thing”. A default desktop install of Debian includes systemd, and uses a multi-core CPU and SSD quite efficiently. Also, sleep/wake works more reliably than it used to, so boot speed is not as important as it used to be.

On a modern desktop PC, booting Debian 9 (default desktop install) takes me 14 seconds with no extra configuration, so that’s our new low water mark.

.

Mainly to illustrate how far open source operating systems have come, I’m going to step through a boot process speed-up, the way it looks in 2018.

Summary

Out

You will read about some of these older tricks if you search for Linux Boot speed, and they are all quite irrelevant in 2018, in my humble opinion-

  • Swapping the /bin/sh shell to dash (already the default, also, init scripts are no longer used).
  • Using readahead (gains are tiny unless you have a HDD).
  • noatime” setting on mounts (“relatime” is a default mount option since Linux 2.6).
In

New things that you wont find in pre-systemd guides:

  • systemd-analyze to instrument the boot
  • systemctl to exclude processes from boot
Still relevant
  • bootchart is still useful for drawing pretty graphs
  • Configure GRUB & UEFI not to prompt for input
  • Don’t enable services you don’t need

Process

Remove bootloader delay

Between UEFI and the OS, you will get the bootloader, which will wait for 5 seconds by default to see if you want to select a different item. Start by switching the grub timeout from 5 seconds to 0.

sudo nano /etc/default/grub

Set GRUB_TIMEOUT=0.

Run:

sudo update-grub2
Look at systemd

Use the tool systemd-analyze to draw a picture:

systemd-analyze plot > plot.svg

In my case, it was clear that 9 seconds of the boot was an optional “waiting for network” step.

So, (thank you askubuntu), I disabled that service and rebooted:

$ sudo systemctl disable NetworkManager-wait-online.service
Removed /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service.
systemd-analyze plot > plot2.svg

The boot was still taking 4.4 seconds, so, more analysis was in order:

The systemd-timesyncd service was holding things up.

This service runs early in the boot process, reads an old time from a file, and tries to update time over the network. Since I have a working RTC, this is all unnecessary for me, so I removed it and replaced it with chronyd, which is happy to operate in the background.

sudo systemctl disable systemd-timesyncd.service
sudo apt-get install chronyd
sudo systemctl enable chrony

After another reboot:

systemd-analyze plot > plot4.svg

There we go, down to 4.096 seconds with a few minutes of effort. I think that’s acceptable.

The systemd developers are quite certain that you can boot in under 2 seconds, but I wasn’t willing to customise my system to that extent.

How to use parallel to speed up your work

GNU Parallel is a tool to execute multiple commands at once. In its basic usage, you would list your commands in a file, so that it can execute them, several at a time.

It gives the most benefit on processes that don’t fully utilise your CPU. Almost every laptop, desktop and single board computer now has multiple CPU cores available, so you are probably missing out if you frequently perform batch operations without it.

Installation

On Debian or Ubuntu:

sudo apt-get install parallel
parallel --cite

On Fedora the package name is the same:

sudo dnf install parallel
parallel --cite

Example 1: Convert loops to pipes

Using the ImageMagick tool to convert a folder of GIF images to PNG format can be done in a loop:

for i in *.gif; do convert $i -scale 200% ${i%.*}.png; done

Or, you could print each command in a loop then pass them to parallel.

for i in *.gif; do echo convert $i -scale 200% ${i%.*}.png; done | parallel

The second command is many times faster on a multi-core computer.

Example 2: Replace xargs with parallel

This command executes a single “pngcrush” command on each PNG file in a directory, one at a time.

find . -type f -name '*.png' -print0  | xargs -0 -n1 -r pngcrush -q -ow -brute

To convert this to use parallel, you would use the following command-line:

find . -type f -name '*.png' | parallel "pngcrush -q -ow -brute {}"

Don’t use xargs in parallel mode

Expert command line users will also know about xargs -P, which seems to do the same thing at a glance.

xargs is good at making really long command-lines, and not so good at executing multiple commands at once. It will mix the output of the commands, and requires you to specify the number of jobs to run.

Parallel is designed to do lots of things at once, and it does it well. It will choose some good defaults for the number of processes to execute, and adds an insane collection of features that you need for large batches. To name just a few:

  • Control spawning of new jobs based on things like available memory, system load, or an absolute number of jobs to keep running
  • Distribute jobs to remote computers
  • Show progress
  • Control of when to terminate the jobs

How to generate star fields

I recently needed a texture for the skybox in a 3D space game.

I used this ImageMagick one-liner to generate a dotted canvas with some grey and white pixels. It displays well if the texture will be stretched.

convert -size 1600x900 xc: +noise Random -channel R -threshold 0.5% \
        -negate -channel RG -separate +channel \
        -compose multiply -composite stars.png

Alternative style

With a simple modification, the bright stars are made bigger, and the dull ones are made smaller, with only black and white used. This works well if the texture will be shrunk for display.

convert -size 800x450 xc: +noise Random -channel R -threshold 0.5% \
        -negate -channel RG -separate +channel \
        -compose multiply -composite -resize 200% \
        -threshold 10% stars-rounded.png

Full process

Although these are great one-liners, the actual process is a bit hard to follow without some smaller steps.

Here, we will generate a 150×90 star field in several steps. I’ve scaled each of these pictures to 200% of their original size and converted them to PNG for display on the web. I’ve used BMP in the commands only because it saves some plumbing around colour spaces.

Start with a blank canvas:

convert -size 150x90 xc: stars-01.bmp

Add random RGB noise:

convert stars-01.bmp +noise Random stars-02.bmp

In the red channel of the image, apply a black/white threshold: If the red channel is greter than 0.5%, it is set to the maximum, otherwise it is set to the minimum.

Mostly the red channel is now 100%, with random dots of 0%. The blue and green channels are still completely random:

convert stars-02.bmp -channel R -threshold 0.5% stars-03.bmp

Negate the image, so that the red channel is mostly 0%, with dots of 100%:

convert stars-03.bmp -negate stars-04.bmp

Extract the red channel and green channel.

convert stars-04.bmp -channel RG -separate +channel stars-05.bmp

The red channel will be dots of white:

The green channel will be random:

Multiply the channels together, so that the white dots become grey dots, each with a random brightness:

convert stars-05-0.bmp stars-05-1.bmp -compose multiply -composite stars-06.bmp

For the alternative style, scale the image up, then apply a new threshold. Brighter dots will appear as larger patches of white, while dull stars will be smaller or invisible:

convert stars-06.bmp -resize 200% -threshold 10% stars-06-rounded.bmp

On running deployments via GitHub

TL;DR; – GitHub can be pretty unreliable, depend on it at your peril.

GitHub was down for about 20 minutes today. I happened to be logged in so I’ll share a few screen captures.

The status page and twitter showed no activity for the first nine minutes of the outage, but were then updated with erroneous information.

Meanwhile, the website started displaying unicorns.

If you are logged into GitHub regularly, you might know that this is not a rare event. I don’t have any data on how often GitHub is actually broken, but based on that status page, I’m not sure that know either.

Why I don’t deploy via GitHub

Around May 2013, a website that I maintained started rendering incorrectly because of a bug in my code: I had made some MySQL fields TEXT type, which have a limited size, and part of the application had exceeded the limit, resulting in truncated pages.

This app is written in PHP, and my deployment workflow at the time involved pushing up a change to GitHub, then then triggering a git pull on the server, which ran an update script to bring everything up to speed.

In this case, I was on mobile internet, so I diagnosed the problem and prepared a hotfix on my laptop. When I tried to push it to GitHub, it was offline. I ended up logging in and running a few ALTER TABLE statements over SSH, which is a long shot from the robust deployment pipeline I had envisaged.

There are some emerging SaaS products that offer to deploy directly from GitHub. For example, I use Travis CI extensively for open source, and you can hook it up with deployment keys.

This stuff seems really cool, but I imagine that an error message containing rainbow unicorns would not be very funny if you wanted to fix something in a hurry. This particular app is still deployed with a git pull, but I’ve started to avoid mixing deployment with version control, so that I can run a build and deploy anywhere in case of emergency.

How to use a Radeon graphics card on Debian 9

I have previously blogged about Radeon graphics cards on different Debian installs.

ATI has now released a new free driver which works brilliantly on Debian. In the past, Debian users had to choose between using the community-provided free software driver, or the proprietary one. Generally the proprietary driver was more feature-rich, but the free driver worked more reliably across upgrades. So now, you can safely ignore old guides and start using it.

Here’s how:

Upgrade

Make sure you are on Debian 9 (Stretch) or newer.

These steps apply to a fresh install.

Identify

You should use lspci to confirm that you have an ATI card.

$ lspci | grep Radeon
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970/8970 OEM / R9 280X]

Install firmware

You need to install a package called firmware-linux-free to get the driver working at all. If you want decent graphics performance, you will need firmware-linux-nonfree as well, which involves adding non-free sources:

nano /etc/apt/sources.list

Add the words “contrib non-free” to the end of your mirror:

deb http://.../debian/debian/ stretch main contrib non-free

Add the packages:

apt-get update
apt-get install firmware-linux-free firmware-linux-nonfree

And reboot:

reboot

What, that’s it?

Well, yes, for a fresh install that’s it. If your install is old, you might also have to remove old drivers or install the xserver-xorg-video-amdgpu and xserver-xorg-video-ati packages (in my case, these were already installed).

The Debian Wiki AtiHowto contains some more detailed information, most of which is not relevant for a simple desktop setup.

How to assemble a Linux software RAID array on a different computer

With Linux software RAID, if you ever toast your computer, you can retrieve the disks and open up the array on a different computer.

They appear as “Linux Software RAID Member” in the disk utility.

Simply install mdadm, and scan for arrays:

$ sudo apt-get install mdadm
$ sudo mdadm --assemble --scan
mdadm: /dev/md/0 has been started with 2 drives.

The array will then appear as a new disk, which can be formatted, mounted, or cloned via the usual tools.

Crowd-sourced POS printer compatibility site is online

 

Point-of-sale (POS) printers are all different, so the implementation of the more advanced ESC/POS  features varies considerably between vendors and models.

Some months ago I extracted compatibility information out of the escpos-php library, so that it could be used by the python-escpos team as well. We’re still in the early stages, but both of these projects now have similar compatibility features.

You can the new shared database on GitHub as receipt-print-hq/escpos-printer-db, and I’m beginning to add it to some printing projects that need it.

Today I’m blogging a few screen captures from the new web viewer for this database that has come out today, hosted here (source code here).

escpos-printer-db landing page

escpos-printer-db landing page

Character encodings

escpos-printer-db character encoding list

escpos-printer-db character encoding list

escpos-printer-db character encoding detail

escpos-printer-db character encoding detail

Vendors

escpos-printer-db vendor list

escpos-printer-db vendor list

escpos-printer-db vendor detail

escpos-printer-db vendor detail

Printer profiles

escpos-printer-db printer profile list

escpos-printer-db printer profile list

escpos-printer-db printer profile detail

escpos-printer-db printer profile detail

Tech used

I’ve used materialize.css with backbone.js and jQuery. This is the first time I’ve attempted using materialize – I’m normally a bootstrap.css user.

The back-end data is generated with Python, but the actual site is served as a single-page application, with static JSON files hosting the data.