kvm – Mike's Software Blog

I recently built a compact, quiet rackmount NAS for home. I haven’t seen any builds quite like it online, so I’m writing a bit about how it came together.

The problem

My old backup was a mirrored pair of 2 TB hard drives in an old desktop computer, with a portable 2 TB hard drive as an off-site copy. The disks are now 9 years old, and 2 TB is small enough that I need to ration the space. I also recently repurposed most of the components in that system, so I no longer have an up-to-date backup.

I want to solve this properly, and hopefully build a replacement setup which I can install in my network cabinet, to run 24/7.

But if I’m going to do that, the must-haves are:

It fits in a 1U rack-mount form-factor with maximum 25cm depth
It’s quiet
It has front USB for making an offline copy of the backups
It has 2 SATA disks – I don’t want to be running a NAS off USB-SATA adapters, for example
It has wired ethernet – my network is 1 Gigabit

Nice-to-haves would be:

Fast front USB
Hot-swap disks or more disks
Faster network

I’ve worked with servers, and I’ve built small-form-factor computers, so how hard can it be to build a small-form-factor server?

Parts list

I spent a lot of time sketching out possible builds in LibreOffice Draw. Every combination of parts had some compromise, which is a familiar theme from small-form-factor PC building.

I ended up deciding on parts which follow normal PC standards, hopefully giving me a good chance of keeping it working for many years.

Case: Case Athena Power RM1UC138
Power supply: HDPLEX GAN 250W
Motherboard: Topton N6005 Mini ITX
RAM: Crucial 16GB (2 x 8GB) DDR4 3200 SO-DIMM
Boot disk: Samsung 970 EVO Plus 1TB M.2 SSD
Storage disks: 2 x Samsung 870 QVO 8TiB 2.5″ SSD
Fans: 4 x Noctua A4x20 PWM
Drive cage: Icy Dock MB608SP-B – 6 x 2.5″ SATA bays
Set of 6 x 50cm SATA cables

There are some compatibility issues in the above part list, and it took a little bit of problem-solving to get everything working together.

Closer look at the Athena Power RM1UC138

Short-depth 1U computer cases are nearly impossible to find in Australia. I ordered the Athena Power RM1UC138 from the United States, which is an OEM case with some flexibility built in.

On the front it has a 5.25″ bay, which I plan to use to add a drive cage. The front USB is only USB 2.0, which would normally be a disadvantage, but in this instance is a good match for the motherboard I’m using. The power button looks like a toggle switch, but is actually a momentary switch.

The side shows that the rack ears can be put on the front or back, allowing the case to be mounted in either direction.

On the back there is a wire mesh panel for cutting out a custom I/O shield, which is handy since I have an off-brand motherboard.

On the inside, it’s configured for 2 x 3.5″ hard drives by default. It also includes 2 x 2-pin 12v fans. They are loud like you would find in most managed network switches, but are not jet-engine loud like most servers. The fan controller simply distributes 12v, and has no speed control.

This is not a very common case, and I read everything I could find online about it. In order to help the next person who is searching for it, here are two more random pieces of information which I could not confirm until I had the case in-hand:

Stand-offs on the case are all 4mm tall and non-removable
Screw spacing of USB 2.0 front panel connector is approx 30.5mm (from centre of each screw). Stacked USB 3 headers that are 30mm spacing are available online and could be made to work.

Closer look at the Topton N6005 Mini ITX

I chose to use a Topton motherboard with a built-in Intel N6005 CPU for this build, since the alternatives were either too tall, use a socketed CPU, use an old CPU, would require add-in cards to get multiple SATA ports, or were not sold in Australia. All of these would make it far more difficult to complete the small, quiet build which I was aiming for.

From the few threads online about this board, I gathered that it is fussy about RAM compatibility, so I booted it up at the first opportunity with an SSD containing Pop!_OS to check that it worked. It’s not my use case, but this motherboard would definitely be viable as a lightweight desktop.

The specific memory I used was a 16 GB kit with the model number CT2K8G4SFRA32A. According to Intel’s product documentation, the N6005 only supports 16 GB maximum, and while I could find claims that higher-capacity memory does work, I couldn’t find anybody who posted actual part numbers.

I was happy to find that the built-in cooler is inaudible at idle loads. This was a bit of a risk: the cooler doesn’t have standard dimensions, so I couldn’t have easily replaced it with an alternative if it was noisy. Based on other people’s experiences with this board, I re-pasted the cooler with Noctua NT-H1 thermal paste, to hopefully help keep temperatures down at higher loads so that the fan will not need to spin up as much. I also also avoided using the M.2 slot which receives the most hot air from the cooler.

Topton also sells an N5105 variant which appears to be more popular (more info here), as well as an alternative layout which has a PCIe slot instead of a second M.2 slot.

Custom power supply adapter plate

The case is designed for a Flex ATX power supply, which is not what I’m using. I’ve instead opted to use a passively-cooled HDPLEX GAN 250W power supply, which ships with both an IEC C6 and IEC C14 cable.

I needed to choose one of these cables, and figure out how to securely mount it to the case.

I designed an adapter plate in FreeCAD around the included IEC C6 cable, since it had threaded holes, and screws were included.

I ordered it from a prototype supplier in laser cut 1mm steel, painted in matte black.

This is the first time I’ve used FreeCAD to design a part, and parametric CAD certainly has a learning curve. For this build, it was well worth it, since the result is better (and safer) than anything I could have improvised.

At the time of writing this post, HDPLEX sells plates for mounting their IEC C14 cables in cases accepting SFX and ATX power supplies, but none for cases which accept Flex ATX power supplies.

Custom fan controller

Cooling this build quietly was always going to be a challenge. The case shipped with 2 x 12 V fans, and had a simple splitter which ran them at max speed, which was just too loud.

I designed a replacement fan controller in KiCad, which allows me to upgrade to high-quality 4-pin fans with PWM speed control, and to set the speed using a potentiometer. I wrote about prototyping this in a separate blog post.

This photo shows the custom controller alongside the original one it replaces.

As you can probably guess, I’ve designed this to use the same mounting location, at the front of the case. My power supply has no Molex connectors, so I’m using a SATA-Molex power adapter.

The main drawback to this simple design is that once I close up the case, I can no longer adjust the fan speed.

Final assembly

Before continuing any further, I took apart the case completely and deleted three standoffs with a belt sander, to leave a flat area for power supply installation later.

Once I got the case back together, the motherboard went in first. I raised it by 1mm using plastic washers, hoping to line it up better with the I/O shield included with the motherboard.

The I/O opening for 1U servers is narrower than standard PC builds, so I needed to cut the I/O shield, which I unfortunately did not do correctly.

Since that did not work, I carefully marked and cut out the wire mesh I/O shield included with the case instead. I still left the motherboard raised up on 1mm washers, though this is not necessary anymore. You need to use slightly longer screws if you try this.

After that I installed the four case fans, plus the fan controller. I’m using a front-to-back airflow, with 2 x 40mm fans mounted at the front, and 2 x 40mm fans at the back. I added a Y splitter to the back fans, which did not have long enough cables to reach the fan controller.

The next component I installed was the drive cage. It’s worth mentioning at this point that the drive cage also has a fan header, which is the same as a 3-pin header that you would find on a PC motherboard. It supplies a different voltage for each of the speed settings. Medium is approximately 7.5 volts and is relatively quiet with the included “Good quality DC fan” fan, and high speed is 12 volts. I set mine to off but left the fan installed.

To install items into the 5.25″ bay in this case, you attach a bracket, then fasten it from above. The bracket allows the depth to be adjusted as well.

I also installed disks in the drive cage at this point, and numbers on the front. Disk 1 is connected to SATA0 on the motherboard, disk 2 is connected to SATA1, and so on.

Next was the power supply. I installed the custom plate for the power connector, and also installed the mounting plate on the bottom of the PSU so that it would have a flat surface. After confirming that it would fit, I cleaned both surfaces with alcohol, and applied double-sided tape.

I then followed a rehearsed path to drop the PSU into place. There is no opportunity to adjust it once it sticks.

At this point I connected everything up and booted up the system to start checking for problems, since it’s easier to troubleshoot in this state. Two modifications I made here were to disconnect the bright red HDD LED, and to introduce a SATA power Y splitter, because the power supply SATA cables were stretched to the limit.

It took a lot of work (and cable ties) to arrange the cables flat so that the case could close. In defence of cable ties, they do make maintenance more difficult, but that’s a worthwhile trade-off for keeping cables clear of airflow paths, fan blades, and the guillotine-like action of the top cover sliding shut.

Completed build

After closing the case, the build is, 434mm × 254mm x 44mm, or 4.8 litres, excluding rack ears.

This is how it appears from the front.

And this is how it appears from the back.

Software

I’m starting with Proxmox, with OpenMediaVault deployed as a virtual machine. I haven’t used either of these before, but both are Debian-based and provide convenient web front-ends to the tools I would otherwise be configuring on the command-line.

I’m passing through the disks as block devices. Running the NAS like this should make it possible to provision extra workloads which need their own SATA disks in future, or to switch from OpenMediaVault to stock Debian if necessary, all without connecting a monitor.

Within OpenMediaVault, I’ve configured Linux software RAID, with an ext4 filesystem, shared via Samba, and can access that file share over the network.

I’ve enabled some basic power management features such as C-states. The system idles in the range of 12-14 watts measured from the wall, and goes up to 20 watts when moving files around.

I don’t need a lot of disk capacity, so I’ve been able to preserve a useful property of my old setup, where every disk in the system has a full copy of the data, in a format which can be understood by a normal Linux system. This it makes single-disk recovery possible using any surviving disk from the system on practically any computer, and that disk can be from either an offline copy or one of the disks in the RAID mirror.

I haven’t tested the process of making an offline copy of the backup volume, but that will be up next.

Wrap-up

This is possibly the most effort I’ve ever put into a PC build. The only unexpected issue I encountered is how heavy it is, and wont be rack-mounting it until I get some generic rails.

The computer uses a strange mix of parts, but meets my requirements well. I hope that by writing this up, I’ll be providing some useful notes to anybody attempting to build something similar.

This project also gave me a chance to practice my entry-level CAD skills to build something which I’ll actually be using. I find a lot of utility in paper prototyping, and printed each design in 1:1 scale to check the physical dimensions before ordering anything.

For the circuit board, I used a print-out to check each part footprint, as well as the hole locations for fitting it in the case.

As with many of the projects which I blog about, I’ve put the design files up on GitHub. The fan controller is at mike42/fan-controller-athena-power, while the Flex ATX adapter plate is at mike42/flexatx-adapter-hdplex.

libvirt: Migrate a VM from qemu:///session to qemu:///system

kvm libvirt linux sysadmin

September 29, 2016October 12, 2022

In recent versions of the libvirt virtualisation libraries, you to create and manage virtual machines as a regular user, using the qemu:///session connection.

This is great, but the networking is quite limited. I found that machines defined in Gnome Boxes could not speak to each-other, and that libvirt commands for networking were unavailable.

For this reason, I’ve written this quick guide for booting up an existing same VM image under the qemu:///system instance, which is faster than re-installing the machine. Unlike most sorts of migrations, this leaves the disk image at the same location on the same host machine.

There’s many different ways to do VM’s in Linux. This setup will be useful only if you use libvirt/kvm using qcow2 images on Debian. As always, consider doing a backup before trying new things.

Configurations

First, find your virtual machine in virsh, and dump its configuration to a text file in your home directory, as a regular user.

$ virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     foo-machine                    shut off
$ virsh dumpxml foo-machine > foo-machine.xml

Now remove the VM definition from your user:

$ virsh undefine foo-machine
Domain foo-machine has been undefined

Import the definitions into virsh as the root user:

$ sudo virsh define foo-machine.xml 
Domain foo-machin defined from foo-machine.xml

Attempt to start the new VM definition. Depending on where the disk image is, expect an error.

$ sudo virsh start foo-machine

Disk images

The disk image needs to be accessible to the libvirt-qemu user. There’s two basic ways to achieve this: Re-permission the directories above it, or move it.

I chose to just re-permission it, since it’s not an issue to have world-readable directories on this particular box:

$ cat foo-machine.xml | grep source
      <source file='/home/example/.local/share/gnome-boxes/images/foo-machine'/>

This one-liner outputs the commands to run to make a directory work/navigable:

$ dir=`pwd`; while [ "$dir" != "/" ]; do echo "chmod o+x,g+x \"$dir\""; dir=`dirname $dir`; done
chmod o+x,g+x "/home/example/.local/share/gnome-boxes/images"
chmod o+x,g+x "/home/example/.local/share/gnome-boxes"
chmod o+x,g+x "/home/example/.local/share"
chmod o+x,g+x "/home/example/.local"
chmod o+x,g+x "/home/example"
chmod o+x,g+x "/home"

And the user account needs to be able to write as well:

$ sudo chown libvirt-qemu /home/example/.local/share/gnome-boxes/images/foo-machine

Once you have the permissions right, the VM should start, using the same command as before:

$ sudo virsh start foo-machine

More importantly, you can now hook up virt-manager and view your machine on qemu:///system, allowing you to configure the VM with any network settings you need.

How to resize a Windows VM image with virt-resize

backup debian kvm windows

March 10, 2016May 3, 2016

I recently had a Windows 7 Virtual Machine stored on an undersized qcow2 file. This post steps through the simplest way that I know to produce a new, bigger disk and expand the filesystem onto it.

Empty out the empty space

Because the guest VM is stored on a QCOW2 file, we can recover un-used space on disk by zeroing it out now. Download the sdelete utility from Microsoft and run it on the system.

sdelete -z

One this is done, power off the guest.

Assuming the host is linux, you need the qemu-utls and libguestfs-tools packages to follow these steps. On Debian-

apt-get install libguestfs-tools qemu-utls

Move the VM image to a new filename and inspect it.

mv windows.img windows.img.bak

The file command indicates that the disk is about 30GB expanded.

$ file windows.img.bak 
windows.img.bak: QEMU QCOW Image (v3), 32212254720 bytes

The qemu-img command shows that the disk is 83% full:

$ qemu-img check windows.img.bak 
No errors were found on the image.
411337/491520 = 83.69% allocated, 5.66% fragmented, 0.00% compressed clusters
Image end offset: 26961969152

Check out your FS names, note that /dev/sda2 is the disk we want to up-size in this case

$ virt-filesystems -a windows.img -l
Name       Type        VFS   Label            Size         Parent
/dev/sda1  filesystem  ntfs  System Reserved  104857600    -
/dev/sda2  filesystem  ntfs  -                32105299968  -

Make a new, bigger disk image

Create a new disk of the desired size. In my case, 50G is sufficient:

$ qemu-img create -f qcow2 windows.img 50G
Formatting 'windows.img', fmt=qcow2 size=53687091200 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16

The file command shows that this new empty disk image is larger than the old image.

$ file windows.img
windows.img: QEMU QCOW Image (v3), 53687091200 bytes

Copy the old disk to the new one. The --expand option names a partition which will be grown to fill the extra space.

virt-resize --expand /dev/sda2 windows.img.bak windows.img

The virt-resize command shows a progress bar while it works, and zero-blocks will be reclaimed as a result of the output format:

The final line of output suggests holding on to your backup until you’ve checked it, which is wise:

Resize operation completed with no errors. Before deleting the old disk,
carefully check that the resized disk boots and works correctly.

Check that the new disk is valid and contains partitions at the expected size:

$ virt-filesystems -a windows.img -l
Name       Type        VFS   Label            Size         Parent
/dev/sda1  filesystem  ntfs  System Reserved  104857600    -
/dev/sda2  filesystem  ntfs  -                53579939840  -

Boot up the guest

When the machine boots up, you may get a disk check prompt. Because the console I was using triggered the ‘Press any key to cancel’ prompt, I had to reboot and leave the console disconnected in order for the check to start.

After booting, the C:\ drive should display at its new size:

Crash course: Run Windows on desktop Linux

howto kvm linux windows

June 9, 2014November 5, 2015

Sometimes, you need to use a tricky windows-only proprietary program on a GNU/Linux desktop. If you have a Windows install disk and licence at your disposal, then this post will show you how to get a Windows environment running without dual-booting.

The host here is a Debian box, and the guest is running Windows 7. The instructions will work with slight modifications for any mix of GNU/Linux and Windows

On the desktop, some things are not as important as the server world. Some things are excluded for simplicity: network bridging, para-virtualised disks, migration between hosts, and disk replication.

Software setup

Everything required from the host machine can be pulled in via Debian’s qemu-kvm package.

sudo apt-get install qemu-kvm

Install

Prepare a disk image for Windows. The qcow2 format is suggested for the desktop as it will not expand the file to the full size until the guest uses the space:

qemu-img create -f qcow2 windows.img 30G

Launch the Windows installer in KVM with a command that looks something like this:

kvm -hda windows.img --cdrom windows-install-disc.iso -vga std -localtime -net nic,model=ne2k_pci -m 2048

Note the -m option is the number of megabytes of RAM to allocate. You can set it a little lower if you don’t have much to spare, but if it’s too low you’ll get this screen:

If you have a physical disk but no .iso of it, then using the disk drive via --cdrom /dev/cdrom will work.

Install

If you have GNU/Linux, chances are you have installed an OS before. In case you haven’t seen the Windows 7 installer, the steps are below:

Select language, accept the licence agreement, choose the target disk, and let the files copy:

After reboot, enter the user details, licence key, update settings and timezone:

After another reboot, Windows is installed in the virtual machine:

Post-install tasks

The guest you have now will only run at standard VGA resolutions, and will probably not be networked. This section will show you how to fix that.

Network drivers

You will notice that we are launching the guest with -net nic,model=virtio. This means that we are using a virtual network card, rather than simulating a real one. You need to fetch a disk image with the latest binary drivers, which are best tracked down on linux-kvm.org via google.

Once you have the disk image in the same folder as your virtual machine, shut down and launch it with a CD:

kvm -hda windows.img --cdrom virtio-win-0.1-74.iso -vga std -localtime -net nic,model=ne2k_pci -m 2048

Under "My Computer" track down the "Device Manager", find your network card, and tell Windows to update the drivers. You can then point it to the CDROM’s "Win7" directory (or other one, if you are installing a different guest). After the network adapter is recognised, you will be connected automatically.

Note that you are using "user-mode" networking, which means you are on a simulated subnet, and can only communicate via TCP and UDP (ie, ping will not work). This can be a little slow, but will work on a laptop host whether plugged in or running on WiFi.

Remote desktop

You may also be annoyed by the screen resolution and mouse sensitivity having strange settings. The best way around this is not to fiddle with settings and drivers, but to enable remote desktop and log in via the network. This lets you use an arbitrary screen size, and match mouse speed to the host.

This is set up to run locally, so it is neither laggy nor a security issue, and makes it possible to leverage all RDP features.

First, in the guest, enable remote desktop using these Microsoft instructions.

Then shut down and boot up with the extra -redir tcp:3389::3389 option:

kvm -hda windows.img -vga std -localtime -net nic,model=ne2k_pci -m 2048 -redir tcp:3389::3389

On the host, wait for the guest to boot, then use rdesktop to log in:

rdestkop localhost

One this works, you can shut down and boot with the extra -nographic option to leave remote desktop as the only way to interact with the guest:

kvm -hda windows.img -vga std -localtime -net nic,model=ne2k_pci -m 2048 -nographic -redir tcp:3389::3389

The rdesktop tool supports sound, file and printer redirection. It can also run fullscreen when launched with -f. All the details are in man rdesktop

If you end up using the guest operating system more, it is worth investigating USB-redirection for any peripherals (printers or mobile phones), running a virtual sound card, or running SAMBA on the host to share files.

A tour of ReactOS 0.3.15

debian kvm linux reactos

February 1, 2014November 5, 2015

ReactOS is a project which aims to create an open source operating system which is binary-compatible with Windows. Although it is still cautiously labelled “alpha”, its basic use is about as reliable as Windows once was.

This post runs through the steps to install ReactOS 0.3.15 as a KVM guest on Linux.

Preparation

Before attempting anything, check that you a CPU supports Intel VT or AMD-V. This command will return the number of CPU cores with svm or vmx flags:

cat /proc/cpuinfo | grep -E 'svm|vmx' | wc -l

Now download the ReactOS 0.3.15 disk from reactos.org, extract it to get the .iso, and fetch some packages if you don’t have them installed:

apt-get install libvirt-bin kvm qemu-utils

Prepare a disk image to install to. If your hardware is slower, then a raw image is a better idea than the qcow2:

qemu-img --help
qemu-img create -f qcow2 reactos.img 4G

The working directory now has:

mike@mikebox:~/vm/reactos$ ls -Ahl
total 77M
-rw-r--r-- 1 mike mike  77M May 19  2013 ReactOS-BootCD.iso
-rw-r--r-- 1 mike mike 193K Jan 30 21:05 reactos.img

Installation and first boot

The kvm command will pop up a window with the guest operating system. To boot from the install disk, run:

kvm -hda reactos.img --cdrom ReactOS-BootCD.iso -vga std -localtime -net nic,model=ne2k_pci -net user

The meaning of each of these options is:

-hda reactos.img: Sets the HDD image file.
--cdrom ReactOS-BootCD.iso: Sets the CDROM image file. Because reactos.img is blank, this will boot.
-vga std: Sets the VGA card.
-localtime: Emulates a system clock in local time, rather than UTC.
-net nic,model=ne2k_pci: Sets the network card to something ReactOS will recognise.
-net user: Enables user-mode networking. Your computer will emulate a network and pass on TCP and UDP connections. This is the easiest mode to use, but ICMP packets (such as pings) will not work, and the VM will not be accessible from other computers.

Installation was fast, error-free, and did not require a network connection. The first screen capture below was taken at 16:04:49, and the desktop was captured at 16:06:07 (1 minute 18 seconds later). Most of that time would have been wasted waiting for user input.

After copying files, the installer reboots to a more user-friendly mode (similar to the Windows installer):

ReactOS Install - Welcome to the ReactOS Setup Wizard

ReactOS Install - Personalize your Software

ReactOS Install - Computer name and Administrative Password

ReactOS Install - Registering Components

ReactOS Install - Completing the ReactOS Setup Wizard

The installed system

After installation, the --cdrom option can be dropped:

kvm -hda reactos.img -vga std -localtime -net nic,model=ne2k_pci -net user

The first thing I did was correct the colour depth, and then attempt to install VLC. This did not turn out well (the console screen is QEMU-monitor):

VLC installation in ReactOS Applications Manager

I used command prompt to verify that networking was fine (note the lack of ICMP in user-mode networking):

Ping from command prompt with user-mode networking as KVM guest

The Firefox 22 install worked, but it went awry after that. Several reboots later I gave up:

The built-in programs were much more usable:

ReactOS Start Menu showing Administrative Tools

PuTTY installed flawlessly, and I was able to SSH to the host computer:

Using command prompt to find host computer address:

PuTTY connected to host computer, showing 'uname -a' output

An example of a frozen program causing graphics glitches (Windows up to XP does this as well):

And an obligatory screenshot of the “Properties for System” dialog, showing the build as 20130518-r59037:

Conclusion

ReactOS is a cool idea and project, but the OS is still very glitchy. The built-in apps are stable and familiar-looking, but you would require a lot of patience (and a lot of rebooting) to use a ReactOS system for more than a few minutes.

Being open source is a big plus, as there is no need to activate the installation or enter software keys. GNU/Linux users will already be accustomed to this.

Tag: kvm

Building a 1U quiet NAS

The problem

Parts list

Closer look at the Athena Power RM1UC138

Closer look at the Topton N6005 Mini ITX

Custom power supply adapter plate

Custom fan controller

Final assembly

Completed build

Software

Wrap-up

libvirt: Migrate a VM from qemu:///session to qemu:///system

Configurations

Disk images

How to resize a Windows VM image with virt-resize

Empty out the empty space

Make a new, bigger disk image

Boot up the guest

Crash course: Run Windows on desktop Linux

Software setup

Install

Install

Post-install tasks

Network drivers

Remote desktop

A tour of ReactOS 0.3.15

Preparation

Installation and first boot

The installed system

Conclusion