Introduction

I’ve been doing some work on my homelab that I haven’t documented here.

I recently decommissioned my Raspberry Pi 3 that was running my Unifi controller, Dokuwiki, and Network UPS Tools (NUT). I replaced the RPi3 with an Intel i3 NUC with 12GB RAM and a Crucial 2.5" SSD. I chose to use Proxmox as the hypervisor on the NUC because it is open source, has a well-proven record, and has a low learning curve. At the time of this writing, I’m running five KVM virtual machines:

  • mgmt01 - Jump server and Ansible master
  • unifi01 - Unifi controller
  • nginx01 - LEMP stack, running Dokuwiki and NUT
  • vpn01 - Pritunl
  • log01 - Graylog

However, the one thing I’m missing is a backup server.

3-2-1 backup

The golden rule is to follow the 3-2-1 method for backups:

  • 3 backups
  • 2 different types of media
  • 1 backup offsite

Currently, I’m following the 3-2-1 backup method, except the central location is my desktop PC. This is both inconvenient and a waste of space on my desktop. Ideally, I’d like have all the backups go to a NAS-type device, and have that device push the backups to a cloud provider (I already use Backblaze B2).

3-2-1 backup

Devices

I looked at quite a few options, from single-board devices to full NAS devices, but ruled most of them out.

Software Advantage Disadvantage
Raspberry Pi 3 Model B+
  • Cheap
  • Small form-factor
  • Large community and great aftermarket support
  • CPU/memory is limited
  • Fastest storage connection is USB2.0
  • Gigabit ethernet, but bound to USB bus, so max throughput is 300Mbps
Any Raspberry Pi clone (e.g., BananaPi, OrangePi, NanoPi, etc…)
  • Cheap
  • Small form-factor
  • Typically isn’t in mainline kernel
Any x86 single-board device (e.g., Udoo, UP board, Minnowboard, etc…)
  • Small form-factor
  • SATA3 and USB3.0
  • x86 architecture
  • Small community and limited aftermarket support
  • Expensive for what the devices offer
ODROID C2
  • Cheap
  • Small form-factor
  • Gigabit ethernet
  • Fastest storage connection is USB2.0
ODROID XU4
  • Cheap
  • Small form-factor
  • Gigabit ethernet
  • USB3.0
  • No SATA ports
ESPRESSObin
  • Cheap
  • Small form-factor
  • Gigabit ethernet
  • SATA3 and USB3.0
  • Relatively new
  • Small community and limited aftermarket support
Helios4
  • NAS form-factor
  • Gigabit ethernet
  • SATA3 and USB3.0
  • Relatively new
  • Small community and limited aftermarket support
  • For the same price, I could get a NAS from Synology or QNAP
GnuBee PC2
  • NAS form-factor
  • Gigabit ethernet
  • SATA3 and USB3.0
  • CPU/memory is limited
  • Relatively new
  • Small community and limited aftermarket support
  • For the same price, I could get a NAS from Synology or QNAP
An actual 2-bay NAS (e.g., Synology, QNAP, etc…)
  • NAS form-factor
  • Gigabit ethernet
  • SATA3 and USB3.0
  • Official support and updates
  • CPU/memory is limited
  • Expensive

In the end, I decided on the ODROID-HC2. The HC2 is the larger version of the HC1 (the HC1 holds 2.5" drives, while the HC2 holds 3.5" drives).

single HC2 in case

Both the HC2 and HC1 are based on the XU4, but lack the XU4’s HDMI port, USB 3.0 ports, eMMC slot, and GPIO connectors. The HC2 features:

  • Samsung Exynos 5422 eight-core CPU
  • 2GB DDR3 RAM
  • USB3.0-based SATA3 port
  • Gigabit ethernet (Realtek chipset)
  • USB2.0 port
  • MicroSD slot
  • UART for serial console
  • Stackable aluminum frame that acts as a heatsink and drive cage

multiple HC2s in cases

I purchased the HC2 from ameriDroid, along with a power cable and clear top cover. I also picked up a WD Red 4TB 5400 RPM hard drive. The Red drives are WD’s NAS drives, and the 5400 RPM speed will keep the unit quiet.

I chose the HC2, which is only a single-drive device, because I’m not looking for a 2-bay NAS with RAID support. The primary purpose of this device will be to backup all my other devices, then ship those backups offsite. If the drive dies, I’ll put a new drive in and the only thing I’ll have lost is old backups (locally, anyways).

Software

The HC2 has wide support for Linux, including an Ubuntu image from ODROID, Armbian, Arch Linux Arm, and OpenMediaVault. From these, I decided to go with OpenMediaVault (OMV). OMV is a NAS solution based on (at the time of this writing) Debian 9 (Stretch). It runs services like SSH, FTP, SFTP, SMB/CIFS, NFS, rsync and many more. It also features a nice web interface, S.M.A.R.T. monitoring, and a plugin system to enhance functionality (e.g., LDAP, iSCSI, etc…).

Currently, OMV3 is being retired, to be replaced with OMV4. However, OMV4 is still being tested for ARM boards. Because of this, you can install OMV4 one of three ways:

  • Install Armbian first, then install OMV on top of Armbian
  • Install OMV3, then upgrade to OMV4
  • Install the current OMV4 test image

For this install, I chose to use the OMV4 test image.

Installation

The installation of OMV4 was pretty straight-forward, and there are plenty of guides for the XU4 that apply to the HC1 and HC2 (e.g., here, here, and here). You can also use any guide for OMV (e.g., here, here, and here) and take just the parts you need. I’m going to document what worked for me and what I enabled, but you can enable/disable whatever services you like.

Writing the MicroSD card

First, download the newest OMV4 image for the XU4/HC1/HC2 from here. You’ll need to extract the .img file from the .xz file, then pipe that through dd to write to your MicroSD card (I’m assuming you’re running Linux here).

sudo unxz OMV_4_Odroid_XU4_HC1_HC2.img.xz
sudo dd status=progress bs=4M if=OMV_4_Odroid_XU4_HC1_HC2.img of=/dev/sdX
sudo sync

First boot

When you boot the HC2 for the very first time, expect it to take 30 minutes or so. Since this server is headless (i.e., it has no graphics output), you won’t have the option to watch a screen to verify its state. Wait for the server to ping, then head to the web interface. By default, the web interface username/password is admin/openmediavault.

Date and time settings

Navigate to System, then Date & Time, set your timezone, and enable NTP.

screenshot

Update packages

Navigate to System, then Update Management. Click Check at the top of the page to check for available updates.

Check the box next to each package, then click Upgrade at the top of the page to update the package(s).

screenshot

Once completed, reboot the HC2 from the menu at the top-right of the page.

Enable HTTPS

Navigate to System, then Certificates, then select the SSL tab, click Add, then click Create. Fill in the information as necessary.

screenshot

Navigate to System, then General Settings. In the Secure connection section, enable SSL/TLS and select the certificate you created. You will need to open a new tab using HTTPS instead of HTTP.

screenshot

Change default web admin password

Navigate to System, then General Settings, then select the Web Administrator Password tab to update the password.

screenshot

Enable SSH

I do most of my updating and management through Ansible, which requires SSH access.

Navigate to Services, then SSH, and enable the SSH service. Also, enable Permit root login.

screenshot

Change default root password

By default, the root username/password is root/openmediavault.

Now that SSH is enabled, you can SSH to the server as root. Upon login, you should be prompted to change root’s password

ssh root@IP_goes_here

Once you’ve changed the password, logout.

exit

Disable SSH for root

Navigate to Services, then SSH, and disable Permit root login.

screenshot

Add a secondary user

Navigate to Access Rights Management, then User. Click Add, name your user, and set a password.

screenshot

Enable S.M.A.R.T

Navigate to Storage, then S.M.A.R.T., then select the Settings tab, and enable S.M.A.R.T. monitoring.

screenshot

Navigate to the Devices tab, select your device and click Edit at the top of the page, then enable S.M.A.R.T. monitoring.

screenshot

Drive setup

Physical setup

Navigate to Storage, then Disks. Select your hard drive (e.g., /dev/sda) and click Wipe at the top of the page.

screenshot

Setup LVM - SKIP THIS STEP

I tried to setup LVM so that I could have one encrypted partition, and one unencrypted partition. I was able to get it working, however, every reboot would break the setup. Specifically, under Filesystems, the filesystem would be marked Missing after a reboot. Checking dmesg, I would see the drive was offline with the error Medium access timeout failure. Offlining disk!.

root@backup01:~# dmesg | grep sda
[ 14.588107] sd 0:0:0:0: [sda] 7814037168 512-byte logical blocks: (4.00 TB/3.64 TiB)
[ 14.588119] sd 0:0:0:0: [sda] 4096-byte physical blocks
[ 14.589011] sd 0:0:0:0: [sda] Write Protect is off
[ 14.589025] sd 0:0:0:0: [sda] Mode Sense: 53 00 00 08
[ 14.589455] sd 0:0:0:0: [sda] Disabling FUA
[ 14.589466] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 14.603406] sd 0:0:0:0: [sda] Attached SCSI disk
[ 14.868930] device-mapper: table: 254:0: adding target device sda caused an alignment inconsistency: physical_block_size=4096, logical_block_size=512, alignment_offset=0, start=33553920
[ 14.868939] device-mapper: table: 254:0: adding target device sda caused an alignment inconsistency: physical_block_size=4096, logical_block_size=512, alignment_offset=0, start=33553920
[ 14.870900] device-mapper: table: 254:1: adding target device sda caused an alignment inconsistency: physical_block_size=4096, logical_block_size=512, alignment_offset=0, start=2194476629504
[ 14.870908] device-mapper: table: 254:1: adding target device sda caused an alignment inconsistency: physical_block_size=4096, logical_block_size=512, alignment_offset=0, start=2194476629504
[ 30.572176] sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN 
[ 30.572231] sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x85 85 08 0e 00 00 00 01 00 00 00 00 00 00 40 ec 00
[ 45.597301] sd 0:0:0:0: [sda] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD IN 
[ 45.597352] sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x88 88 00 00 00 00 00 ff 78 7f 7f 00 00 00 08 00 00
[ 61.612177] sd 0:0:0:0: [sda] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD IN 
[ 61.612228] sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x85 85 08 0e 00 00 00 01 00 00 00 00 00 00 40 a1 00
[ 76.637308] sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN 
[ 76.637361] sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 ff 78 7f 7f 00 00 00 08 00 00
[ 76.786240] sd 0:0:0:0: [sda] tag#0 Medium access timeout failure. Offlining disk!
[ 76.786522] sd 0:0:0:0: [sda] killing request
[ 76.786690] sd 0:0:0:0: [sda] UNKNOWN(0x2003) Result: hostbyte=0x01 driverbyte=0x00
[ 76.786787] sd 0:0:0:0: [sda] CDB: opcode=0x88 88 00 00 00 00 00 ff 78 7f 7f 00 00 00 08 00 00
[ 76.786834] blk_update_request: I/O error, dev sda, sector 4286087039
[ 76.859645] blk_update_request: I/O error, dev sda, sector 0
[ 1462.638016] blk_update_request: I/O error, dev sda, sector 0
[ 1462.640632] blk_update_request: I/O error, dev sda, sector 0
[ 1462.655726] blk_update_request: I/O error, dev sda, sector 0
[ 1462.674249] blk_update_request: I/O error, dev sda, sector 0
[ 1462.678512] blk_update_request: I/O error, dev sda, sector 0
[ 1462.697718] blk_update_request: I/O error, dev sda, sector 0
[ 2675.814067] blk_update_request: I/O error, dev sda, sector 0
[ 2698.300818] blk_update_request: I/O error, dev sda, sector 0

I could verify this by checking the file below.

cat /sys/block/sda/device/state

The only way to bring the device back online was to echo running to that file. In fact, this is the official “fix” according to RedHat.

echo running > /sys/block/sda/device/state

Upon reboot, the device would go offline again. Because of this, I recommend skipping the LVM setup and moving straight onto creating one large filesystem.

Instead of having shares that are at a fixed size, I’m going to use Logical Volume Manager (LVM) to create smaller volumes that I can extend/shrink as needed. If you want to skip this setup, you can move on to Format and mount volumes.

Navigate to System, then Plugins. From the search box, search for lvm. Select openmediavault-lvm2 and click Install at the top of the page.

For LVM to work, we need to create three types of groups

  • a physical volume
  • a volume group
  • one or more logical volumes

Navigate to Storage, then Logical Volume Management.

Navigate to the Physical volumes tab, click Add, and select your primary drive.

Navigate to the Volume groups tab, click Add, name your volume group, and select the physical volume you just created.

Navigate to the Logical volumes tab, click Add, name your logical volume, select the volume group you just created, and choose a partition size.

Format and mount volumes

Navigate to Storage, then File System, then click Create. Select the logical volume you just created, then choose a name and filesystem type (I’m using XFS).

screenshot

Once the filesystem is created, select your new filesystem and click Mount at the top of the page.

screenshot

Create shared folder

Navigate to Access Rights Management, then Shared Folders. Click Add, name your folder, and select the filesystem we created earlier.

screenshot

Enable SMB/CIFS

I have Windows, Linux, and Mac clients on my network. Because of this, I’m choosing to use Server Message Block (SMB) to share files.

Navigate to Services, then SMB/CIFS. On the Settings tab, enable the SMB service.

screenshot

Navigate to the Shares tab, click Add, then enable the share and select the shared folder we created earlier.

screenshot

Grant privileges

Navigate to Access Rights Management, then Shared Folders, select your shared folder, and click Privileges. Here, grant Read/Write privileges to the user you created earlier.

screenshot

Client setup

I won’t cover how to setup every single type of client. In this case, Google “how to connect to smb from YOUR DEVICE HERE”.

Performance

As you can see below, the performance is close to 1Gbps, which means the HC2 is able to almost saturate the network connection. Because of this, I don’t believe any of the tweaks suggested here or here apply. Specifically, the CPU governor is set to ondemand by default (shown in the command below) and since we’re using XFS, the NTFS tweaks don’t apply.

sudo grep GOVERNOR /etc/default/openmediavault

Write performance

Here, I’m copying a 1GB file from a Windows 10 laptop to OMV4 over wired ethernet.

screenshot

Here is the same test, but with a 10GB file.

screenshot

Read performance

Here, I’m copying a 1GB file from a OMV4 to aWindows 10 laptop over wired ethernet.

screenshot

Here is the same test, but with a 10GB file.

screenshot

-Logan

Comments

Old comments from WordPress