Forem: James Newman

# My Homelab: 2022 A year in review

James Newman — Thu, 29 Dec 2022 11:18:54 +0000

This article is also available on my blog.

2022 has truly been my year of homelabbing, with learning, iteration and change being a reoccurring theme. I began the year with a somewhat humble homelab, and while I’m not ending it with anything outrageous, it has certainly come a long way. Looking into the next year, I plan on less hardware changes, but there’s certainly a lot more to learn.

How it started

I began 2022 with a Synology DS720+ as my file and Plex server, a Dell Optiplex small form factor PC as an application server and the Unifi Dream Machine (UDM) for my router.

Yes that’s a Makita belt sander and a car cleaning kit behind, storage is tight in this apartment 😅

The Synology is a great little machine, and while pricier than their competitors, the hardware and software quality makes up for the price difference.

The Dell Optiplex is a low power, small form factor PC with a 6 core i5-9500T, 16GB RAM, 256GB NVME SSD and a 1TB SATA SSD. Bought fairly cheap on eBay, these are a common choice for low power and low budget homelabs. For me it was essentially a docker box. I started on Ubuntu server, but quickly swapped to Proxmox and then an Ubuntu VM. While it was a good little machine, the single NIC was becoming a bottle neck along with the lack of cores / threads. And so I sold this in the summer, but in hindsight wish I’d kept it.

Finally, the Unifi Dream Machine, UDM from here on, is a good little router and certainly a nice upgrade from your ISP or run of the mill (insert brand name) router. However, I have certainly run into some issues with it and has me considering a pfSense box.

The big upgrade

I mentioned above I sold my Optiplex in the summer, replacing this was my new custom built “server”. With more RAM, cores, storage and expandability. This new server also managed to change two more times before arriving at the current configuration.

Initial Specs:

CPU: Intel i7-10700
RAM: 32GB Corsair Vengeance LPX 3200
Motherboard: MSI MAG B560M Mortar
SSD: 2TB Samsung 980 Pro
PSU: BeQuiet Pure Power 11 400w
CPU Cooler: BeQuiet Shadow Rock LP
NIC: Intel I350 T4
Case: Fractal Node 804

The CPU has 8 cores, 16 threads, it’s 65W TDP (not “K” series), and importantly for me, supports Intel QuickSync - Intel's video encoder, supported by applications like Plex and Handbrake.

I opted for a consumer motherboard and RAM, being much cheaper than server hardware and sufficient for my use cases. This freed up some budget for the Samsung 980 Pro, a solid NVME SSD for fast and reliable storage.

The case was purchased very cheap second hand from a friend and did the job nicely, for now…

My idea for this new server was to replace my Synology and the Optiplex. It would be my file server, Plex server, a small Minecraft server for friends and a Docker host.

I had played with Unraid and TrueNAS but encountered a number of issues with both, so eventually ended up going back to my Synology as my primary file server. The Minecraft server also didn’t go to plan, but not from a lack performance. I didn’t want to port forward and so used FRPC to proxy though a Linode VPS, however the latency was too much for my friend on the other side of the planet.

So, the new server became a somewhat overpowered Docker host, running Proxmox with a few VMs, using Ubuntu as the Docker host.

Iteration 2

The case was a little large and bulky for the space it was in. I also didn’t need the 3.5” bays now I had switched back to my Synology. Additionally I was after a 5.25” bay for a blu-ray reader. While the fractal supports slim slot readers, I wasn’t able to find one that would support the custom firmware to rip (backup) my UHD blu-ray collection. So I switched to the Silverstone GD09, a compact case with support for a 5.25” and fitted in my shelving unit. It’s worth noting the sides of this unit is mesh, so the server had plenty of airflow.

I also needed to add a PCI SATA card in order to pass through the SATA controller to the Windows VM in order to use the blu-ray drive with it. Paired with MakeMKV, I was ripping and backing up my blu-ray collection. On occasion MakeMKV would no longer find the drive, but shutting down the sever and power cycling resolved this.

Iteration 3 (The current configuration)

After much consideration and a little convincing from a friend, I decided to give Unraid another chance. With only two bays on my Synology, I needed more storage capacity and with Synology moving to AMD CPUs across their new line up, a new NAS was out of the question.

I decided to rebuild my server in yet another case, this time with the Fractal Refine R5. The plan was to rehouse the server in the cupboard that was its original home. However, this time I purchased a shelving rack that would better fit the awkward space.

Not in situ

This time the Unraid setup and configuration went a lot more smoothly, although I did have some issues (from user error) setting up a bond with the 4 NICs on the Intel network card. Finally I setup the 4 NICs in a round robin balancer bond (balance-rr) configuration.

I was also able to take out the PCI SATA card and use the MakeMKV docker image in Unraid, passing the device path instead of a SATA controller.

Final Thoughts

Over the last few months of running Unraid I have really dialled in the configuration to work for me. All of my app data is stored on the SSD, making docker images and VMs blazing fast, while allowing the HDDs to spin down. The rest of my files are spread across these hard drives and for now a single parity drive.

For the most part I am using the Unraid docker interface, although this can be a little lacking for more complicated configurations where the use of Docker Compose would simplify things. Portainer may fill in the blanks here for some people, however I just span up a minimal Ubuntu server in a VM to run docker for these specific use cases.

I have set up a local Nextcloud to replace Synology drive for syncing smaller files like documents across my computers and devices. I do need to better utilise its other features like the Calendar and Contacts if I hope to further replace cloud services like iCloud. I still use SMB shares for larger files like backups and media.

Other changes and improvements

Networking

This year I also began to step further into networking! With help from a friend I setup VLANs on my network, purchased some managed Netgear switches and segmented my network appropriately. This coincides with some firewall configuration and also helped me configure a VPN server for remote connections to my home network when I am out and about.

While I’ve done all this on the UDM, I can’t help but feel that Unifi makes some of this configuration long winded and trickier than it should be. In addition, there are known problems like not being able to create a firewall rule preventing VPN clients from accessing the controller. I am hopeful that the new controller OS update will resolve some of my bugbears. Failing that, a switch to pfSense is almost inevitable.

Raspberry Pis

I played a little with the Raspberry Pi this year, creating a small cluster to learn Kubernetes. This was a super cool little setup with a fancy cluster case and all, but I just wasn’t using it. Like the Optiplex, I sold them on at a fair price to a friend. However, I intend to revisit the Pi moving PiHole from the sever to a Pi, along with my reverse proxy, NUT (a network UPS server) and potentially a second Pi for redundancy.

Currently, if the Unraid server is offline for any reason, devices are unable to resolve the DNS as it’s hosted on the server. “Well setup a second DNS entry in your router!” I hear you cry. It would seem doing so results in my devices using this second DNS entry to resolve domains in my PiHole blacklist, rendering the PiHole useless…

I also tried the Pi TV hat, allowing me to receive digital DVB-T2 signals, a bit like the HD Homerun, but seemingly worse… I had a great deal of issues configuring, receiving and streaming live TV with the Pi TV hat. The tuner seemed hit and mis on whether it’d find channels that my TV would otherwise pick up when plugged-in directly to the buildings antenna. I tried a signal booster which did improve my results, but still found lacking.

UPS

After putting it off for a long time, I finally have a UPS. I originally ordered an Eaton Eclipse 500, which the product photos on Amazon showed a USB for management, however this model didn’t include a USB management port. I ended up keeping this UPS and eventually ordering a second, the 650 with USB. Now if the power goes out, my server and NAS can gracefully shutdown.

Synology

The Synology still has its uses in my homelab, primarily acting as a backup server using some of their great apps. Synology Photos has become my tool of choice for backing up and viewing my photo library from my phone across devices. Active Backup allows me to backup appdata, files and media from my Unraid server via SMB. And finally I use Hyper backup to then backup the most important files to Synology C2 for offsite backups.

All together

Capping off the year

So, as you can see, there have been plenty of changes this year and I have certainly learned a lot. But going forward there’s a number of things I plan to change in 2023:

Add a Unifi Wifi AP, the UDM Wifi is patchy at best.
Implement monitoring for my servers and containers.
Replace NGINX Proxy Manager with Traefik.
Expose limited access to services like Plex via Tailscale funnel.
Potentially replacing my UDM with a pfSense box.
Potentially replace my Ikea smart lights and hub with Phillips Hue - I have to restart this thing multiple times a week for it work in HomeKit.

I’m also really interested in finding ways to reduce my homelab's power usage. I am by no means an eco warrior, but certainly notice the increase on my power bill here in the UK this year. Combined with smaller form factor systems, I really feel the lower power / small form factor homelab community is pushed side in favour of big, power hungry server racks.

And finally, more guides and blog posts! Stay tuned.

This article is also available on my blog.

Accessing a Unifi network over VPN

James Newman — Tue, 15 Nov 2022 18:43:05 +0000

This article is also available on my blog.

In this post I'm going to walk you through how I setup my VPN server on the Unifi Dream Machine, and it'll likely be the same for the Dream Machine Pro or Security Gateway.

I have a few services running in my homelab that I'd like to access when I'm not home, such as Plex and Nextcloud. But exposing these to the open web can be risky and a bit scary. Even though I have my network nicely segregated into VLANs, (big thanks to my friend Blag for assisting), and could use a proxy tool like FRP or Cloudflare tunnels to avoid opening ports, it's still risky. Of course setting up a VPN server isn't bullet proof, but it is less risky. In fact, if you're working remotely there's a good chance you're using a corporate VPN on your work computer to connect to their systems.

Backup

Before we change any settings on the router I'd suggest creating a backup of your current configuration. Nothing should go wrong, but I've learnt the hard way from accidentally locking myself out of my managed switch that it's good to have a rollback point. You can do this from the Unifi OS System settings right after you log in, before selecting the network - Settings > System > Cloud Config Backup https://[your_gateway_ip]/settings/system.

Create a new backup and then download it. You should have a download similar to unifi_core_backup_[some_string].unifi.

VPN Server Configuration

Go ahead and click into your Unifi network > Settings > Teleport & VPN https://[your_gateway_ip]/network/default/settings/vpn.

Now Unifi does have a zero configuration VPN "Teleport", however access with this is limited. You'll need to use the WiFiman app on mobile, and desktop... well as of writing this the macOS app is "coming soon" and there's no support for Windows.

We're going to opt for the traditional VPN server further down on the page. So check the enable VPN Server box and we can start entering our settings.

Fields of note:

Pre-shared Key - You'll need this later when connecting, I added it to my password manager with the username and password further down.
User Authentication.
Network Name - This will show in the network column of the "Clients" page in the Unifi Network along with the username connected.
Gateway/Subnet - This is the Subnet / IP range you want your VPN clients to be on. I'd recommend setting the network size as small as you require, such as 29 (5 IPs).
Require Strong Authentication - It may cause issues for some devices, but I've not encountered this so far.

It's worth noting the VPN cannot be on a VLAN, nor can it be in a VLANs IP range. However we can still configure firewall rules with groups, which we'll cover further down.

With those configured, your VPN server should be ready. You'll connect shortly via your public IP, but I would recommend using a DDNS (dynamic domain name system) if you don't have a static IP like me. My Synology NAS has a service built-in for this, but there's many others out there including DuckDNS and Cloudflare.

Connecting

Now we have our VPN server running, lets try connecting! I'll be connecting from an iPhone and mac. But here's steps for Android and Windows, the steps below should give you an idea.

iPhone

From settings, either search for VPN, or go to General > VPN & Device Management > VPN > Add VPN Configuration.

Mac

From settings, either search for VPN, or go to Network > Three dot button in the bottom right > Add VPN Configuration > L2TP over IPSec.

Creating Groups

Once connected, a VPN user will be on your network almost as if they're connected locally. Depending on your preference or threat tolerance you might want to lock this down. I've configured my firewall to block all LAN activity except for specific rules I've created, such as connecting to PiHole for DNS based ad blocking, or Plex as mentioned at the beginning.

First, we want to setup a group for the VPN. We can then use this group when creating firewall rules rather than adding each possible IP. To do this, go to Profiles in the Settings sidebar and scroll down to "Port and IP Groups" - https://[your_gateway_ip]/network/default/settings/profiles. Click "Create New Group".

We'll create a group called "VPN Users", with the "IPv4 Address/Subnet" type. For the address we want to enter the subnet used by the VPN server in the last section. You might notice I've used 24 instead of the 29 that I configured for the VPN Server. This is incase I resize my VPN network later, I still want this group to apply to all IPs on the VPN subnet.

I also have another group named "RFC1918". This is a standard which defines the subnets used in private networks, you can read more about it here. I use this group to make blanket rules, like blocking all access to something. I then followup with another rule higher in the hierarchy that allows access for specific networks, IPs or groups. I'd recommend making this group too.

You should now have 2 groups ready to use in the firewall.

Firewall Rules

Navigate to the Firewall & Security page from the Settings sidebar and scroll down to Firewall Rules - https://[your_gateway_ip]/network/default/settings/security. Click "Create New Rule".

First we're going to create a rule to block the VPN group from connecting to any IP on the network. We'll then follow up with additional rules allowing access to specific IPs, groups or networks (VLANs).

Fields of note:

Type - Set to LAN out - I'll be honest, I still don't fully understand this, but it's to do with the flow of traffic.
Action - We want to drop all connections.
Source Type - Where the traffic's coming from, our VPN Users group.
Destination Type - Where the traffic's going, anywhere on the LAN.

Note this doesn't block connections from the VPN user to the gateway, even if you use port groups. At the time of writing this it's a well documented issue with Unifi which is yet to be resolved.

Now we've blocked all traffic from the VPN subnet to our LAN, we want to create some rules allowing specific connections. I'm going to create one allowing access to PiHole, but the steps are similar for allowing access to another group or network.

This time the Action is Accept and the Destination Type is IP Address, but the source is the same - the VPN Users group.

Now that's added, we need to change the order of the rules as by default new rules are added to the bottom of each type, meaning anything above them such as the Block VPN from LAN rule will take precedence. On the left of each row you can click and drag, you want Allows above the Blocks.

Go ahead and add any other rules you want to allow and that's it. Connect your phone or computer, test you can connect to the allowed IPs, and also test that you can't connect to the IPs you shouldn't be able to.

In Closing

So that's now how I access my home network while remote. I plan on testing it properly on a small trip in the coming weeks, but so far it seems to work nicely.

I also planned on making a shortcut on my iPhone to automatically connect and disconnect from the VPN whenever I'm not on my WiFi, however iOS 16 doesn't have any support for VPN settings. Additionally, automations only support connection to a specific network, not disconnecting. Hopefully this will be expanded in future iterations.

This article is also available on my blog.

I switched from Vercel to Cloudflare Pages

James Newman — Thu, 03 Nov 2022 19:04:35 +0000

This article is also available on my blog.

Cloudflare has launched its Pages product, Cloudflare Pages. Aimed at JAMstack websites and applications, it joins the many hosting services that offer easy deployment for simple sites like this one!

I've been using Vercel for a while, it was known as now.sh, then Zeit when I first started using it. Using the hobby tier, their free tier, it's served its purpose just fine and in all honesty I didn't have much of a reason to leave them. However, I already manage my domains through Cloudflare and use their basic but free, privacy respecting analytics, so I figured I'd give Pages ago.

Set up was a breeze. If you've setup a site on a similar service, the workflow is familiar. You link the repository, point it at the branch to monitor, add build configuration and deploy. They have presets for a number of JAMstack frameworks, including the one I use for this site, 11ty (Eleventy). However I did need to tweak the build steps slightly to my configuration. This included the build command and output directory as shown below.

I use sass, or scss, to style the site, so those stylesheets should be compiled into a css file during deployment. Doing this is simply part of my npm run build command:

{
  ...
  "scripts": {
    ...
    "styles": "sass src/styles/main.scss src/assets/styles.css --style=compressed --no-source-map",
    "build": "npm run styles && eleventy"
  },
  ...
}

I also needed to adjust my rewrite rule for 404s so any urls not found are redirected to the 404 page. In Vercel, you can achieve this like so:

{
    ...
    "rewrites": [{ "source": "/(.*)", "destination": "/404/index.html" }]
}

I struggled to find a straight forward way to do this with Cloudflare Pages, but after digging around a little, I found Cloudflare can resolve 404s automatically to a 404.html file in the top level of your site, meaning I didn't need a _redirects file to handle this. So in my 404.ndk file I simply updated the front matter to include the permlink. This placed it in the top level, instead of 404/index.html:

---
title: "404"
layout: base
templateId: not-found
permalink: 404.html
---
<section>
    <h2>404</h2>
    <hr />

    <p>Hmm, we couldn't find what you were looking for...</p>

    <a class="button" href="/" title="Home">Back to Home</a>
</section>

And that was it! My nameservers for jamesnewman.dev were already pointing at Cloudflare, so the DNS just needed updating which Cloudflare can also do at the click of a button. It really is straight forward.

My thoughts so far

As of writing this, the sites only been on Pages for a few days. However Cloudflare analytics seem to be working better and more importantly, the site seems to load faster! Better yet, it's backed by the power, scale and security of Cloudflare.

So should you switch? Maybe. Like I said at the beginning, I didn't really need to switch and you probably don't need to either. But with that in mind, take a look at the feature list on their site, the pricing tiers and evaluate if it's worth it for you.

This article is also available on my blog.

Routing to multiple Minecraft servers on a host with Docker

James Newman — Tue, 25 Oct 2022 07:58:57 +0000

This article is also available on my blog.

In my previous post I walked through the setup of a Minecraft server with docker. But now you might want to setup a few servers on the same host and route domains to them without different ports, a bit like reverse proxying.

By default Minecraft runs on port 25565, but when we have multiple servers running on the same host, they can't all expose port 25565 and so you end up with something like 25565, 25566, 25567 etc. With mc-router, we can route all those domains to 25565 and resolve them to the configured server!

The Environment

At a minimum you'll need Docker installed on the host, but we'll also be using Docker Compose. If you followed the previous post you're all set. If you need to install Docker Compose, you can follow their guide here.

This solution also requires at least one domain to route to our server. Point your desired domains or subdomains at the server ip hosting your servers. If you're experimenting with this locally, you can add an entry to your hosts file to point to localhost. In this post I'll be using the following example domains set in my hosts file:

nano /etc/hosts

mc.example.com
modded.example.com
events.example.com

Setting up the router

We'll be using another docker image from itzg, this time itzg/mc-router. So go a head an open you docker compose file. If you followed the previous post, we'll continue using the same docker-compose.yaml file.

First we're going to need to remove our ports definition for each server, we don't need to expose these any more as we'll make use of dockers internal networking. Then we can go ahead and add our mc-router service.

version: "3.8"
services:
    vanilla:
        container_name: vanilla
        image: itzg/minecraft-server
        environment:
            EULA: "TRUE"
            MEMORY: 2G
        volumes:
            - ./vanilla:/data
        restart: unless-stopped
        tty: true
        stdin_open: true

    mc-router:
        image: itzg/mc-router
        environment:
            API_BINDING: ":25564"
        ports:
            - 25565:25565
            # bind the API port to only loopback, this avoids external exposure
            - 127.0.0.1:25564:25564
        command: --mapping=mc.example.com=vanilla:25565,modded.example.com=modded:25565,events.example.com=events:25565

You can see in the command line, we've mapped our domains to our containers:

mc.example.com      -> vanilla
modded.example.com  -> modded
events.example.com  -> events

Now go ahead and startup the stack with docker compose up and test them out in your Minecraft client!

If you'd like to know a little more about how it works, I'd recommend taking a look at the itzg/mc-router doc where they include some diagrams and alternative configurations.

This article is also available on my blog.

Running Minecraft servers from Docker

James Newman — Mon, 17 Oct 2022 07:50:33 +0000

This article is also available on my blog.

Since picking up docker last year it's completely changed how I set up projects, home network applications, and now how I deploy Minecraft servers for friends. Gone are the days of figuring out which version of JDK I need on an ubuntu box and remembering the commands to reconnect to a screen session.

In this post I'll walk you through setting up a few different Minecraft servers, running vanilla, Paper and Forge, all on the same host. In a follow up post I go through routing to these server on the same port.

Before we get started, I'd recommend having some basic command line and linux knowledge. In addition, pre-existing docker and docker-compose experience is a bonus but not required.

The Environment

Docker being docker, you can run this on Windows, Linux, macOS - A spare computer, the cheapest DigitalOcean droplet, heck even a Raspberry Pi! Just make sure your systems up to date, has docker and docker-compose installed. Here's what we'll be using:

A docker host, the steps below are for Ubuntu but you can use any.
itzg/minecraft-server docker image to run the server.

Install Docker & Docker Compose

Go ahead and follow the Docker install steps for the platform you wish to run the server on. Here's the Ubuntu guide.

Handy tip: if you're on linux, make sure you add your user to the docker group:

sudo usermod -aG docker $USER

You'll need to log out and back in again after.

Now you should be all set! To double check, run the following:

# Check the Docker version
docker -v

# Check Docker Compose version
docker compose -v

Creating our first server with Docker Compose

It's vary rarely I'll run a docker image with the docker command line, remembering the command and all the arguments I need is a faff. So lets define our first Minecraft server, a simple vanilla server, in a docker compose YAML file.

First up, we'll create a directory to contain all our files including the Minecraft servers:

# Create directory
mkdir minecraft-servers

# cd into the directory
cd minecraft-servers

# Create a docker-compose.yaml file
touch docker-compose.yaml

# Create a directory to store our vanilla server files in
mkdir -p vanilla/server

Go ahead and open the docker-compose.yaml. You can use nano, vim or even connect VS Code with SSH.
In here we want to define our vanilla service:

version: "3.8"
services:
    vanilla:
        container_name: vanilla
        image: itzg/minecraft-server
        ports:
            - 25565:25565
        environment:
            EULA: "TRUE"
            MEMORY: 2G
        volumes:
            - ./vanilla/server:/data
        restart: unless-stopped
        tty: true
        stdin_open: true

This is the bare minimum configuration, but there's a whole bunch of options you can define such as ops, whitelist, difficulty etc, all documented here. Now we're ready to test our first server works, so lets spin it up!

# Start the service
docker compose up

You'll should now see the server begin to start up. It'll take a little while on the first launch as it downloads the image and generates server files. You'll know it's ready when you see:

vanilla | [17:22:18] [Server thread/INFO]: Done (13.045s)! For help, type "help"

Congratulations, you can now connect to your new server using the server IP or localhost if you're running it on the same machine! To exit, you can use ctrl + c in the terminal.

If you take a look in the vanilla/server directory we created earlier, you'll see it's generated all the server files needed. You can edit the server.properties to make changes to the servers configuration, however I'd recommend adding these to the environment block in our docker-compose.yaml and simply deleting the server.properties file. This file will be recreated when you next start the server with your newly added environment options.

Adding different server types

So we have our vanilla server set up, but if we want to use some plugins to manage permissions or add prefixes to usernames in chat, we'll need to use a different server type. For a vanilla server that you can connect an unmodded client to, you have a few options: Bukkit, Spigot and Paper. We'll use Paper for the sake of this post.

First, lets create a directory to keep our plugins in:

mkdir vanilla/plugins

You can go ahead and add any plugins to this directory and we'll configure the container to mount these. You can also leave it blank for now and once we start the server you'll see bStats automatically added by Paper.

Now in our docker-compose.yaml file, we'll add a new entry to the environment block of our vanilla service, defining the server type. We'll also mount the plugins directory we just created.

version: "3.8"
services:
    vanilla:
        container_name: vanilla
        image: itzg/minecraft-server
        ports:
            - 25565:25565
        environment:
            # Add our server type
            TYPE: PAPER
            EULA: "TRUE"
            MEMORY: 2G
        volumes:
            - ./vanilla:/data
            # Mount our plugins directory
            - ./vanilla/plugins:/plugins
        restart: unless-stopped
        tty: true
        stdin_open: true

Lets spin up the updated server, you'll see it download the new jar and create some new files.

# Start the service
docker-compose up

Now you should be able to edit the configuration files for the plugins you added, log in and see them working!

Proxying and Routing domains to Minecraft Servers

In a follow up post, I go through adding adding itzg/mc-router to route domains to multiple servers, all using the default 25565 port. No more ugly server addresses with different ports! You can read about that here

Closing thoughts

So in this post we've created a Vanilla server, maintained and managed through docker. I highly recommend reading through the docs for this image as there's tons of configuration available, including support for modded servers. You can add more servers to this config and use something like BungieCord or PaperMC Waterfall switch servers ingame.

This article is also available on my blog.