Forem: Nathan Bland

Setting up Drone CI for CI/CD homelab use - build log

Nathan Bland — Fri, 27 Oct 2023 01:16:41 +0000

Constantly on the search for something to replace jenkins as my be-all, end-all continuous build/deployment tool, I decided to try setting up a local instance of drone, as it looked fairly promising. This is a build log of my results.

Step 1 - setup and configuration

As with my previous build-log setting up goCD, this is going to use docker-compose on portainer. It's where almost all of my home-lab things live at this point, and I haven't really had strong reason to change it. Having said that, I have weird volume configurations, so I'll share simplified versions of those when/if needed.

The docs for drone first have you go through the setup process of creating an application on your desired source control platform. I find this to be a little backwards personally, so I'm going to start with the compose file instead and see if I regret it.

The instructions for getting going with docker are pretty simple at first:

$ docker pull drone/drone:2

Which is absolutely nothing to write home about, but that just gets the image. Next up is the listed configuration options, most of which are required. Below is pulled straight from the setup instructions:

DRONE_GITHUB_CLIENT_ID
Required string value provides your GitHub oauth Client ID generated in the previous step.
DRONE_GITHUB_CLIENT_SECRET
Required string value provides your GitHub oauth Client Secret generated in the previous step.
DRONE_RPC_SECRET
Required string value provides the shared secret generated in the previous step. This is used to authenticate the rpc connection between the server and runners. The server and runner must be provided the same secret value.
DRONE_SERVER_HOST
Required string value provides your external hostname or IP address. If using an IP address you may include the port. For example drone.company.com.
DRONE_SERVER_PROTO
Required string value provides your external protocol scheme. This value should be set to http or https. This field defaults to https if you configure ssl or acme. This value should be set to https if you deploy Drone behind a load balancer or reverse proxy with SSL termination.
DRONE_USER_FILTER
Optional comma-separated list of GitHub users or organizations. Registration is limited to users in this list, or users that are members of organizations in this list. Registration is open to the public if this value is unset.

Given some of these values, it is easy to see why they want you to configure an application first, as you'll need those values. Oh well, off to github I go to get credentials.

Once I had those, I could put together the start of my compose file

version: "3.7"
services:
  drone:
    image: drone/drone:2
    environment:
      - DRONE_GITHUB_CLIENT_ID=super-secret-github-id
      - DRONE_GITHUB_CLIENT_SECRET=super-secret-github
      - DRONE_RPC_SECRET=super-shared-secret
      - DRONE_SERVER_HOST=drone.my-domain.local
      - DRONE_SERVER_PROTO=https
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/lib/drone:/data
    restart: unless-stopped

I've filled in a couple values here with example data for security reasons, but details do need to match what you got from github, except for the DRONE_RPC_SECRET. This is going to be a shared key between our main instance, and our runners, more on that later I think. You can use any random string, or use openssl to generate one:

$ openssl rand -hex 16

A final note about that above compose. The ports will be different, or may need to be. Changing the left side changes what is exposed on your machine, and you can set that to whatever you like, so long as you have a way of directing your domain to it. Nginx-proxy-manager is great for this.

This is, in theory, enough to spin the server up. So let's do that and see what we get.

Step 2 - First user setup

This landed me at the login page, which prompted a quick authorization from github, follwed by, a user login prompt again? Seems strange, but filling in the details and following the prompts worked flawlessly, so I won't question it.

This landed me on a screen with all the repositories I had granted access to this application, which was pretty neat to see. Clicking on any of them displayed another screen mentioning how this repository is not active, which makes sense, and is exciting, but now we are getting ahead of ourselves. We need to setup a runner.

Step 3 - Drone runner node

We can add this to our existing docker compose file, directly below the main instance. This was also pretty short and simple.

drone-runner-1:
    image: drone/drone-runner-docker:1
    container_name: drone-runner-1
    environment:
      - DRONE_RUNNER_CAPACITY=4
      - DRONE_RUNNER_NAME=docker-runner
      - DRONE_RPC_SECRET=super-shared-secret
      - DRONE_RPC_HOST=drone.my-domain.local
      - DRONE_RPC_PROTO=https
    ports:
      - "3331:3000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped

I added this to my existing compose, and the instructions say to check the logs for the following:

$ docker logs runner

INFO[0000] starting the server
INFO[0000] successfully pinged the remote server

I had that, so it seemed to be working.

Step 4 - Test an actual pipeline

I seemed to be done with server setup, which shocked me, as that was really easy compared to some other options. Next up I needed to actually push a build-pipeline to a repo, and see if it would take.

I found the repo I wanted, and clicked on the "activate" button from the web ui in drone.

The example pipeline they gave seemed like I good start, so I went with that:

#.drone.yml
kind: pipeline
type: docker
name: default

steps:
- name: greeting
  image: alpine
  commands:
  - echo hello
  - echo world

While I don't super love yaml for all the configuration, the naming and matter that the steps and commands were setup seemed pretty straight forward to me, which was nice.

I added those changes to .drone.yml in my repo and pushed it up.

This, produced a very nice result, as my push got a little green check-mark in github, and clicking on that "build passing" took me directly to drone where the job had run with a list of the steps, and a console log of the output. It was beautiful, and simple.

Conclusion

So far, I really, really like using drone. I'm still very new to it, and getting the hang of the build steps and configuration will take time, but getting the initial setup took no time at all, and produced amazing results. Stay tuned for more, thanks for reading.

Setting up GoCD in a docker environment - build log

Nathan Bland — Fri, 07 Jul 2023 05:54:54 +0000

Having worked with Jenkins in the past, I've been curious about GoCD for quite a while, and decided I would finally take the plunge and figure out how two set it up.
A deployment of GoCD needs at least two parts, a server, and an agent. This will cover my experience and notes from doing that configuration and setup.

Step 1 - Docker things

Choose the specific version of the container/OS you'd like to use, for my build, I'm going with gocd/gocd-server version 23.1.0 which can be pulled with:

docker pull gocd/gocd-server:v23.1.0

Simple usage from their docker docs would indicate that you can run it with the following:

docker run -d -p8153:8153 gocd/gocd-server:v23.1.0

This would spin up GoCD locally on port 8153, but since I'm going to be deploying this using portainer, I want to build out a docker compose file to do this.

The docker docs do provide more useful information, such as what paths in the image contain useful data that you may want to preserve with volumes:

/godata
/home/go

It includes the note that the go user in the container should have access to these directories, and shares its user id - 1000.

An additional note that seemed useful here was around running custom entrypoint scripts, and loading configuration from an existing git repo, but at this point I wasn't sure I needed any of that, so I left it for now.

Step 2 - Portainer things

I'm going to create a portainer stack to deploy this in, which probably adds unneeded complexity, but it is what I run most of my home lab systems in, so I'm at least fairly familiar with its pitfalls. Because I already have services deployed in this system, I stole one of my own configs and tweaked it to run GoCD. Below is what my stack/compose file looks like to test things out:

version: '3.2'
services:
  gocd-server:
    restart: always
    image: gocd/gocd-server:v23.1.0
    container_name: gocd-server
    ports:
      - "8153:8153"
    volumes:
      - type: volume
        source: gocd-data
        target: /godata
        volume:
          nocopy: true
      - type: volume
        source: gocd-home
        target: /home/go
        volume:
          nocopy: true
    #environment:
    #depends_on:
    networks:
      - gocd-net

networks:
  gocd-net:

volumes:
  gocd-data:
    driver_opts:
      type: "nfs"
      o: "addr=<nas-ip>,nolock,soft,rw"
      device: ":/storage-path/gocd/gocd-data"
  gocd-home:
    driver_opts:
      type: "nfs"
      o: "addr=<nas-ip>,nolock,soft,rw"
      device: ":/storage-path/gocd/gocd-home"

I should note I have a pretty unique storage system in place for my volumes on my portainer system, and I wouldn't recommend following that setup unless you absolutely know what you're doing. I've obscured some values in those volumes, but I wanted to include the setup for the sake of completeness.

After a bit of debugging to get to the compose file you see above, it finally deployed without complaint, so now I could test and make sure the UI actually spun up. A quick visit to the IP and port showed I had moderate success so far, I had the start-up wizard.

Step 3 - Agents

My first visit to the gocd ui dropped me into a Add a new pipeline screen, which present a whole host of things I hadn't used before, but made sense upon review. It wanted a material which is a source of some kind, for most people - myself included - this will probably be a git repo.

This did in turn, provide me with my first hiccup. Attempting a test connection to a public repo of mine of course failed, as I lacked any kind of authentication. I also feared I was getting ahead of myself as I knew that GoCD required an agent - which I hadn't setup yet - but I was already messing with pipelines.

After poking around the GoCD docs a little more, I decided I wasn't quite ready for a pipeline, and went back to the install portion of the documentation to see what I should do next. It turns out, that was setting up an agent.

I browsed the available docker images for agents, and decided on an ubuntu 22 agent. This would also be version 23.1.0

Unlike the server that required absolutely no env configuration unless you were in the advanced options, the agent docker required at least one env value, the server for it to talk to. Makes sense.

Back to the docker compose file, I added another service entry:

gocd-agent-ubuntu:
    restart: always
    image: gocd/gocd-agent-ubuntu-22.04:v23.1.0
    container_name: gocd-agent-ubuntu
    volumes:
      - type: volume
        source: gocd-agent
        target: /godata
        volume:
          nocopy: true
      - type: volume
        source: gocd-home
        target: /home/go
        volume:
          nocopy: true
    environment:
      - GO_SERVER_URL=https://ip.add.re.ss:8153/go
    #depends_on:
    networks:
      - gocd-net

It should be noted that their own docs say an https server url is required. I have my setup behind an nginx proxy manager, so I added an entry for it to get a signed cert, but I believe it can self-generate one for you to use if you toggle off a specific security setting for the agent (see the Configuring SSL section of the docker agent linked above).

The next steps in the GoCd docs told me to head to the agents tab in the UI where I looked for this option in the admin drop-down menu despite it being a top-level menu item for far too long. I did see mine listed, and getting it attached to the server was as simple as selecting the checkbox next to its name and hitting the enable button at the top.

Step 4 - Users

The next thing I needed to do before actually giving any real data to GoCD to deal with, was configure user management, as right now it was wide open. Turns out this was not as simple as I was hoping. GoCD does not do user management of its own. It takes in an authentication service of some type. You can import users from that source, and give them specific permissions, but it won't let you just spin up a user in its own system.

What it does allow out of the box, is either a password file, or LDAP server. At some point, I may spin up a docker container for LDAP, that would be cool for all sorts of things, but for this case I opted for a password file to try and keep things simple.

The docs pointed me to a github repo for the plugin where I learned gocd-passwd can be used to generate the hashed password.

The example in the docs showed the file being in /godata/config/ on the container, so I connected to the container console with portainer, and set about making the file. However this was a misunderstanding on my part, as that command and file actually isn't part of the docker image.

I did however, have an ubuntu agent, and the docs on the github repo showed how to use htpasswd to generate a password file on ubuntu, so I attached to that console instead, and ran the following

# first grab the utils
apt-get update
apt-get install apache2-utils

# genereate a password file called passwd, using bcrypt, for the user admin
htpasswd -c -B passwd admin

Because I had the home directory mapped across both of these containers, I should be able to share this file between them, or simply copy it and delete the original once I had it on the server container.

After reattaching to the server container, I confirmed the file was there, and moved it to where it belonged in the /godata/config folder.

Back in the UI, I went to Admin -> Authorization Configurations and clicked add. Selected password file authentication plugin, and filled in my details. I wasn't sure exactly what to call the id for this auth config, so I went with the value in the example screenshot - file-auth-config, and figured it would yell at me if that wasn't allowed.

I then did the following:

I hit Check connection and it reported that it was ok
I hit Allow only known users to login. do not do this.
I clicked Save.

This was, a massive mistake, as I had not allowed my newly created user to login yet - it just got added - so I essentially locked myself out of the ui.

If you happen to do this, good news, this configuration is written to a file, and you can manually edit that config. The file lives in /godata/db/config.git/cruise-config.xml, and `/godata/config/cruise-config.xml

Edit the line that looks like this
`xml <authConfig allowOnlyKnownUsersToLogin="true" id="file-auth-config" pluginId="cd.go.authentication.passwordfile"> `

To look like this:

`xml <authConfig allowOnlyKnownUsersToLogin="false" id="file-auth-config" pluginId="cd.go.authentication.passwordfile"> `

Save those files, and if needed, restart your container.

Once you have yourself back in the system - or hopefully didn't lock yourself out at first, you can navigate to Admin -> Users management and check the little box that makes you a system admin.

Once this is done, you can navigate back to Admin -> Authorization configuration and edit that password file configuration, and check that allow only known users to login if you want.

Yay, a working user!

Step 5 - Pipelines!

Having gotten users and agents setup, I was now back at my Add a new pipeline screen from earlier. I realized I was being silly, as the volume I created for the home directory was for exactly this, to allow adding ssh keys. If you haven't done this, github has good docs on this subject.

I created a new key from the docker container, and saved it to the volume so it wouldn't be lost on container restart.

Next I added it to my github account - docs are here - and now it was time to see if it actually worked.

It did not.

This made me sad, but this felt like such a common thing that I had to be missing something, so back to the docs I went.

Under faq/troubleshooting I found a very useful page on ssh keys and docker. The step I had been missing was running a clone in the console to accept the github server signature. This could have also been done with ssh-keyscan. I opted to try and clone a repo, and it did prompt me. I accepted, and then tried to test my connection again in the UI.

This time, it worked!

I set my repo URL in the materials section, gave my pipeline a name of build-and-deploy - we'll see if it actually ends up doing that - added a build stage, and a task. This got me to the point where I could click save + run which finally landed me on the dashboard page.

🎥 How to Build Your Own Continuous Automation & Build Server - Using Jenkins & Docker - #1

Nathan Bland — Fri, 11 Oct 2019 01:48:46 +0000

Cloud providers everywhere are trying to get you to use their automation pipelines and services. Let's build our own instead.

Video

Cover Photo by Christopher Burns on Unsplash

🎥 Setting up a Kubernetes Cluster - Live Stream Replay (with notes)

Nathan Bland — Wed, 02 Oct 2019 21:42:09 +0000

Ever wanted to setup your own Kubernetes(k8s) cluster for development? Wanted to go a bit further than using minikube? We'll walk through setting up 3 vms to create our very own k8 cluster.

Video - notes below

Notes and Summary

notes

The first ~30 minutes of this are VM install and setup, feel free to skip that.
The GUI I'm using for KVMs is virt-manager.
I did not have virtualization turned on in my system bios (new system didn't have the switch flipped yet) so this severely impacted performance.
virt-clone should have provisioned a different mac address for each VM clone, still not sure what happened there.

Summary

Most of the instructions I ran through aside from networking woes can be boiled down to this:

1. Setup support for installing packages over https

sudo apt-get update && sudo apt-get install -y apt-transport-https

2. Add trusted key for packages we want to install

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add

3. Add repository for Kubernetes(k8s)

sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"

4. Update package listings

sudo apt-get update

5. Install kubernetes, its tools, and docker.io

sudo apt install kubectl kubeadm docker.io

6. Disable swap memory. (it isn't supported)

sudo swapoff -a

You'll also want to remove the swap entry from fstab so you don't have to do this every time your system comes up (vm or bare metal).

sudo nano /etc/fstab

Remove the entire line with swap, but nothing else, be careful here.

7. Enable and start the docker service.

sudo systemctl enable docker.service

8. Setup the Docker Daemon

sudo nano /etc/docker/daemon.json

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}

sudo mkdir -p /etc/systemd/system/docker.service.d

9. Reload the docker daemon and system process

sudo systemctl daemon-reload && sudo systemctl restart docker

10. Clone our VM.

At this point you can clone your VM, you should be able to do this and get the same installs, but with different network MAC addresses. Some issues with my own setup prevented this from going smoothly, but this is the time to clone the VM if you want to go that route.

Spin up two clones of our first VM. You'll need to change the hostname at a minimum, and reboot (or restart the networking service).

sudo nano /etc/hostname

I used kube-worker-1 and kube-worker-2, but use whatever you like. So long as it doesn't have _ or CAPS characters. I learned that the hard way.

sudo reboot

Steps on our Manager node

1. Initialize our k8 cluster

sudo kubeadm config images pull - (optional) The next command will do this by default, but it can take a while, my sanity prefers pulling the images first.

sudo kubeadm init

After running this command, you'll get output similar to this:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Along with a join command for the cluster. Copy this, save it to a safe place. You'll need it later.

If you lose this token, you can make a new one with:

kubeadm token create --print-join-command

2. Establish Cluster Network - Using Weave

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

3. View the networking services we just spun up

kubectl get pods --all-namespaces

Steps on our Worker nodes

1. Join our cluster

sudo kubeadm join .... - This should be the command you copied/saved form the init command above.

Back to the Manager Node

We can now check the status of our cluster:

kubectl get nodes

Profit

Let's Build a Home Lab Server - A Build Log

Nathan Bland — Tue, 28 May 2019 19:47:41 +0000

A home lab is something I've been wanting for a very long time. In this post, I'll drag you along for the adventure that has been my build out, some troubles I faced, and how I solved them.

A lot of people can get by starting out with a raspberry pi as a home lab. I also started with one of these, but my needs have grown beyond what it can do. The pi (especially the pi 3 model b+) is still a great little tool, and if you are just getting started I'd recommend one. For the price they can't be beat. This is a bit of a mind dump, so if you want to jump around, here are the sections:

Intro (you are here)
Research
The Build
The Storage
The Network
The Software
Extras
Install and Troubleshooting

Research

I spent a lot of time trying to figure out what parts, or system I should get for this. I wanted solid performance, but like everyone I didn't want to pay a truck load of money to get it. I fell into the rabbit hole of /r/homelab and /r/DataHoarder/ looking for suggestions.

I found myself over at serverbuilds.net looking through their build lists. This led me to looking through ebay, which is how I came across MET Servers who specialize in used enterprise hardware. I had found my sweet spot.

The Build

The prices on MET simply blew me away. I chose to start my configuration with a Dell R710 for a couple of reasons. One, the backplate it uses (PERC H700) can handle large drives and can do SATA 6Gb/s. Here is the full configuration I ended up going with:

The Brain

• Dell PowerEdge R710 6-Bay LFF 2U Rackmount Server
--------------------------------------------------------
• Processors: 2x Intel Xeon X5660 2.8GHz 6 Core 12MB Cache Processors.
• Memory: 24GB DDR3 ECC Registered Memory (6 x 4GB)
• RAID Controller: Dell PERC H700 6Gbps SAS/SATA RAID Controller 512MB Cache (0 1 5 6 10 50 60)
• Hard Drive Bay x6 1: Dell 3.5 Tray with screws 
• Daughter Card: Embedded Broadcom NetXtreme II 5709C Gigabit Ethernet NIC
• Management: iDRAC 6 Express Management Module
• Networking: Additional Network Card Not Included
• Power Supplies: 2x Redundant 870W Power Supplies
• Rail Kit: Rail Kit Not Included
• Front Security Bezel: Bezel Included with Key

The configuration started at $50, which is crazy. The CPUs were a $40 upgrade, the ram cost $54, upping the PSUs to 870 watts was $15, and the drive bays were $10 a piece. The amazing thing about this configuration is that it comes with a 4 port gig Ethernet adapter built in.

This cost me around $245 shipped, which is less than I was finding some empty chassis for online. The issue with this of course, is that it had no drives. They did have an option for shipping it with drives, (4Tb @ $80) which wasn't a bad price. However, I wanted to try something else I had found online, which was shucking large external hard drives (which cost less for some reason) and using them as internal storage. While getting CPUs and ram used isn't really a big deal, write hours on hard drives can be, so sadly I had to buy that new.

The Storage

Bad idea - $6.99 - 1x Kingston Digital 16 GB DataTraveler SE9 G2 USB 3.0 Flash Drive (DTSE9G2/16GB)
Good idea - $67 -  WD Blue 3D NAND 500GB PC SSD - SATA III 6 Gb/s, 2.5"/7mm - WDS500G2B0A 
$8.95 - Protronix SATA Optical Bay 2nd Hard Drive Caddy, Universal for 12.7mm CD/DVD Drive Slot
$110 each - 6x WD 6TB Elements Desktop Hard Drive - USB 3.0 - WDBWLG0060HBK-NESN

The seeming outlier here is the USB flash drive, but the cool thing is that it can be used as an OS drive, ~~so that I don't have to take up any of the storage bays for the operating system.~~ The original plan was to use the flash drive to store the base OS. The flash drive couldn't handle the read/write cycles of the main os and died after about a month. I replaced it with the optical bay ssd holder and ssd combo, which has been working flawlessly ever since. I also didn't lose any data, which was a big relief.

The WD elements drives all arrived earlier than everything else, which allowed me to test the drives in their enclosures to make sure they were all functional. I did a simple file transfer (around 10Gb) and S.M.A.R.T. status check on each of them. Afterwards it was time to coax them out of their shells. I followed a youtube video on the process by the same person who wrote the serverbuilds.net guide, JDM_WAAAT, who sadly, doesn't appear to have an account here. The process was actually easier for the 6Tb drives I had, and only required removing two screws.

The drive inside is not the best, but for my use will be fine. It also costs $50 more if you buy it without the enclosure, which still makes no sense to me.

The Network

Netgear r7500v2 (already owned)
$40 - Dell PowerConnect 2748 48-port gigabit Ethernet switch
$20 - Cable Matters 5-Pack Snagless Cat 6a / Cat6a (SSTP/SFTP) Shielded Ethernet Cable in Blue 10 Feet

Given that I was planning to create a Link Aggregation Group (LAG) with the four gig ports that came with this box, I knew I was going to need some additional networking hardware. My router is fine, but only has four ports total - most of which are already in use, and doesn't support link aggregation.

I went back to ebay and found a preowned Dell PowerConnect 2748 for around $40 shipped which was more than enough for what I need now, and the future. Even better, the switch is managed, so I can set up my link aggregations, and VLANs if I so choose.

For cables, I picked up some additional cat 6a cables, which are better than cat 5e in every way, and cost just slightly more. They also support 10Gbe, which means if 10Gbe networking is ever affordable, I can upgrade without having to buy new cables.

The Software

• FreeNAS 11
  • NextCloud
  • Plex Media Server
  • Docker

I originally was going to use unRAID for this setup, as it's ease-of-use was attractive to me, as was it's ability to add a single disk to an existing drive array. However, further research showed that the downsides of this benefit include reduced write speed to drives as you add more devices. This is an issue because one primary goal I have for this system is backups from my other devices, as well as a replacement for dropbox so I can stop shipping them my money. Frequent writes are common in both of these situations, which ultimately made me lean to FreeNAS, which has better (supposedly) write speeds, and most of the same feature set. It also happens to be completely free (unlike unRAID) and is OSS.

The two services (plugins in the world of freeNAS) that I've started with are nextCloud and plex, though plex is currently protesting.

Docker comes installed and supported out of the box on FreeNAS, which is great.

Extras

$137 - CyberPower CP850PFCLCD PFC Sinewave UPS System, 850VA/510W
$50 - UNIVERSAL DEVICE PLACEMENT PORTABLE 12U FLOORSTANDING 19" SERVER RACK
$40 - Dell R610 R710 Static Server Rail Kit 2 Post 4 Post R G483G L K291G
$7.50 - Pasow 50pcs Cable Ties Reusable Fastening Wire Organizer
$70 - SiliconDust HDHomeRun Connect Duo 2-Tuner

The first four items on this list aren't really "extras" but don't really fit anywhere else. I'd argue a UPS is required to protect a system like this. A rack, though small - does wonders for space management, and organization. The rails go with the server, but also with the rack, since I didn't buy the kit from MET.

The cable ties - which are reusable - is a purely quality of life item, but makes dealing with networking and power cables much easier.

The TV tuner is definitely an extra, and I wouldn't even want it if I wasn't going to use it with plex's excellent live TV & DVR system.

Install and Troubleshooting

Installing freenas 11 was straight forward, but not pain free. If you go the route I initially tried, which is running the install off of a flash drive, you technically need two flash drives. One to put the installer image on, and one to install the OS to. Fortunately, I had another flash drive that worked for this purpose. A pain point of this install though, was that the system - being the older hardware that it is - does not have usb 3. As such, installing onto the flash drive using usb 2.0 was a very time consuming process. Installing to the ssd later was faster, mostly because of its SATA interface, but also because I discovered the remote management utility built into this server let's you mount an ISO over the network, which meant I didn't have to use USB at all to install the system, which was great.

The hard drives were easy enough to shuck as I mentioned above, however I did break a drive caddy when I accidentally installed the drives too far forward in the caddies, and thus wasn't making contact with the backplane. One of the screws seized on a caddy and I stripped it (which was totally my fault), this cost me another $7 on ebay to get a caddy.

Once the drives were all properly installed in the drive bays, I did have to configure them each to be their own RAID 0 array to get them to pass through to freenas, something freenas really doesn't like because it removes some drive monitoring. It would be better if I was using a RAID controller that supported direct pass through, but this one doesn't and at this point I'm tired of spending money on this project.

The ZFS pool configured easily. I'm only using one parity disk, something that at this size of pool (36 total TB) is frowned upon, but losing 12TB of space to parity isn't something I'm currently willing to do.

The freenas interface is fairly easy to use, and after stumbling through creating the data pool above, the next challenge was the software I actually wanted. Plex and nextcloud.

Nextcloud was easy, freenas has nextcloud as a "plugin" which uses a concept of a jail - similar to a virtual machine - to run the software you want in isolation from the rest of the system. The only configuration I had to do here was setting up a username and password during the nextcloud web based install. Super easy.

Plex was a different matter. I wanted to be able to access plex files (things like pictures and videos) directly off of a network drive. This meant I had to setup a windows samba share in freenas. Freenas actually makes this trivial, and has an entire wizard to help you do this. I setup a user, named a share, and could connect to it from my network. Awesome. At this point I thought I was set, as plex also has an official plugin provided by freenas, just like nextcloud. No matter what I did though, I could not get the web interface to load to get past the initial configuration of plex. This was very frustrating and almost made me give up.

I finally found a guide that went through setting up plex in your own jail, so essentially creating the plugin, but from scratch. After a little trial and error, this method worked for me. Which meant both of my services were up and running as I had hoped.

Conclusion

This was a long process - mostly thanks to USB 2, and a failing flash drive - that involved a lot of headaches and trial and error. So much so that I know I've missed details when creating this build log. If you have any questions on any step, or anything I did that seems to skip forward in time, please drop a comment and I'll try to update and address it. I ended up not setting up a docker environment on this system, and am instead using jails for that purpose. Due to the way that freenas allocates memory and cpu resources, jails are much simpler. Got a cool idea for something I should do with this? Also please drop a comment.

Thanks for reading, and coming along with me on this adventure.

Creating a Usable Backup Setup for Linux (ElementaryOS 5)

Nathan Bland — Wed, 13 Mar 2019 00:56:34 +0000

Creating a Usable Backup Setup for Linux (ElementaryOS 5)

Creating a backup solution is something most of us don't think about until we need it.

This is a guide that is meant to be a step by step solution to that problem before it happens.

Note: This is made for elementaryOS, but should work on any Debian based Linux distro, however YMMV if you are using something else.

Step 1: Backup Media Device

Before we even begin with some backup software, we need to have something to back it up to. This can be a network location like a samba share, a secondary disk in the system itself, or a removable disk like an external hard drive.

In my case, I'm using a 4Tb Western Digital Passport drive.

Step 2: Install The Software

There are a ton of backup solutions in terms of software for Linux, most being based on either rsync, or duplicity. For this guide, we are going to be using a tool called Déjà Dup Backup Tool. It's based on duplicity, which means we can back up to any kind of file store (even NTFS) and retain permissions, take snapshot backups, and keep file history, which is a very good thing for us. In my case, it allows me to make backups from both my windows install, and Linux install to the save drive, without needing different partitions. An honorable mention goes to Timeshift, which I would have used if it provided any kind of support for NTFS drives

Getting this installed in Elementary is straight forward. First, we need to open up the AppCenter, either by finding it on the dock, or by going to the applications tab and typing in the name.

Once we've got that open, we are going to type in Backup in the search bar in the top right of the AppCenter. This will provide a list of many, many apps, most of which are "Non-Curated". Déjà Dup falls into this category as well, and for me was the 5th listing from the top.

After locating the listing, click on the Free button on the far right, this will prompt you for your administrator (sudo) credentials to install the software. Enter them and hit Authenticate.

You you will be shown a progress bar for downloading, and then installing. Once the install is complete, you can close the AppCenter.

Step 3: Configuring Backup

Now that we have our backup tool installed, we need to open it. Click the Applications menu button in the top left, and look for Backups or type it in to search.

We are greeted by an interface that has only two visible buttons, and several menu items. The buttons, Restore... and Back Up Now... provide quick ways to get started.

Before we click through one of these, let's check our our menu items:

Overview - where we are now
Folders to save - The things we want backed up, by default, this only backs up your home folder.
Folders to ignore - Things you don't want included in your backup.
Storage location - Where your backups will live.
Scheduling - How often you want the system to backup

3.1: Scheduling

Let's start with that last menu item, and work our way up. First, turn Automatic backup on. Now, how often we want this to run can vary.

This will depend on your needs, and available storage. If you have more storage you may be able to do daily backups and keep them forever, or if you have much less you may only do weekly, and only keep a few

A lot of this depends on what you feel is appropriate for your data. How often do you change files on your computer? Is that data saved somewhere else like google docs or GitHub?

Personally, most of the work I do in Linux is code, and I use git for that. However, I do still have a lot of personalization that would take a lot of effort to get back, so for me, I'm going to set mine to backup every Week and keep for At least six months. This gives me a reasonable amount of data, without going too far for what I do. Remember that this software uses snapshot backups, so they won't take as much storage as you probably think.

3.2: Storage Location

A cool feature of Déjà Dup is that you can backup to a cloud account (Google Drive or Next cloud) if you desire. I'm just going to do a local backup, but feel free to use one of those options if you want. Keep in mind that backups to a remote server will probably be much slower.

At this point of Déjà Dup we have two options, Storage location and Folder. You should adjust at least your storage location to point to your external drive.

In my case the Storage location is set to my Passport external drive, and folder is just my computer name, nathan-Z170XP-SLI. You can change that folder name to whatever you want.

3.3: Folders to ignore

This menu is slightly deceiving. Anyone who has dealt with Linux backups before will know there are some system folders you should definitely ignore if you want a system backup to be successful. Fortunately, Déjà Dup provides are larger list, that is always ignored. You can view this list by clicking the top right icon, selecting help, and browsing to Settings. The help guide is generally useful and well written, I would recommend it if you have any other questions.

I'm going to add one additional entry by hitting the plus, /media, as I'm going to include my entire system, and I don't want to backup the drive I'm backing up to.

3.4: Folders to save

By default, Déjà Dup only backs up your home directory. I want more than this included in my backup, so I'm going to hit the plus icon near the bottom of the interface, and add an entry for root /.

This is going to tell it to backup everything - excluding the folders on the ignore list - on the system, so you may not want to set a backup to this level.

3.5: Backup.

Clicking back to the Overview section, we can now run our backup. Click Back Up Now..., you will be informed that the system requires duplicity with an option to install it in the top right, click Install. Just like before you will be prompted for your administrator credentials, and be greeted by a progress bar.

Once this install finishes you will be asked if you want to require a password for your backup. Depending on the nature of your work, and your tendency to remember or forget passwords, I would recommend password protecting this. If you don't use a password manager, you should start. However, if you don't feel you either need, or would remember a password, then select Allow restoring without a password.

In my case, I gave a password. Once you have an option selected, and a password given, the Forward button becomes active, click it.

This will start the backup itself. Depending on how much data you have, and which storage location you selected, this will take a long time.

You can expand the details if you want to view exactly what it is doing.

Step 4: Testing The Restore

Do not test this on the original source drive

This may not be a practical use for everyone, as not everyone has extra drives laying around to try a backup with. If at all possible you should try restoring the system with your backup to know that it works. This particular solution requires you install an operating system first, and then restore your backup. From there you can choose a full system restore, or cherry pick individual files to restore instead.

Please, try it. Not only does this give you experience with using the restore function so you know how should the time actually arise when you need to. It will let you walk through any issues you encounter knowing your data is actually still safe on the original drive. This let's you create an action plan - even if it's just in a google keep note, or dropbox - for when you need to do a real restore, and can help reduce stress significantly during that time.

Conclusion

We've made it, and you can now sleep a little easier knowing if your hard drive (or ssd) decides not to wake up tomorrow, you have a place to rescue your data from.

I hope this guide was useful for you, I'm by no means an expert in the backup space, this is just a solution I use and is simple enough to run without getting in the way. If you have suggestions/corrections/comments please feel free to let me know below!

Thanks for reading.

Photo by Chris Yates on Unsplash

Episode 14: Building a JSON API - Twitter OAuth (OAuth 1.0)

Nathan Bland — Thu, 07 Feb 2019 15:40:05 +0000

Sometimes users don't want to make yet another account. In this episode we'll cover how to let them use a twitter account with OAuth 1.0 instead.

Thanks for watching!

Source code:

NathanBland / core

Repository for the "core" video series

core

An express.js powered json api built for the "core" series.

Getting started

$ git clone git@github.com:NathanBland/core.git
$ cd core

$ docker-compose up

View on GitHub

Episode 13: Building a JSON API - Quickly Adding HTTPS

Nathan Bland — Tue, 29 Jan 2019 01:41:42 +0000

In this episode, we cover adding HTTPS using nginx with docker-compose to our api.

Thanks for watching!

Source code:

NathanBland / core

Repository for the "core" video series

core

An express.js powered json api built for the "core" series.

Getting started

$ git clone git@github.com:NathanBland/core.git
$ cd core

$ docker-compose up

View on GitHub

Episode 12: Building a JSON API - User Authentication in Under an Hour

Nathan Bland — Tue, 04 Dec 2018 10:01:44 +0000

Grab a cup of coffee, maybe two.

In this episode we add local-user authentication (username, password) to our JSON API in under an hour with help from passport.js, jwt, and a strong dose of patience.

This episode had so much, I almost broke it up several times while editing as it felt like it was too much. However, having an incomplete basic solution at the end of several episodes also felt wrong, so here we are. One hour later, one authentication enabled api.

Thanks for watching!

Source code:

NathanBland / core

Repository for the "core" video series

core

An express.js powered json api built for the "core" series.

Getting started

$ git clone git@github.com:NathanBland/core.git
$ cd core

$ docker-compose up

View on GitHub

Episode 11.5: Building a JSON API - Input Validation and Sanitation - Lightning Round

Nathan Bland — Fri, 30 Nov 2018 07:10:12 +0000

Are half episodes a thing? (Apparently) Is 10 minutes a lightning round?

If 10 minutes can be called a lightning round, then this is it. We cover all of our item route in what previously took us two episodes to do for user. In particular we speed past some testing portions that work to speed things up. Hang on to your seats..

Whew. That was a lot. Think I should have included more, or done something extra? Let me know.

Thanks for watching!

Source code:

NathanBland / core

Repository for the "core" video series

core

An express.js powered json api built for the "core" series.

Getting started

$ git clone git@github.com:NathanBland/core.git
$ cd core

$ docker-compose up

View on GitHub

Episode 11: Building a JSON API - Input Validation and Sanitation - Part 2

Nathan Bland — Thu, 29 Nov 2018 23:06:35 +0000

Input validation and sanitation, mostly sanitation. Finishing up the Read route, then continuing on to Update, and Delete.

There is still more to cover, in particular the item validations, but given that is a dynamic object almost entirely, almost all we could do is escape the values that are passed to it. Does it deserve its own episode? Let me know.

Thanks for watching!

Source code:

NathanBland / core

Repository for the "core" video series

core

An express.js powered json api built for the "core" series.

Getting started

$ git clone git@github.com:NathanBland/core.git
$ cd core

$ docker-compose up

View on GitHub

Episode 10: Building a JSON API - Input Validation and Sanitation - Part 1

Nathan Bland — Thu, 29 Nov 2018 08:21:14 +0000

Dropping user provided data directly into your database is a bad idea. In this episode, we'll look at using validator.js to help with that. We'll also spend a little time talking about npm packages, and trust.

I really didn't want to do this in more than one part, but it was going to be a very, very long episode. Part 2 will be along very soon (Source code already has it). As always, drop a comment if you would like to see me cover something else, or cover something differently.

Thanks for watching!

Source code:

NathanBland / core

Repository for the "core" video series

core

An express.js powered json api built for the "core" series.

Getting started

$ git clone git@github.com:NathanBland/core.git
$ cd core

$ docker-compose up

View on GitHub

Forem: Nathan Bland

Setting up Drone CI for CI/CD homelab use - build log

Step 1 - setup and configuration

Step 2 - First user setup

Step 3 - Drone runner node

Step 4 - Test an actual pipeline

Conclusion

Setting up GoCD in a docker environment - build log

Step 1 - Docker things

Step 2 - Portainer things

Step 3 - Agents

Step 4 - Users

Step 5 - Pipelines!

🎥 How to Build Your Own Continuous Automation & Build Server - Using Jenkins & Docker - #1

Video

🎥 Setting up a Kubernetes Cluster - Live Stream Replay (with notes)

Video - notes below

Notes and Summary

Summary

1. Setup support for installing packages over https

2. Add trusted key for packages we want to install

3. Add repository for Kubernetes(k8s)

4. Update package listings

5. Install kubernetes, its tools, and docker.io

6. Disable swap memory. (it isn't supported)

7. Enable and start the docker service.

8. Setup the Docker Daemon

9. Reload the docker daemon and system process

10. Clone our VM.

Steps on our Manager node

1. Initialize our k8 cluster

2. Establish Cluster Network - Using Weave

3. View the networking services we just spun up

Steps on our Worker nodes

1. Join our cluster

Back to the Manager Node

Links:

Let's Build a Home Lab Server - A Build Log

Research

The Build

The Brain

The Storage

The Network

The Software

Extras

Install and Troubleshooting

Conclusion

Creating a Usable Backup Setup for Linux (ElementaryOS 5)

Creating a Usable Backup Setup for Linux (ElementaryOS 5)

Step 1: Backup Media Device

Step 2: Install The Software

Step 3: Configuring Backup

3.1: Scheduling

3.2: Storage Location

3.3: Folders to ignore

3.4: Folders to save

3.5: Backup.

Step 4: Testing The Restore

Conclusion

Episode 14: Building a JSON API - Twitter OAuth (OAuth 1.0)

NathanBland / core

Repository for the "core" video series

core

Getting started

Episode 13: Building a JSON API - Quickly Adding HTTPS

NathanBland / core

Repository for the "core" video series

core

Getting started

Episode 12: Building a JSON API - User Authentication in Under an Hour

NathanBland / core

Repository for the "core" video series

core

Getting started

Episode 11.5: Building a JSON API - Input Validation and Sanitation - Lightning Round

NathanBland / core

Repository for the "core" video series

core

Getting started

Episode 11: Building a JSON API - Input Validation and Sanitation - Part 2