Forem: Vitor Monteiro

Setup Graylog on Synology DSM 7.x with Docker for UniFi logs

Vitor Monteiro — Fri, 20 Jan 2023 16:20:14 +0000

If you're just looking for a functional docker.compose.yml that works on Synology DSM 7, you can find it here.

I recently encountered some issues with my UniFi devices and attempted to review the logs for troubleshooting. However, I found that the UniFi OS lacks efficient log management capabilities. With this in mind, I recalled that Lawrence Systems utilizes Graylog for indexing and reading UniFi logs, so I set out to do the same myself.

I run most of my shared services on my Synology NAS that's running DSM 7. To ensure reliability and ease of restoration in case of a data loss, I have invested time in running all my services via docker-compose and regularly backup all my service configurations. While adding Graylog to my setup through this method should have been relatively simple, the particular context of DSM and my specific NAS created some complications.

So how hard can it be to just add a few more lines to the docker-compose.yml file and spin up Graylog for all my log ingestion needs? Not hard, but troublesome in the particular context of DSM and your particular NAS...

Setup Overview

This is my target setup:

Run Graylog's most recent version as a shared service in my NAS
Use docker and docker-compose to define all required services
Forward logs from my UniFi controller to Graylog
Consume those logs via any internal client in my network

Issues running vanilla configuration on DSM 7

When I was looking for official docs on the matter I found two links which give you slightly different instructions:

They differ in some aspects, but ultimately, as I tried to run and fix the provided docker-compose.yml for DSM 7, I started encountering some issues 👇.

MongoDB 5 requires AVX Support on the NAS CPU

WARNING: MongoDB 5.0+ requires a CPU with AVX support, and your current system does not appear to have that!

So what exactly is AVX? 🤔

Advanced Vector Extensions (AVX) are extensions to the x86 instruction set architecture for microprocessors from Intel and Advanced Micro Devices (AMD). They were proposed by Intel in March 2008 and first supported by Intel with the Sandy Bridge[1] processor shipping in Q1 2011 and later by AMD with the Bulldozer[2] processor shipping in Q3 2011. AVX provides new features, new instructions and a new coding scheme. - Wikipedia

It seems this is not a thing in Celeron CPUs and as such support on Synology NAS will be reduced, especially for the home and semi-pro variants of their products.

✅ Solution: Downgrade MongoDB to latest 4.x which doesn't require AVX support.

Graylog 5 requires MongoDB 5

org.graylog2.bootstrap.preflight.PreflightCheckException: You're running MongoDB 4.4.18 but Graylog requires at least MongoDB 5.0.0. Please upgrade.

As an end-user who primarily wants to ingest logs into Graylog and be able to query them, I was not overly concerned with this downgrade.

✅ Solution: Downgrade Graylog to latest 4.x which doesn't require MongoDB 5.x

Incorrect mapping of elasticsearch folder

This is the issue that really got me. When playing with docker every time I have path access issues it's either the user I'm defining lacking permissions of the local folder I'm mapping simply doesn't exist. While troubleshooting this, I went nuts trying to find why docker didn't have access to the folder in question, to the point I gave up. After a day or so I picked up this project again, and then I found a Stackoverflow post suggesting to change the path on the docker image itself:

Solution ✅ : Change the elasticsearch internal path from /usr/share/elasticsearch/data to /var/lib/elasticsearch/data

Graylog not able to write files in the mapped volume

Unfortunately, this is an issue for which I do not have a solution on DSM. As it stands, graylog doesn't use the root user in the container, so it creates a user:group 1100:1100. I've read plenty of solutions to just chown the folder to that user:group but this didn't solve it.

I then tried to create a group and user on DSM with addgroup, but then I found that DSM does not allow you to create users and groups with specific ids. I went to the trouble of trying to create some of the files that graylog was trying to generate like the graylog.conf, but as I was doing that I've figured this will be a non-ending problem.

ERROR: Unable to access file /usr/share/graylog/data/journal/graylog2-committed-read-offset: Permission denied #2155

woody3549 posted on Apr 28, 2016

Problem description

ERROR: Unable to access file /usr/share/graylog/data/journal/graylog2-committed-read-offset: Permission denied

Steps to reproduce the problem

While doing a docker-compose up Graylog server is stopping with the above error

Here is the docker-compose file

mongo:
  image: "mongo:3"
  volumes:
    - /graylog/data/mongo:/data/db
elasticsearch:
  image: "elasticsearch:2"
  command: "elasticsearch -Des.cluster.name='graylog'"
  volumes:
    - /graylog/data/elasticsearch:/usr/share/elasticsearch/data
graylog:
  image: graylog2/server:2.0.0-rc.1-1
  volumes:
    - /graylog/data/journal:/usr/share/graylog/data/journal
    - /graylog/config:/usr/share/graylog/data/config
  environment:
    GRAYLOG_PASSWORD_SECRET: somepasswordpepper
    GRAYLOG_ROOT_PASSWORD_SHA2: 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
    GRAYLOG_REST_TRANSPORT_URI: http://127.0.0.1:12900

  links:
    - mongo:mongo
    - elasticsearch:elasticsearch
  ports:
    - "9000:9000"
    - "12900:12900"

Environment

Graylog Version: 2
Elasticsearch Version:2
MongoDB Version: 3
Operating System: CentOS 7
Browser version:

View on GitHub

Working docker file

As it is, I've failed to setup a persistent configuration for graylog on DSM. If anyone has solved this please let me know in the comments. The docker file you see below will use volatile storage, so you'll loose your logs and configurations upon a single container stop.

Setting up a new input on Graylog

After Graylog is running, and you login via http://hostname:9000, you'll be alerted there are no inputs set. Navigate to System -> Inputs:

On the Inputs screen, find the dropdown with Select Input and pick Syslog UDP:

Give it whatever name you'd like. The input is pretty much ready to go, with the exception of the default port. Change it from 514 to 1514:

This is it on the Graylog side, lets go to the Unifi controller to forward the logs.

Setting up log forwarding on UniFi controller

When I say UniFi controller, I mean any of their controller products: CloudKey, UniFi Dream Router or UniFi Dream Machine. I'm personally a CloudKey user.

Go into Network -> Settings and expand support, until you see this block. Tick the Syslog checkbox, and fill in your NAS hostname, as well as the port 1514

Checking our logs on Graylog

If everything was configured properly you should see your UniFi logs on graylog. If you have any comments and improvements for the post, I'm always looking for advice and guidance 🙇

GitHub Codespaces: Developing and testing React Native apps just with Expo and Codespaces

Vitor Monteiro — Thu, 13 Aug 2020 08:04:39 +0000

I'm personally really excited about GitHub Codespaces. This is coming from someone who's always been passionate about their IDEs and local setup. This past weekend I was hacking some really amateurish React Native using Expo.io. I decided I was tired of locally setting up IDEs on every new or formatted device. It's 2020, I should be able to get my dev environment completely setup in the cloud, so I can jump to action on any device, anywhere with the same user experience.

What I'm about to show, especially for folks experienced with Expo and React Native, is nothing special or new. Expo is an amazing framework and and most of the magic of this tutorial is due to those tools. Having said that, I'm still blown away with the fact that to run a test or do an experiment on a native mobile app, I just need to launch a codespace, run expo and see the changes live on my phone without even cloning a repo.

This 30 sec video will explain what this guide is all about:

What do you need before you get started?

A GitHub account (I guess everyone has one of those), otherwise go create one;
Access to the GitHub Codespaces beta. If you don't have access to the GitHub Codespaces beta, you can request early access.
An Expo.io account and Expo installed on your mobile device

If you don't have access to the GitHub Codespaces beta, I still think it's worthwhile checking the rest of the tutorial for when it becomes available.

1. Jumping straight into a Codespace 🚀

Usually guides like this start with Clone this repository or a screenshot of how to fork a project. But we're talking about Codespaces and with it, you can launch any project, not just yours, directly on a new codespace. Go straight to https://github.com/bitoiu/expo-start-octocat and click Code -> Open with Codespaces:

Not long after you click that, you'll see your codespace ready for code:

2. Installing the expo-cli

The expo client is doing the heavy lifting in this demo, so we'll need to install it by getting a Codespaces terminal and up running first. Open a terminal with ^` or View -> Terminal, then run:



npm install -g expo-cli

3. Login to Expo

At this point I'm assuming you have an Expo.io account, and you also downloaded the mobile client to your phone. Now let's ensure you're logged in both on your phone, and on our codespace. To login from the expo-cli in our codespace just run:



expo login

4. Start your app and check this out 😍

Back in the terminal run:



expo start --tunnel

If everything worked as expected you should be seeing a QR code. Scan the QR code with the Expo app (Android) or the Camera app (iOS). In any of the scenarios, you should be seeing the application loading and a status bar on the Codespace terminal.

The first time loading the app could take a bit of time, and you might have to click Reload JS if you get an error in slow connections. In fact, via this method, the codespace and your phone might disconnect with inactivity so if you see a Disconnected from the Metro server just literally shake your phone to get the Expo devtool menu and click Reload. When it all works you should see this screen:

For the grand finale, go into your codespace and locate what should be line 20 where you have backgroundColor and change it to pink for example then save the file (⌘S). If the phone is still connected you should see it change color automatically.

ℹ️ Remember if the device disconnected from our codespace, just shake it to bring up the Expo menu and click reload.

Exciting future

I find it awesome that I could create or clone a fully fledged app, on a fully configured cloud IDE. Not only that, but I can easily test a small change or develop across the world in a minimum spec laptop I just borrowed.

I think Codespaces have the potential to be a key catalyst in the years to come for:

Faster open source contributions;
Faster ramp-up and time to productivity within companies;
For those that prefer VS Code and Codespaces, the ability to have your settings and your IDE as you've customized it anywhere, anytime;

Improvements

As I write this there are some things I still haven't completely figured out, but I plan to and will update the article accordingly. Those involve accessing the Expo Dev Tools on the codespace and ensuring the tunnel connection is more stable.

Stay safe ;)