Forem: Oluwajuwon Odunitan

Linux Health Sentinel Phase 2: From Metrics to Meanings with Grafana Loki

Oluwajuwon Odunitan — Thu, 05 Feb 2026 17:17:33 +0000

In my last post, I shared how I moved from blindly running commands to seeing my infrastructure breathe through metrics. But as every DevOps learner quickly discovers, metrics tell you there is a problem; logs tell you what the problem is.

Today, we’re giving our "Linux Health Sentinel" the ability to listen. We are adding centralised logging using Loki and Promtail.

The Concept: Metrics vs. Logs
If your server’s CPU spikes to 99%, Prometheus will show you a scary red line on a graph. That’s a metric. But why did it spike? Was it a brute-force SSH attack? A memory leak in a script?

To find out, you need the text records, the logs.

The Architecture
We are adding two new components to our existing setup:

Loki (The Library): Lives on your laptop. It stores the logs and lets you search them.
Promtail (The Spy): Lives on the Vagrant VM. It "tails" the log files (like tail -f) and ships them to Loki.

Prerequisites

Ubuntu laptop with Grafana + Prometheus running
Vagrant VM from Phase 1 (or any local VM of your choice)
Basic networking between host and VM

Step 1: Setting up the Library (Loki)
On your control centre (laptop), we need to get Loki running. Loki is "Prometheus, but for logs." While some package managers have Loki, the safest and most consistent path is using the official binaries.

# Download and unzip Loki
wget https://github.com/grafana/loki/releases/latest/download/loki-linux-amd64.zip
sudo apt update && sudo apt install unzip -y
unzip loki-linux-amd64.zip
chmod +x loki-linux-amd64

# Download the default config file
wget https://raw.githubusercontent.com/grafana/loki/main/cmd/loki/loki-local-config.yaml

Run Loki:

./loki-linux-amd64 -config.file=loki-local-config.yaml

PS: This setup is for local learning only and runs without authentication. Do not expose Loki directly to the internet.

Step 2: Deploying the Spy (Promtail)
Now, hop into your Vagrant VM. We need an agent to grab those system logs and send them over the network to your laptop.

Install Promtail:

curl -O -L "https://github.com/grafana/loki/releases/download/v3.5.9/promtail-linux-amd64.zip"
unzip promtail-linux-amd64.zip
chmod a+x promtail-linux-amd64

Configure Promtail: Download the basic config file to tell Promtail where to send the logs.

wget https://raw.githubusercontent.com/grafana/loki/main/clients/cmd/promtail/promtail-local-config.yaml

Edit the Config: Change the clients URL to your laptop's IP address. Tip: Use hostname -I on your laptop to find the IP address your VM needs to talk to.

clients:
  - url: http://<YOUR_LAPTOP_IP>:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      host: vagrant-vm
      __path__: /var/log/*log

Run Promtail:

./promtail-linux-amd64 -config.file=promtail-local-config.yaml

Step 3: Visualisation in Grafana

Go to Grafana (localhost:3000).
Add Data Source -> Select Loki.
Set URL to http://localhost:3100. Click Save & Test.
Go to the Explore tab (compass icon).
Use the Label Browser to select job="varlogs" or host="vagrant-vm".
Click Run Query.

Generating Test Logs
You should get some logs. We can also generate some noise on the VM:

sudo logger "Sentinel Test: Can you hear me, Grafana?"
sudo logger "Hello Loki, this is a test"
sudo logger "Sentinel Alert: Testing log flow to Grafana"
sudo logger -p user.err "Simulating a critical system error"
sudo logger "Hello Loki, this is test-2."

Don't expect a "Matrix-style" scrolling screen immediately! By default, Grafana shows a static snapshot. To see logs fly across your screen in real-time:

Enable "Live" Mode: Look for the Live button in the top right of the Grafana UI.
Adjust Auto-Refresh: Set the timer to 5s or 10s.

Conclusion
We have metrics, and we have logs. Our Sentinel is getting smarter. But we still have to look at the screen to know something is wrong.

Next, we’ll teach it to speak to us on Slack or Discord or email when it detects trouble, with Alerting.

Monitoring and Observing My First Linux Server: A Beginner’s Guide to Prometheus & Grafana.

Oluwajuwon Odunitan — Tue, 03 Feb 2026 09:18:48 +0000

Have you ever wondered what’s actually happening inside a Linux server?
As a DevOps engineer and learner, you will eventually move past "just running commands" to actually observing your infrastructure to know what is happening under the hood.

So I built a simple observability stack using Prometheus and Grafana, which I called "Linux Health Sentinel". This guide walks you through that exact beginner-friendly setup to see how data flows from a raw event to a visual graph.

🔍 Monitoring vs. Observability: What's the Difference?
Before diving in, it's important to understand these two core concepts in DevOps and cloud environments:

Monitoring: Tells you if a system is working and/or if something is broken (e.g., "Is the CPU over 80%?").
Observability: helps you understand why. by looking at the data—the "signals"—the system emits.

Observability is a system, not just a tool:
sensors → data pipeline → dashboard

The 4 phases of observability

Phase 1 → Instrumentation (Sensors) Tools that emit signals from infrastructure and apps. (on the target)
Phase 2 → Collection (The Pipeline) Collectors clean, label, and route the data.
Phase 3 → Storage (The Library) Metrics, logs, and traces live in optimised databases.
Phase 4 → Visualisation & Alerting Dashboards + alerts turn data into action.

Project Goal
Visualise the real-time CPU, memory, and disc health of an Ubuntu VM using the industry-standard Prometheus and Grafana stack.

🛠️ The Tech Stack
Target → VM
Control Center → Laptop

To build my sentinel, I used the "Golden Trio" of open-source observability:

Node Exporter: The "spy/sensor and collector" on the VM (target) that gathers hardware stats.
Prometheus: The "brain and storage" that pulls data and stores it in a time-series database.
Grafana: The "face" that turns raw numbers into beautiful, readable dashboards.

🚀 Step-by-Step Implementation
Step 1: Set Up the "Target" (VirtualBox VM)
This VM represents a production server in a data centre.
a. Network Setup (Crucial): * In VirtualBox, select your VM > Settings > Network.
- Change "Attached to" from NAT to Bridged Adapter.
- Why? This gives the VM an IP address on your home network so your laptop can "talk" to it.
b. Start the VM and find its IP address:

hostname -I

Note this down; let's assume it is 192.168.1.50
c. Install the "Spy" (Node Exporter):
Node Exporter is a small tool that translates Linux system stats into a format Prometheus understands.

sudo apt update
sudo apt install prometheus-node-exporter -y

d. Verify: Open the browser on your laptop and go to http://192.168.1.50:9100/metrics. If you see a wall of text, the spy is active.

Step 2: Set Up the "Control Center" (Your Laptop)

Your laptop will act as the monitoring server that "pulls" data from the VM.

a. Install Prometheus (The Database):

sudo apt update
sudo apt install prometheus -y

b. Configure Prometheus to watch the VM: Edit the configuration file:

sudo nano /etc/prometheus/prometheus.yml

Scroll to the end and add this "job":

- job_name: 'ubuntu_vm'
  static_configs:
    - targets: ['192.168.1.50:9100']

Save and Exit (Ctrl+O, Enter, Ctrl+X).
c. Restart Prometheus:

sudo systemctl restart prometheus

Step 3: Visualise with Grafana

a. Install Grafana:

wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /usr/share/keyrings/grafana.gpg > /dev/null
sudo apt update && sudo apt install grafana -y
sudo systemctl enable --now grafana-server

b. Access Grafana: Open http://localhost:3000 (User/Pass: admin/admin).
c. Connect the dots:
- Go to Connections > Data Sources > Add Data Source.
- Select Prometheus.
- Set URL to http://localhost:9090. Click Save & Test.

d. Import the Pro Dashboard:
- Click the + (top right) > Import.
- Enter ID 1860 (this is the community standard for Node Exporter).
- Select your Prometheus data source and click Import.

💡 The Result
With just a VM, Prometheus, and Grafana, we built a real observability pipeline.
We can now see CPU, memory, and disk metrics in real time, exactly how production monitoring works in professional environments.

This small project is the foundation of modern DevOps observability. From here, you can explore:

alerting with Alertmanager
container monitoring
Kubernetes observability
logs and distributed tracing

Observability isn’t just about dashboards; it’s about understanding your systems deeply.

🌩️ Cloud Resume Challenge (AWS Edition)

Oluwajuwon Odunitan — Mon, 27 Oct 2025 13:15:46 +0000

✨ Introduction

Two years ago, I started a simple project to explore Infrastructure as Code (IaC) using Terraform. I recently revisited it with a fresh perspective — not just to complete it, but to dive deeper into managing existing cloud resources with IaC.

What sparked this? I noticed that many people skip the backend IaC part of the Cloud Resume Challenge, and I wanted to change that.

This challenge is a hands-on cloud engineering project that helps you build a real-world, serverless web application using AWS, Azure, or GCP. I chose AWS. It’s more than just a static site — it connects frontend, backend, CI/CD, and IaC concepts into one project, with security baked in.

🧱 Project Overview

Note: To begin, you’ll need a domain name (e.g., from Namecheap), an account with a cloud provider, and programmatic access credentials.

This project:

Hosts a personal resume website
Displays a visitor counter powered by AWS Lambda and DynamoDB
Uses a CI/CD pipeline for automated testing and deployment
Secures the site with HTTPS via AWS Certificate Manager

📐 Architecture Diagram

🔧 Core AWS Components

1. Amazon S3 (Static Website Hosting)

Website files are uploaded to an S3 bucket.
Public access is disabled; HTTPS is enabled via CloudFront.

2. AWS CloudFront (CDN)

Serves S3 content securely over HTTPS.
Uses an SSL/TLS certificate from AWS Certificate Manager.

3. AWS Certificate Manager (SSL)

Provides a free SSL/TLS certificate for the domain.
Attached to the CloudFront distribution for secure access.

4. AWS Route 53 (DNS)

Manages DNS settings linked to Namecheap.
Routes traffic to CloudFront via an alias record.

5. AWS Lambda + DynamoDB (Visitor Counter)

DynamoDB stores visitor count.
A Python-based Lambda function retrieves and increments the count.
Triggered via Lambda Function URL when the site loads.

This setup is fully serverless — no EC2 instances to manage!

🔄 CI/CD Pipeline with GitHub Actions

To automate testing and deployment:

On every push to main, GitHub Actions triggers two workflows:
- One syncs updated website files to S3.
- The other runs tests on the Lambda function and deploys infrastructure via Terraform (if tests pass).

This ensures fast, reliable deployments with every change.

📦 Infrastructure as Code with Terraform

Following the challenge, most people use ClickOps (manual setup via AWS Console) to get started. But when I reached the IaC stage, I realized I’d need to recreate everything in Terraform — even though my site had over 500 views!

This led me to discover the power of the terraform import command.

🧠 What is `terraform import`?

terraform import brings existing resources under Terraform’s management — even if they weren’t created with Terraform.

Syntax:

terraform import [options] ADDRESS ID

ADDRESS: Resource block name (e.g., aws_dynamodb_table.cloudResumeViewsTable)

ID: Unique identifier (e.g., ARN of the resource)

Example: Importing DynamoDB Table

In main.tf:

resource "aws_dynamodb_table" "cloudResumeViewsTable" {
  name           = "cloudResumeViewsTable"
  billing_mode   = "unknown"
  read_capacity  = "unknown"
  write_capacity = "unknown"
  hash_key       = "unknown"

  attribute {
    name = "id"
    type = "S"
  }
}

Import command

terraform import aws_dynamodb_table.cloudResumeViewsTable arn:aws:dynamodb:us-east-1:357078656374:table/cloudResumeViewsTable

This command imports the existing DynamoDB table into Terraform’s state, allowing you to manage it as code.

The terraform import command only imports the resource into the Terraform state file; it does not automatically generate the corresponding configuration in the .tf file, also the resources block must exist in the .tf file for the resources to be imported.

🧩 Terraform Import Use Cases

Migrating Existing Infrastructure: Bring manually created resources under Terraform control.
Hybrid Environments: Gradually transition resources to IaC.
Disaster Recovery: Re-import recreated resources after recovery.

🗺️ Planning Imports for Complex Deployments

Importing resources in a complex deployment can be tedious. You often can’t import everything at once — instead, bring resources into Terraform gradually and intentionally.
A well-drawn architecture diagram helps you:

Visualize the system
Break it into manageable pieces
Map out a clear import plan

✅ Conclusion

Revisiting this project helped me bridge the gap between manual cloud setup and full IaC adoption. By using terraform import, I now manage all resources through code — making future updates safer, faster, and more scalable.