Forem: shashankpai

Modern Infrastructure as Code: OpenTofu vs. Crossplane vs. Pulumi

shashankpai — Mon, 07 Apr 2025 11:38:52 +0000

Introduction
IaC Evolution Timeline
IaC Comparison At-a-Glance
OpenTofu: The Open Source Terraform Alternative
Crossplane: Kubernetes-Native Infrastructure
Pulumi: Infrastructure as Actual Code
Choosing the Right Tool
Quick Start Guide
Common Pitfalls to Avoid
FAQ
Conclusion

Introduction

In today's rapidly evolving cloud-native landscape, Infrastructure as Code (IaC) has transformed from a novel concept to an essential practice. As organizations embrace DevOps and platform engineering principles, selecting the right IaC tool becomes increasingly critical for operational success.

Who is this guide for? Platform engineers, DevOps practitioners, and technical decision-makers who need to select an appropriate IaC tool for their organization's needs.

This guide provides a focused comparison of three leading IaC tools—OpenTofu, Crossplane, and Pulumi—examining their architectures, capabilities, and ideal use cases.

🤔 Have you already started implementing Infrastructure as Code in your organization? What challenges are you facing?

IaC Evolution Timeline

Before diving into specific tools, let's understand how infrastructure management has evolved:

DevOps → SRE → Platform Engineering

DevOps established fundamental principles:
- Cross-functional collaboration
- Infrastructure defined as code
- Automation of repetitive tasks
- Rapid feedback loops for continuous improvement
Site Reliability Engineering (SRE) applied DevOps with emphasis on:
- System reliability and service level objectives
- Error budgets and acceptable failure rates
- Customer experience focus
- Incident response and postmortem analysis
Platform Engineering extended SRE concepts by focusing on:
- Internal developer customers
- Self-service capabilities
- Integrated toolsets and abstractions
- Developer experience optimization

IaC Comparison At-a-Glance

Factor	OpenTofu	Crossplane	Pulumi
Language	HashiCorp Configuration Language (HCL)	YAML with Functions	TypeScript, Python, Go, .NET, etc.
Learning Curve	Moderate	Steep (requires Kubernetes)	Low-Moderate
State Management	State file (local or remote)	Kubernetes reconciliation	Service-based state
Best For	Terraform users wanting open-source	Kubernetes-native environments	Teams with programming expertise
Community	Growing (Linux Foundation)	Strong (CNCF)	Commercial with open-source core
Governance	Open	Open	Commercial

OpenTofu: The Open Source Terraform Alternative

OpenTofu emerged as a community-driven fork of HashiCorp Terraform following licensing changes. Now under the Linux Foundation with plans to join the CNCF, OpenTofu preserves the familiar HCL syntax while ensuring an open governance model.

Project Structure Example

my-opentofu-project/
├── main.tf           # Primary configuration file
├── variables.tf      # Input variable declarations
├── outputs.tf        # Output value declarations
├── modules/          # Reusable modules
└── environments/     # Environment-specific configurations

Code Example: AWS VPC Configuration

# Simple VPC example
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = {
    Name        = "${var.environment}-vpc"
    Environment = var.environment
    Project     = var.project_name
    ManagedBy   = "OpenTofu"
  }
}

# Create two public subnets
resource "aws_subnet" "public" {
  count                   = 2
  vpc_id                  = aws_vpc.main.id
  cidr_block              = var.public_subnet_cidrs[count.index]
  availability_zone       = "${var.region}${count.index == 0 ? "a" : "b"}"
  map_public_ip_on_launch = true

  tags = {
    Name = "${var.environment}-public-subnet-${count.index + 1}"
  }
}

Key Architectural Concepts

Declarative Resource Definition: Define the desired state, and OpenTofu determines how to reach it
Provider Model: Plugins enable interaction with various cloud APIs
State Management: State file maps real-world resources to your configuration
Plan-Apply Workflow: Preview changes before applying them

When to Choose OpenTofu

When you need compatibility with existing Terraform code
When you prefer a declarative, configuration-based approach
When you want an open-source tool with broad provider support
When state management is acceptable for your use cases

🤔 What aspects of OpenTofu's approach most appeal to your team's workflow?

Crossplane: Kubernetes-Native Infrastructure

Crossplane extends Kubernetes by enabling it to provision and manage infrastructure resources across cloud providers. As a CNCF incubating project, it brings cloud resources into the Kubernetes resource model.

Project Structure Example

my-crossplane-project/
├── apis/                      # Custom resource definitions
├── package/                   # Package configuration
├── examples/                  # Example usage
└── infrastructure/            # Infrastructure definition

Code Example: PostgreSQL Database in AWS

# Define a PostgreSQL resource claim
apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
  name: production-db
  namespace: default
spec:
  parameters:
    storageGB: 50
    version: "13"
    tier: "medium"
  writeConnectionSecretToRef:
    name: production-db-conn

Key Architectural Concepts

Kubernetes Extension: Extends K8s API with custom resources for infrastructure
Composition-Based: Complex infrastructure composed from individual managed resources
Claim-Based Model: Developers "claim" infrastructure without understanding the complexity
Continuous Reconciliation: Controllers ensure actual state matches desired state

When to Choose Crossplane

When you're already using Kubernetes for application deployment
When you want to avoid state file management
When you need a continuous reconciliation model
When you want to provide self-service infrastructure to developers

🤔 If you're using Kubernetes, how would continuous reconciliation benefit your infrastructure management?

Pulumi: Infrastructure as Actual Code

Pulumi allows defining infrastructure using general-purpose programming languages like Python, TypeScript, and Go. This approach brings the full power of software development to infrastructure management.

Project Structure Example

my-pulumi-project/
├── __main__.py          # Main infrastructure code
├── Pulumi.yaml          # Project configuration
├── infrastructure/      # Infrastructure components
└── tests/               # Infrastructure tests

Code Example: AWS VPC in Python

import pulumi
import pulumi_aws as aws

# Create a VPC and subnets
vpc = aws.ec2.Vpc(
    "my-vpc",
    cidr_block="10.0.0.0/16",
    enable_dns_hostnames=True,
    enable_dns_support=True,
    tags={"Name": "my-vpc"}
)

# Create a public subnet
public_subnet = aws.ec2.Subnet(
    "public-subnet",
    vpc_id=vpc.id,
    cidr_block="10.0.1.0/24",
    availability_zone="us-west-2a",
    map_public_ip_on_launch=True,
    tags={"Name": "public-subnet"}
)

# Export the VPC ID
pulumi.export("vpc_id", vpc.id)

Key Architectural Concepts

General-Purpose Languages: Define infrastructure using familiar programming languages
Resource Model: Resources correspond to cloud provider offerings
Stack Management: Different environments managed as separate stacks
Testing Framework: Test infrastructure using familiar language testing frameworks

When to Choose Pulumi

When your team has strong programming skills
When you need to leverage programming language features (loops, conditionals)
When you want to use the same language for infrastructure and applications
When you need comprehensive testing of your infrastructure

🤔 Would using a familiar programming language improve your team's productivity with infrastructure code?

Choosing the Right Tool

Use this decision flow to help select the most appropriate tool for your situation:

If you need...	Then consider...
Compatibility with existing Terraform code	OpenTofu
Kubernetes-native infrastructure management	Crossplane
Infrastructure defined in a programming language	Pulumi
Open-source solution with broad community support	OpenTofu or Crossplane
Avoiding state file management	Crossplane
Advanced testing capabilities	Pulumi

Decision Flowchart

Consider these questions to determine which tool is right for you:

Are you already invested in Kubernetes?
- Yes → Crossplane is likely a good fit
- No → Continue to question 2
Do you have existing Terraform configurations?
- Yes → OpenTofu provides the smoothest transition
- No → Continue to question 3
Does your team prefer programming languages over configuration languages?
- Yes → Pulumi is probably your best choice
- No → OpenTofu offers a simpler learning curve

Quick Start Guide

OpenTofu Quick Start

Installation:

brew install opentofu/tap/opentofu

First Project: OpenTofu Getting Started Tutorial

Community Resources: OpenTofu Community Forum

Crossplane Quick Start

Installation:

kubectl create namespace crossplane-system
helm install crossplane --namespace crossplane-system crossplane-stable/crossplane

First Project: Crossplane Getting Started Guide

Pulumi Quick Start

Installation:

curl -fsSL https://get.pulumi.com | sh

First Project: Pulumi Getting Started Guide

Common Pitfalls to Avoid

OpenTofu Pitfalls

State File Corruption: Always use remote state storage with proper locking.
Provider Version Conflicts: Pin provider versions explicitly.
Module Management: Structure modules for reusability without excessive nesting.
Blast Radius: Use workspace separation for critical environments.

Crossplane Pitfalls

RBAC Complexity: Carefully plan your permissions model from the start.
Composition Learning Curve: Start with simple compositions before tackling complex ones.
Resource Drift: Understand how reconciliation handles manual changes.
API Versioning: Plan for provider API version changes.

Pulumi Pitfalls

Language-Specific Issues: Consider team expertise when choosing a programming language.
State Management: Handle secrets properly in your state storage.
Import Existing Resources: Have a strategy for importing existing infrastructure.
Dependency Management: Address language-specific package management challenges.

Conclusion

Infrastructure provisioning has come a long way, with IaC tools playing a crucial role in automation and scalability. OpenTofu, Crossplane, and Pulumi each offer unique approaches, making them suitable for different use cases.

The best tool for your organization depends on your team's skills, existing investments, and specific requirements:

OpenTofu: Best for teams with Terraform experience seeking an open-source alternative
Crossplane: Ideal for Kubernetes-centric organizations wanting unified management
Pulumi: Perfect for teams wanting to leverage programming language capabilities

As you evaluate these tools, consider starting with small, non-critical infrastructure components to gain experience before wider adoption.

What IaC tool are you using? Let us know your thoughts in the comments!

Further Reading:

From Zero to Hosted: Building a Static Website Platform with Pulumi and MinIO

shashankpai — Wed, 02 Apr 2025 15:56:20 +0000

Introduction

Have you ever wanted to create your own cloud infrastructure at home? Whether you're looking to learn more about cloud technologies, test deployments before pushing to production, or simply set up a personal storage solution, a home lab environment can be incredibly valuable. In this guide, I'll walk you through setting up a HomeLab MiniCloud using Pulumi as infrastructure-as-code (IaC) and Docker with MinIO as our object storage service.

What You'll Learn

How to set up Pulumi for infrastructure as code
Deploying MinIO (an S3-compatible object storage) using Docker
Configuring NGINX as a reverse proxy with SSL
Hosting a static website from your MinIO instance
Troubleshooting common issues

Prerequisites

Before we dive in, let's make sure you have all the necessary dependencies installed on your system. We'll need Docker for containerization, Python for our Pulumi code, and a few other utilities:

sudo apt update && sudo apt install -y \
  docker.io \
  python3.10 \
  python3.10-venv \
  curl \
  unzip

What is Pulumi?

Pulumi is an open-source infrastructure as code (IaC) tool that allows you to define and manage cloud infrastructure using familiar programming languages rather than domain-specific languages. In our case, we'll use Python to define our infrastructure, which means we can leverage Python's full feature set, including loops, conditionals, and functions, making our infrastructure code more flexible and maintainable.

Installing Pulumi

Let's install Pulumi using their official installation script:

curl -fsSL https://get.pulumi.com | sh
export PATH=$PATH:$HOME/.pulumi/bin

To make the Pulumi command available in your shell permanently, add it to your shell configuration file:

echo 'export PATH=$HOME/.pulumi/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

Setting Up Our MiniCloud Infrastructure

1. Creating the Project Directory

Let's start by creating a directory for our project:

mkdir -p ~/Home-Lab/Pulumi/homelab-minicloud
cd ~/Home-Lab/Pulumi/homelab-minicloud

2. Initializing a Pulumi Project

Now, let's initialize a new Pulumi project with Python as our language of choice:

pulumi new python -y

This command creates a new Pulumi project with the necessary scaffolding to get started.

3. Installing Dependencies

Navigate to the infrastructure directory and install the required Python packages:

cd infra
pip install -r requirements.txt

I recommend using a virtual environment to isolate our project dependencies:

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

4. Installing the Pulumi Docker Provider

Since we'll be deploying Docker containers, we need to install the Pulumi Docker provider:

pip install pulumi_docker

5. Defining Our MinIO Infrastructure

Now comes the exciting part! Let's define our MinIO infrastructure by editing the __main__.py file in the infra directory:

"""A Python Pulumi program"""

import pulumi
import pulumi_docker as docker

# MinIO credentials
minio_access_key = "minioadmin"
minio_secret_key = "minioadmin"

# Create a shared Docker network
network = docker.Network("homelab-network", name="homelab-network")

# Pull MinIO image
minio_image = docker.RemoteImage("minio-image",
    name="minio/minio:latest",
    keep_locally=True
)

# Run MinIO container
minio_container = docker.Container("minio-container",
    image=minio_image.repo_digest,
    name="minio",
    ports=[
        docker.ContainerPortArgs(internal=9000, external=9000),
        docker.ContainerPortArgs(internal=9001, external=9001),
    ],
    envs=[
        f"MINIO_ROOT_USER={minio_access_key}",
        f"MINIO_ROOT_PASSWORD={minio_secret_key}"
    ],
    command=["server", "/data", "--console-address", ":9001"],
    volumes=[
        docker.ContainerVolumeArgs(
            host_path="/home/arjun//minio/data",
            container_path="/data",
        )
    ],
    networks_advanced=[docker.ContainerNetworksAdvancedArgs(name=network.name)],
)

pulumi.export("minio_container_name", minio_container.name)

Let's break down this code:

We define MinIO credentials (in a production environment, you'd want to use Pulumi's secrets management)
We create a Docker network for our containers to communicate
We pull the latest MinIO image
We define and run the MinIO container with:
- Port mappings (9000 for the API, 9001 for the web console)
- Environment variables for credentials
- Volume mapping to persist data
- Network configuration

6. Deploying MinIO

Now let's deploy our infrastructure:

cd infra
pulumi up

This command will:

Preview the changes Pulumi will make
Ask for confirmation
Deploy the infrastructure according to our code

After deployment, you can verify that MinIO is running:

docker ps

You should see output similar to:

CONTAINER ID   IMAGE       COMMAND     STATUS    PORTS           NAMES
xxxxxxxxxxxx   minio/minio "..."       Up       0.0.0.0:9000->9000/tcp   minio

7. Accessing MinIO

Once deployed, you can access the MinIO console at http://localhost:9001 with the following credentials:

Username: minioadmin
Password: minioadmin

Enhancing Our Setup with NGINX and SSL

Now that we have MinIO running, let's enhance our setup by adding NGINX as a reverse proxy and implementing SSL for secure connections.

Generating a Self-Signed Certificate with SAN

For development purposes, we'll create a self-signed SSL certificate with Subject Alternative Name (SAN) support. First, create an OpenSSL configuration file named cert.conf:

[req]
default_bits       = 2048
distinguished_name = req_distinguished_name
req_extensions     = req_ext
x509_extensions    = v3_ca
prompt             = no

[req_distinguished_name]
C  = IN
ST = Karnataka
L  = Bangalore
O  = HomeLab
OU = IT
CN = minio.local

[req_ext]
subjectAltName = @alt_names

[v3_ca]
subjectAltName = @alt_names
basicConstraints = critical, CA:true

[alt_names]
DNS.1 = minio.local

Now, generate the certificate and key:

openssl req -x509 -nodes -days 365 \  
  -newkey rsa:2048 \  
  -keyout minio.key \  
  -out minio.crt \  
  -config cert.conf

This creates two files:

minio.crt - The certificate
minio.key - The private key

Setting Up NGINX as a Reverse Proxy

We have two options for configuring NGINX: subpath routing or subdomain split. Let's look at both approaches.

Option 1: Subpath Routing

With this approach, we'll serve both the MinIO console and our static site from the same domain but different paths:

server {
    listen 443 ssl;
    server_name minio.local;

    ssl_certificate     /etc/nginx/ssl/minio.crt;
    ssl_certificate_key /etc/nginx/ssl/minio.key;

    # MinIO Console
    location / {
        proxy_pass http://minio:9001/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # Static Site
    location /static/ {
        proxy_pass http://minio:9000/static-site/;
        proxy_ssl_verify off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

With this configuration:

The MinIO console will be accessible at https://minio.local/
The static site will be accessible at https://minio.local/static/

Option 2: Subdomain Split

Alternatively, we can use different subdomains for the MinIO console and the static site:

First, update your /etc/hosts file:

127.0.0.1 minio.local static.local

Then, configure NGINX:

# Static site on static.local
server {
    listen 443 ssl;
    server_name static.local;

    ssl_certificate     /etc/nginx/ssl/minio.crt;
    ssl_certificate_key /etc/nginx/ssl/minio.key;

    location / {
        proxy_pass http://minio:9000/static-site/;
        proxy_ssl_verify off;
    }
}

# MinIO Console on minio.local
server {
    listen 443 ssl;
    server_name minio.local;

    ssl_certificate     /etc/nginx/ssl/minio.crt;
    ssl_certificate_key /etc/nginx/ssl/minio.key;

    location / {
        proxy_pass http://minio:9001/;
    }
}

With this configuration:

The MinIO console will be accessible at https://minio.local/
The static site will be accessible at https://static.local/

After configuring NGINX, check the configuration and reload:

sudo nginx -t  # Check config
sudo systemctl reload nginx

Deploying a Static Website to MinIO

Now that we have MinIO and NGINX set up, let's deploy a static website to our MinIO bucket and make it publicly accessible.

Setting Up the MinIO Client (mc)

The MinIO Client (mc) is a command-line tool for working with MinIO and other S3-compatible storage services. Let's create a script to upload our static site to MinIO:

#!/bin/bash

set -e

# Connect to our MinIO instance (using --insecure for self-signed certificates)
mc alias set local https://minio.local minioadmin minioadmin --insecure

# Create a bucket for our static site (if it doesn't exist)
mc mb local/static-site --insecure || true

# Set the bucket to allow anonymous (public) downloads
mc anonymous set download local/static-site --insecure

# Upload our static site files
mc cp --recursive ../static-site/ local/static-site --insecure

echo "✅ Static site uploaded to MinIO!"

Handling Self-Signed Certificate Issues

Since we're using a self-signed certificate, you might encounter certificate verification errors. Here are two ways to handle them:

Option 1: Using the `--insecure` Flag

As shown in the script above, you can use the --insecure flag to skip certificate verification. This is not recommended for production but is fine for a development environment.

Option 2: Trusting the Self-Signed Certificate

A better approach is to add the certificate to your system's trusted certificates:

For Ubuntu/Debian:

sudo cp minio.crt /usr/local/share/ca-certificates/minio.crt
sudo update-ca-certificates

For RedHat-based systems:

sudo cp minio.crt /etc/pki/ca-trust/source/anchors/minio.crt
sudo update-ca-trust extract

After adding the certificate, restart your terminal and try the script without the --insecure flag.

Configuring MinIO for Website Hosting

MinIO has built-in support for static website hosting. Let's configure it:

mc alias set local https://minio.local
mc anonymous set download local/static-site
mc website set local/static-site --index index.html --error index.html

With this configuration, MinIO will serve index.html for directory requests and as the error page.

Troubleshooting Common Issues

Issue: ModuleNotFoundError: No module named 'pulumi_docker'

If you encounter this error, follow these steps:

Ensure you are inside the infra directory
Activate the virtual environment:

   source venv/bin/activate

Reinstall the Pulumi Docker provider:

   pip install pulumi_docker

Verify the installation:

   pip list | grep pulumi_docker

Try running pulumi up again

Issue: Pulumi asks for a passphrase

Pulumi encrypts secrets. If prompted for a passphrase, you can set an environment variable to avoid re-entering it:

export PULUMI_CONFIG_PASSPHRASE="your-passphrase"

Cleaning Up Resources

When you're done with your HomeLab MiniCloud, you can clean up all resources using:

pulumi destroy

This will remove all resources created by Pulumi.

Conclusion

Congratulations! You've successfully set up a HomeLab MiniCloud environment with MinIO for object storage, protected it with SSL, and configured it to host a static website. This setup provides a great foundation for learning cloud concepts, testing deployments, or simply running your own personal cloud services.

Some next steps you might consider:

Add more services to your MiniCloud (like a database or a web application)
Implement proper authentication for production use
Set up automated backups of your MinIO data
Explore other Pulumi providers to expand your infrastructure

Building a home lab is an excellent way to gain hands-on experience with cloud technologies without incurring significant costs. As you grow more comfortable with these tools, you'll find it easier to design, deploy, and manage cloud infrastructure in professional environments as well.

Happy cloud building!

A Beginner’s Guide to Distributed Tracing with OpenTelemetry and Jaeger

shashankpai — Thu, 27 Mar 2025 08:08:04 +0000

Distributed systems are complex, with requests flowing through multiple services. Debugging issues in such systems can be challenging. Distributed tracing, using tools like OpenTelemetry and Jaeger, provides visibility into these interactions, helping us identify performance bottlenecks and troubleshoot effectively.

In this blog, we'll explore how to instrument a simple Python application with OpenTelemetry and send trace data to Jaeger for visualization.

Why Distributed Tracing?

Distributed tracing captures the lifecycle of a request across multiple services. Key benefits include:

End-to-End Visibility: Understand how a request flows through your system.
Performance Insights: Identify slow components and bottlenecks.
Error Tracking: Pinpoint where failures occur in your application stack.

Overview of the Setup

We'll create a simple Python application and configure OpenTelemetry to send trace data to Jaeger. Here's what we’ll cover:

Setting Up Jaeger
Instrumenting a Python App with OpenTelemetry
Visualizing Traces in Jaeger
Code Walkthrough
Demo: Step-by-Step Execution

1. Setting Up Jaeger

Jaeger is an open-source tool for tracing and monitoring distributed systems. We’ll use the all-in-one Docker image for simplicity.

Command to Run Jaeger

docker run -d --name jaeger \
  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
  -p 5775:5775 -p 6831:6831/udp -p 6832:6832/udp \
  -p 5778:5778 -p 16686:16686 -p 14250:14250 -p 14268:14268 \
  jaegertracing/all-in-one:1.31

16686: Jaeger UI (View traces).
14250: GRPC endpoint (Receive traces from OpenTelemetry).
9411: Zipkin-compatible endpoint.

Why Jaeger? It provides a comprehensive solution for distributed tracing with a simple UI for visualizing trace data.

2. Instrumenting a Python App with OpenTelemetry

OpenTelemetry is a set of APIs and tools for collecting telemetry data like traces and metrics. We'll use it to instrument a Python app.

Install OpenTelemetry Libraries

pip install flask opentelemetry-api opentelemetry-sdk \
    opentelemetry-exporter-jaeger opentelemetry-instrumentation-flask

Code: A Simple Flask Application

File: app.py

from flask import Flask, jsonify
from opentelemetry import trace
from opentelemetry.exporter.jaeger.proto.grpc import JaegerExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Initialize Flask app
app = Flask(__name__)

# Configure OpenTelemetry
trace.set_tracer_provider(
    TracerProvider(resource=Resource.create({"service.name": "python_app"}))
)
tracer = trace.get_tracer_provider().get_tracer(__name__)

# Export traces to Jaeger
jaeger_exporter = JaegerExporter(
    collector_endpoint="http://localhost:14250", insecure=True
)
span_processor = BatchSpanProcessor(jaeger_exporter)
trace.get_tracer_provider().add_span_processor(span_processor)

@app.route('/')
def home():
    with tracer.start_as_current_span("home-span"):
        return jsonify(message="Welcome to the home page!")

@app.route('/process')
def process():
    with tracer.start_as_current_span("process-span"):
        return jsonify(message="Processing done!")

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Trace Provider: Defines the application as python_app for Jaeger.
Jaeger Exporter: Sends traces to Jaeger’s GRPC endpoint.
Spans: Represent units of work, like handling a request.

3. Visualizing Traces in Jaeger

After instrumenting the application:

Access the Jaeger UI: http://localhost:16686.
Select the service python_app from the dropdown.
Click Find Traces to view recent traces.

4. Dockerizing the Application

To simplify deployment, we’ll use Docker Compose to run both Jaeger and the Flask app.

Docker Compose File

File: docker-compose.yml

version: '3.7'

services:
  jaeger:
    image: jaegertracing/all-in-one:1.31
    ports:
      - "5775:5775/udp"
      - "6831:6831/udp"
      - "6832:6832/udp"
      - "5778:5778"
      - "16686:16686"
      - "14250:14250"
      - "14268:14268"
      - "9411:9411"

  python_app:
    build:
      context: .
    ports:
      - "5000:5000"
    environment:
      - OTEL_EXPORTER_JAEGER_ENDPOINT=http://jaeger:14250

Dockerfile for Flask App

File: Dockerfile

FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .

CMD ["python", "app.py"]

File: requirements.txt

flask
opentelemetry-api
opentelemetry-sdk
opentelemetry-exporter-jaeger
opentelemetry-instrumentation-flask

5. Running the Demo

Build and Start the Stack:

   docker-compose up --build

Access the App:
- Visit http://localhost:5000.
- Try / and /process routes to generate traces.
View Traces:
- Open http://localhost:16686.
- Select python_app and analyze the traces.

Code Explanation

Spans: Represent individual operations in your app (e.g., home-span, process-span).
Trace Attributes: Metadata added to spans for better debugging.
Jaeger Exporter: Sends trace data to the Jaeger instance.

Conclusion

In this tutorial, we learned to:

Set up Jaeger for distributed tracing.
Instrument a Python app using OpenTelemetry.
Use Docker Compose to simplify the setup.

With this foundation, you can extend the project by adding custom spans, simulating failures, or tracing across microservices. Distributed tracing is a vital skill for debugging and optimizing modern applications.

Building a Simple CDN with NGINX and Docker: A Step-by-Step Guide

shashankpai — Thu, 27 Mar 2025 08:02:33 +0000

Introduction: Why Use a CDN?

A Content Delivery Network (CDN) is a distributed network of servers that delivers content to users based on their geographic location. CDNs improve website performance, reduce latency, and ensure better availability by caching content closer to end users. In this blog, we'll create a simple CDN setup using NGINX and Docker, simulating an edge and origin server to demonstrate how content caching works.

Prerequisites:

Basic knowledge of Docker and NGINX.
Docker and Docker Compose installed on your local machine.
Administrative access to edit the /etc/hosts file (or C:\Windows\System32\drivers\etc\hosts on Windows).

Step 1: Setup Overview

We'll simulate the following:

Origin Server: Stores the original content.
Edge Server: Caches content from the origin and serves it to clients.
DNS Simulation: Redirect traffic to the edge server using a custom domain (cdn.local) by configuring your local hosts file.

Both servers will run as Docker containers, with the edge server fetching content from the origin server.

Step 2: Creating the Origin Server

The origin server hosts the original content that the CDN will cache.

1. Create a Dockerfile for the origin server:

# Dockerfile.origin
FROM nginx:latest
COPY ./origin/nginx.conf /etc/nginx/conf.d/default.conf
COPY ./origin-content /usr/share/nginx/html

2. Prepare the NGINX configuration for the origin server:

# origin/nginx.conf
server {
    listen 80;
    server_name localhost;

    location / {
        root /usr/share/nginx/html;
        index index.html;
    }
}

Explanation:

The root directive specifies the directory containing the website files.
The index directive sets the default file (index.html) to serve.

3. Prepare the content directory:

Create a folder named origin-content and add a simple HTML file (index.html):

<!-- origin-content/index.html -->
<h1>Welcome to the NGINX CDN Origin Server!</h1>

4. Build and run the origin container:

docker build -t origin-server -f Dockerfile.origin .
docker run -d --name origin -p 8080:80 origin-server

You can access the origin content at:

http://localhost:8080

Step 3: Creating the Edge Server

The edge server will cache content from the origin server and serve it to clients.

1. Create an NGINX configuration for the edge server:

# edge/nginx.conf
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m max_size=1g inactive=60m use_temp_path=off;

server {
    listen 80;
    server_name cdn.local;

    location / {
        proxy_pass http://origin-server;
        proxy_cache my_cache;
        proxy_cache_valid 200 60m;
        add_header X-Proxy-Cache $upstream_cache_status;
    }
}

Explanation of Key Directives:

proxy_cache_path: Defines the cache storage location and size.
proxy_pass: Forwards requests to the origin server.
proxy_cache: Activates caching for the defined my_cache zone.
X-Proxy-Cache: Adds a custom header to indicate cache status (HIT, MISS, or EXPIRED).

2. Create a Dockerfile for the edge server:

# Dockerfile.edge
FROM nginx:latest
COPY ./edge/nginx.conf /etc/nginx/conf.d/default.conf

3. Build and run the edge container:

docker build -t edge-server -f Dockerfile.edge .
docker run -d --name edge -p 8081:80 --link origin origin-server edge-server

You can access the edge server at:

http://localhost:8081

Step 4: Configuring DNS Simulation with `/etc/hosts`

To simulate DNS resolution for our custom domain (cdn.local), we’ll modify the /etc/hosts file to map cdn.local to the edge server.

1. Edit the /etc/hosts file:

Open the file with administrative privileges:

On Linux or macOS:

  sudo nano /etc/hosts

On Windows: Open C:\Windows\System32\drivers\etc\hosts with a text editor as an administrator.

2. Add the following entry:

127.0.0.1   cdn.local

Explanation:

This maps cdn.local to 127.0.0.1, which is your local machine. When you access http://cdn.local, the request will be directed to the edge server running on localhost.

Step 5: Testing the CDN Setup

Initial Request (Cache MISS): When you first access http://cdn.local:8081, the edge server fetches the content from the origin. You should see the response header:

   X-Proxy-Cache: MISS

Subsequent Requests (Cache HIT): Reload the page, and this time the response will be served from the edge server’s cache:

   X-Proxy-Cache: HIT

Conclusion

In this exercise, we created a basic CDN using NGINX and Docker. We set up an origin server hosting the original content and an edge server that caches and serves content. By configuring DNS simulation with the /etc/hosts file, we routed traffic to the edge server using a custom domain (cdn.local). This setup demonstrates how CDNs improve performance by caching content closer to the end user.

Future Enhancements:

Implement SSL/TLS: Secure your CDN with HTTPS.
Load Balancing: Add multiple edge servers for redundancy.
Monitoring and Logging: Use tools like Prometheus and Grafana to monitor CDN performance.

Automatic Image Update to Git using Flux and GitHub Actions

shashankpai — Sun, 14 Apr 2024 18:30:00 +0000

Have you ever had to manually update your container images, only to forget to do it or make a mistake? Automatic image updates can help you avoid these problems and ensure that your applications are always running the latest and most secure images. Manual image updates can be time-consuming and error-prone, especially if you have a large number of containerized applications.

In this blog post, we'll explore the advantages of automatic image updates and explain how to implement them in your environment using GitHub Actions. Moreover, we'll cover the process of temporarily pausing these updates to ensure application stability during incidents or any unforeseen issues.

So, let’s deep dive into automatic image updates to Git using Flux and GitHub Actions!

GitOps: A modern approach to software delivery

In the world of application management, GitOps fills a crucial need by simplifying and enhancing how we handle applications. But why do we need it? Managing applications can be complex and prone to errors. To address this, GitOps establishes Git as the central hub for handling configuration and infrastructure changes, allowing for streamlined, efficient, and reliable management. It speeds up deployment, ensures consistent and error-free setups, and reduces operational costs.

Moreover, it's important to note that GitOps isn't just about managing applications; it also intersects with the concept of auto-image updates. These updates involve automatically refreshing the underlying components of your application by regularly updating to the latest images pushed to the container registry, ensuring your application stays in sync with the most up-to-date software and dependencies.

GitOps streamlines software deployment by automating build, test, and deployment processes triggered by Git commits. This enhances reliability through a review and approval system, reducing errors and downtime. Collaboration is promoted through Git's features for change tracking and code reviews, with seamless integration of image updates.

Current pain points of dealing with image updates without GitOps

Here are several challenges you may face when dealing with image updates in the absence of GitOps:

Manual updates to manifest: When a new image is released, the manifest must be manually updated to reference the new image tag. This can be error-prone and time-consuming, especially for large and complex applications.
Delayed deployments: If the manifest is not updated promptly, the deployment of the new image will be delayed. This can impact the time to market for new features or bug fixes.
Inconsistent deployments: If the manifest is not updated consistently across all environments, it can lead to inconsistent deployments. This can make it difficult to troubleshoot problems and ensure that applications are running as expected in all environments.
Manual build triggers: When a new image is released, a new build must be manually triggered. This can be inefficient and time-consuming, especially for large and complex applications.

GitOps using auto-image updates can help to solve all of these pain points by providing a declarative and automated approach to application deployment and management. With GitOps, the manifest is automatically updated when a new image is released, and a new build is automatically triggered. This ensures that deployments are consistent and timely across all environments.

Benefits of automatic image updates using GitOps

Here are several advantages of using GitOps for image updates:

Saves time and effort: GitOps streamlines image updates by automating the process of updating images, tags, and manifests. This eliminates the need for manual intervention, reducing the risk of errors and ensuring a quick and reliable deployment of the latest application version across all environments.
Promotes consistency across environments: Ensuring consistent application versions across various environments become seamless with automated image updates. This practice simplifies application management, reducing the risk of inconsistencies between environments. Consequently, maintenance and troubleshooting become more straightforward.
Fosters collaboration and transparency: Encouraging teamwork and transparency, GitOps facilitates automatic image updates for development teams. Leveraging Git commits for all changes simplifies change tracking, code reviews, and maintains visibility in the software delivery process. This promotes efficient collaboration among developers and cultivates a culture of continuous improvement.
Improves application delivery: Leveraging GitOps for automatic image updates speeds up application delivery and boosts reliability. Streamlining infrastructure and application management enables developers to focus more efficiently on delivering customer value.

Approaches to automating image updates

In this section, we will explore two primary methods for automating image updates to keep your applications up-to-date and secure.

Scripted approach
Automated approach using GitOps tools

Scripted approach

A script can be configured to run on a schedule using a cron job or when certain events occur, such as when a new commit is pushed to the Git repository.

Here are a few pros and cons of the scripted approach:

Pros of scripted approach:

Flexible and customizable method
Can be tailored to meet specific needs
No need to rely on third-party tools

Cons of scripted approach:

Complex and time-consuming to implement
Requires expertise in scripting
Maintaining and troubleshooting the scripts can be challenging

Using GitOps tools for automating image updates

In the previous section, we saw that scripting can be complex and time-consuming. That's where dedicated tools come in handy. In this section, we will explore how GitOps tools make automating image updates easier. We will specifically look at the two most popular tools, Argo CD and Flux, to understand how they help with automatic image updates.

Argo CD image updater

(Image update process using Argo CD Image Updater)

The Argo CD image updater is a tool that can help you update your Kubernetes workloads to the latest versions of their container images. It does this by setting appropriate application parameters for your Argo CD applications.

To use the image updater, you annotate your Argo CD application resources with a list of images to be considered for update, along with a version constraint to restrict the maximum allowed new version for each image. The image updater will then periodically check for new versions of the images and update them if necessary.

The image updater can be used to update images in a variety of ways, including:

Automatic updates: The image updater can be configured to automatically update images to their latest allowed version on a schedule.
Manual updates: The image updater can also be triggered manually to update images.
Event-driven updates: The image updater can also be configured to be triggered by events, such as when a new image tag is pushed to a registry.

Image automation and reflector controllers in Flux

(Image update process using Image Automation and Image Reflector Controller)

Image automation in Flux can be used to automate the following tasks:

Updating image tags in Git: Image automation can automatically update the image tags in your Git repository to the latest stable versions of your images. This can save you time and reduce the risk of errors.
Creating and managing image promotion pipelines: Image automation can be used to create and manage image promotion pipelines. This allows you to automate the process of rolling out new image versions to production in a safe and controlled manner.
Implementing image policies: Image automation can be used to implement image policies, such as requiring all images to be tagged with a specific version number or only using images from a specific registry.

Image automation controllers

In Flux CD, the image automation controllers, which include the Image Reflector Controller and Image Automation Controller, are responsible for maintaining the synchronization between the image metadata in Kubernetes and the latest image metadata from the registry. The image reflector controller achieves this by regularly scanning the registry for changes to image metadata. When a change is detected, they update the Kubernetes resources that reference the affected images accordingly.

Moreover, the image automation controllers are capable of triggering image updates based on changes to the image metadata. For instance, you can configure automation to automatically update the image tags in your Kubernetes deployments to the latest stable versions of your images.

For more details about the implementation and design of the image automation controllers in Flux CD, refer to the official documentation on image reflector and automation Controllers.

Pros and cons of using GitOps approach for automating image updates

Let us explore the pros and cons of employing GitOps for automating image updates, gaining insights into its efficiency and potential challenges.

Pros of GitOps approach:

Reducing human error: Both Argo CD and Flux CD offerings for image update aim to reduce the risk of human error by automating the process of updating container images. This helps in maintaining consistency and accuracy.
Workflow efficiency: Automation in image updates improves workflow efficiency by saving time and allowing teams to focus on more critical tasks.
Automatic updates: GitOps tools, including both Argo CD and Flux CD, provide a mechanism for automatic updates, which enhances the security, reliability, and efficiency of Kubernetes workloads.
Flexibility: Both tools offer flexibility in how image updates can be handled, supporting automatic updates, manual updates, and event-driven updates, allowing users to choose the best approach for their specific requirements.

Cons of GitOps approach:

Complex setup: Setting up and configuring GitOps tools for image automation, especially for users unfamiliar with the underlying technologies like Kubernetes or Flux, can be complex.
Troubleshooting complexity: The complexity of interactions with various components (e.g., Kubernetes, registries, Git repositories) can make troubleshooting challenging if issues arise during the image update process.
Dependency on GitOps tool: Both Argo CD and Flux CD are limited to updating container images for applications managed by their respective tools. This dependency might restrict users who prefer or have other GitOps tools in their workflows.
Image pull secrets location: GitOps tools often require image pull secrets to be present in the same cluster where the tool is running, which might pose challenges for scenarios where secrets need to be fetched from other clusters.
Limited PR support: Neither Argo CD nor Flux CD, as described, fully supports the creation of pull requests (PRs). External CI tools may be required for more comprehensive integration with version control systems.
Production readiness: When considering GitOps tools for image automation in production, it's crucial to note that Flux CD is deemed "production-ready," while Argo CD's image updater lacks the same designation. This underscores the necessity of carefully evaluating the maturity and stability of GitOps tools to align with specific production environment requirements.

How does image automation and image reflector controllers in Flux work

Let's understand how Flux employs two controllers, i.e. image automation controller and image reflector controller, to achieve image update automation.

These controllers, together, collaborate to update a Git repository whenever new container image tags become available.

Image reflector controller

It is composed of two custom resources: Image Repository and Image Policy. Let us understand how they operate together.

Image repository: This component scans the container image repository and retrieves image metadata, including tags and versions. You can create an Image Repository resource and configure the scanning interval.
Image policy: This custom resource informs Flux which semantic versioning range to apply when filtering image tags.

Image automation controller

The image automation controller includes a custom resource called Image Update Automation.

Image Update resource clones the Git repository, updates YAML files based on the latest images from the image reflector controller, and commits changes to the specified Git repository.
After cloning, the image automation controller identifies and updates the deployment YAML manifest, using comments to mark fields for updating. The automation process checks the image policy in the comment and updates the field value accordingly.

Once updated, the image update automation controller commits and pushes changes to the specified branch. The source controller pulls the updated manifest, and the Kustomization controller applies the changes. This is how the Flux image automation controller streamlines the process of automating image version updates.

Automating container image updates with Flux image automation and GitHub Actions

We will explore a live demonstration of a Flux GitOps configuration that seamlessly manages a workflow from staging to production, showcasing the automatic deployment process when making changes to source code. Additionally, we'll highlight its integration with GitHub Actions to enhance the efficiency of the entire deployment pipeline.

In staging, developers make changes to application code, triggering a new build. The new image tag is then propagated to the Kubernetes manifest and finally deployed to the staging environment without requiring any approval in between.

In the case of production, a new PR is raised with an updated Kubernetes manifest, requiring approval for merge in the config repository. Only after approval is given does it get deployed to the production environment.

Prerequisites

Ensure you have following things setup:

Flux CLI - which can be downloaded from the official docs
Kubernetes cluster - for this demo, we will use minikube.
GitHub account - which has GitHub Actions enabled.

Environment setup details

Hello Env Flask application: A simple application displaying the environment name and release number.
Two environments: Staging and production, housed in separate namespaces.
Continuous deployment: Source code changes trigger automated build and deployment to the staging environment.
Release tagging: Tagging a release initiates automatic build processes and creates a pull request for production deployment.

Git repositories

Let's take a look at two Git repositories we will be using.

Application repository - https://github.com/infracloudio/flux-helloenv-app - code and Kustomization.
1. Kustomization for environment separation: Utilizing Kustomize to segregate staging and production environments.
2. Structured repository: The repository combines source code and Kubernetes manifests for a unified demo setup. There are many possible ways to structure your git repositories.
3. CI and automation: Leveraging GitHub Actions for continuous integration and automation.
Management Repository- https://github.com/infracloudio/flux-gitops-helloenv - GitOps manifests.

Flux bootstrap configuration

Beginning with an empty cluster, our initial task is to bootstrap Flux itself. Flux serves as the foundation upon which we'll bootstrap all other components.

In the following sections, we'll take a detailed look at each file within the apps directory, one by one.

├── apps
│   ├── git-repo.yaml
│   ├── image-auto-prod.yaml
│   ├── image-auto-staging.yaml
│   ├── image-policy-prod.yaml
│   ├── image-policy-staging.yaml
│   ├── image-repo.yaml
│   ├── kustomization-prod.yaml
│   └── kustomization-staging.yaml

git-repo.yaml

This instructs Flux on how to interact with the Git repository where your application's source code resides. It contains the following configuration:

  ref:
    branch: main
  secretRef:
    name: ssh-credentials
  url: ssh://git@github.com/infracloudio/flux-helloenv-app

ref: Specifies the Git branch to watch for changes, in this case, main.

secretRef: Refers to a Kubernetes Secret named ssh-credentials, which likely contains SSH keys for secure Git access.

The ssh-credentials is a secret that needs to be created.

url: Indicates the URL of the Git repository, Change this to url: ssh://git@github.com/<your_github_username>/flux-helloenv-app once you fork it.

image-repo.yaml

This file is responsible for scanning the Docker image registry and fetching image tags based on the defined policy. Here's the configuration:

image: docker.io/shapai/helloenv
interval: 1m0s

image: Specifies the Docker image repository (for e.g. docker.io/shapai/helloenv) to scan for image tags. Change this to have your relevant Docker image repository, where the CI job will build and push the image. This same registry needs to be updated in your forked CI file, where the CI build will push images.

interval: Sets the interval at which Flux will scan the image repository (every 1 minute in this case) and fetch image tags according to the defined policy.

Next, we will go through the image policy files image-policy-staging.yaml and image-policy-prod.yaml

image-policy-staging.yaml

This file defines the image tagging policy for the staging environment. Here's the configuration:

  filterTags:
    extract: $ts
    pattern: ^main-[a-f0-9]+-(?P<ts>[0-9]+)
  imageRepositoryRef:
    name: helloenv
  policy:
    numerical:
      order: asc

filterTags: This section specifies how to filter image tags. It extracts the timestamp ($ts) from tags that match the specified pattern. Tags are filtered in ascending order based on this timestamp, ensuring that Flux fetches the latest built image for the staging environment.

image-policy-prod.yaml

This file defines the image tagging policy for the production environment:

  imageRepositoryRef:
    name: helloenv
  policy:
    semver:
      range: '>=1.0.0'

imageRepositoryRef: Refers to the image repository named helloenv.

policy: Defines a Semantic Versioning (SemVer) policy that specifies a range for acceptable image tags (in this case, any version greater than or equal to 1.0.0).

These image policies are crucial in ensuring that Flux deploys the correct images to the respective environments (staging and production) based on the defined criteria.

kustomization-prod.yaml and kustomization-staging.yaml

These YAML files define Kustomization resources for managing Kubernetes resources in both staging and production environments within the flux-system namespace. They are configured to synchronize with a specified Git repository, allowing for automated deployment and management of Kubernetes resources.

Now, let's go through the ImageUpdateAutomation files.

helloenv-staging.yaml

This YAML file configures an ImageUpdateAutomation object to update the image tags in the ./demo/kustomize/staging directory of the flux-helloenv-app Git repository. It will scan the repo every 1 minute (specified by the interval field).

The git section specifies the branch to checkout and the commit message template. The sourceRef section specifies the Git repository containing the Kubernetes manifests to update.

The update section specifies the path to the Kubernetes manifests to update and the strategy to use.

When this ImageUpdateAutomation object is deployed, Flux will periodically check for new image updates. If it finds any new updates, it will update the image tags in the Kubernetes manifests and commit the changes to the Git repository.

spec:
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: fluxbot@users.noreply.github.com
        name: fluxbot
      messageTemplate: '{{range .Updated.Images}}{{println .}}{{end}}'
  interval: 1m0s
  sourceRef:
    kind: GitRepository
    name: flux-helloenv-app
  update:
    path: ./demo/kustomize/staging
    strategy: Setters

helloenv-prod.yaml

This YAML file is very similar to the previous one but has an additional push configuration. This means that after updating the image tags in the Git repository, Flux will commit and push the changes to the flux-image-update branch. This is done specially for prod setup since we are not going to push directly to the main branch for prod manifests.

This is useful in our case, as this will help us create a PR to the main branch from the flux-image-update branch, which will then go through manual approval. The process of creating PR is handled by the CI section in GitHub Actions.

apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImageUpdateAutomation
metadata:
  name: helloenv-prod
  namespace: flux-system
spec:
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: fluxbot@users.noreply.github.com
        name: fluxbot
      messageTemplate: '{{range .Updated.Images}}{{println .}}{{end}}'
    push:
      branch: flux-image-update
  interval: 1m0s
  sourceRef:
    kind: GitRepository
    name: flux-helloenv-app
  update:
    path: ./demo/kustomize/prod
    strategy: Setters

Setting up Flux

Now that we've seen all the configurations, we can proceed to the bootstrap command.

Note the --owner and --repository switches here: we are explicitly looking for the ${GITHUB_USER}/flux-gitops-helloenv repo. Make sure you fork both repos under your user and follow on.

flux bootstrap github \ 
  --components-extra=image-reflector-controller,image-automation-controller \
  --owner=$GITHUB_USER \
  --repository=flux-gitops-helloenv \
  --path=./clusters/my-cluster/ \
  --branch=main \
  --read-write-key \
  --personal --private=false

Understanding `flux bootstrap` command

Let's understand what the above command does.

To enable the Flux image automation feature, the extra components can be specified with the --components-extra flag. We are enabling the image reflector and automation controllers.

--path=./clusters/my-cluster/ specifies the path within the repository where the configuration files will be stored.

my-cluster folder in the repository contains two files infrastructure.yaml and apps.yaml.

infrastructure.yaml defines a Kustomization resource called ingress-nginx. This Kustomization lives in the flux-system namespace, doesn't depend on anything, and has kustomize files at infrastructure/ingress-nginx.
apps.yaml, which defines the flux-helloenv-app application itself. Within this file, you'll find references to the apps directory. This directory is a central location containing Kustomizations and configuration settings for setting up the flux-helloenv-app application in both the prod and staging environments. Additionally, it includes configurations for image automation for helloenv app in staging and prod env.

The process of bootstrapping everything may take some time. To monitor the progress and ensure everything is proceeding as expected, we can utilize the --watch switch.

flux get kustomizations --watch

Verify that all Flux pods are in running state by running get pods.

$ kubectl -n flux-system get pods

NAME                                           READY   STATUS    RESTARTS   AGE
image-automation-controller-6c4fb698d4-zrp78   1/1     Running   0          29s
image-reflector-controller-5dfa39212d-hnnvj    1/1     Running   0          29s
kustomize-controller-424f5ab2a2-u2hwb          1/1     Running   0          29s
source-controller-2wc41z892-axkr1              1/1     Running   0          29s

Since our flux-helloenv-app repository is public, application will get deployed as part of the bootstrap step. You can check both staging and prod environments with following commands.

kubectl rollout status -n helloenv-staging deployments
watch kubectl get pods -n helloenv-staging

kubectl rollout status -n helloenv-prod deployments
watch kubectl get pods -n helloenv-prod

Setup Git authentication for Flux

After bootstrapping Flux, we will grant it write access to our GitHub repositories. This will allow Flux to update image tags in manifests, create pull requests etc.

The flux create secret git command creates an SSH key pair for the specified host and puts it into a named Kubernetes secret in Flux's management namespace (by default flux-system). The command also outputs the public key, which should be added to the forked repo's "Deploy keys" in GitHub.

GITHUB_USER=<your_github_username>
flux create secret git ssh-credentials \
  --url=ssh://git@github.com/${GITHUB_USER}/flux-helloenv-app

If you need to retrieve the public key later, you can extract it from the secret as follows:

kubectl get secret ssh-credentials -n flux-system -ojson \
  | jq -r '.data."identity.pub"' | base64 -d

Use the public key as a Deploy key in your fork of the flux-helloenv-app repo. Browse to the following URL, replacing <your_github_username> with your GitHub username: https://github.com/<your_github_username>/flux-helloenv-app/settings/keys.

The page will appear as follows.

Click "Add deploy key" and paste the key data (starts with ssh-<alg>... ) into the contents. The name is arbitrary, we use flux-helloenv-app-secret here.

Accessing the application

The default image version can be checked by accessing our application using curl command. If you are using minikube, make sure you enable the ingress addon so the ingress functionality works.

minikube addons enable ingress

If you are running on the local cluster, you will have to add the minikube IP to your /etc/hosts file. The command minikube ip will give the IP of your cluster.

For example, the entry in /etc/hosts file will look like this:

192.168.49.2 helloenv.prod.com helloenv.stage.com

You can check the ingress that is available for stage and prod.

$ kubectl get ingress --all-namespaces

NAMESPACE          NAME       CLASS    HOSTS                ADDRESS   PORTS   AGE
helloenv-prod      helloenv   <none>   helloenv.prod.com              80      9m22s
helloenv-staging   helloenv   <none>   helloenv.stage.com             80      9m21s

Now, you can access the application with following commands.

curl helloenv.prod.com

curl helloenv.stage.com

The result of these curl commands will show the current default image versions that the deployment is using.

Releasing to stage

Now, let's make some changes to the app and release it to the stage. We can make a minor change to the app.py file in the application code repository flux-helloenv-app, by changing the message and pushing it to the main branch, of course, in real case it will be pushed to main through a PR.

This change triggers the CI job configured on the application code repository. This job contains the logic to check the tag and then create the image tag based on it.

run: |
  if [[ ${{ github.event.ref }} =~ ^refs/tags/[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
    echo "IMAGE_ID=${{ github.ref_name }}" >> "$GITHUB_OUTPUT"
  else
    ts=$(date +%s)
    branch=${GITHUB_REF##*/}
    echo "IMAGE_ID=${branch}-${GITHUB_SHA::8}-${ts}" >> "$GITHUB_OUTPUT"
  fi

From this snippet, it can be seen that it checks for the condition for prod and stage, as well as how we set the tagging of images.

So, since we are doing this for stage, it will tag the image as ${branch}-${GITHUB_SHA::8}-${ts}.

A useful tag format is <branch>-<sha1>-<timestamp>.

Including the branch information with an image makes it easier to trace the source code's branch and commit associated with that image. Additionally, having the branch information and unix time allows you to filter for images originating from a specific branch when needed.

Since this CI stage for Staging will push the image to the image repository, the image update and image policy will kick in and replace the image with the latest built image based on timestamp.

Check status of stage environment

We can check the status of image automation and image policy for staging.

$ flux get image all

NAME                        LAST SCAN               SUSPENDED   READY   MESSAGE                       
imagerepository/helloenv    2023-10-17T12:19:26Z    False       True    successful scan: found 9 tags   

NAME                            LATEST IMAGE                                        READY   MESSAGE                                                                                
imagepolicy/helloenv-prod       docker.io/shapai/helloenv:1.0.2                     True    Latest image tag for 'docker.io/shapai/helloenv' resolved to 1.0.2                    
imagepolicy/helloenv-staging    docker.io/shapai/helloenv:main-b470560d-1696668188  True    Latest image tag for 'docker.io/shapai/helloenv' resolved to Build-b470560d-1696668188

NAME                                    LAST RUN                SUSPENDED   READY   MESSAGE                                                      
imageupdateautomation/helloenv-prod     2023-10-17T12:18:41Z    False       True    no updates made                                                 
imageupdateautomation/helloenv-staging  2023-10-17T12:18:39Z    False       True    no updates made; last commit f798e64 at 2023-10-07T09:25:32Z

Let's curl the stage domain and see the output:

$ curl helloenv.stage.com

This is staging environment with (Version: main-b470560d-1696668188)

The output will give you the latest version, which you can verify with commit id and timestamp with which the image was created.

Releasing to production

Now, assuming we have tested the release in stage and got go-ahead, we are ready to release in production by tagging the release.

We will be releasing it in prod by tagging the release.

git tag -a 1.0.2 -m "prod modified"
git push --tags

This will, in turn, trigger the CI and create an image tag with that Git tag that we pushed.
Once this is triggered and pushed, the image automation in Flux will commit and push the changes to the flux-image-update branch.

This will then create a PR using workflow in flux-helloenv-app repo. This PR needs a manual review and approval since it is a prod environment. Once this gets approved, it changes the tag with the latest tag in prod kustomization.

curl to the prod DNS should give you the latest tag as the version.

$ curl helloenv.prod.com

This is production environment (Version: 1.0.2)

Incident Management

During an incident, you may want to halt Flux from updating images in your Git repository. You can accomplish this by suspending image automation either in-cluster or by editing the ImageUpdateAutomation manifest in Git.

In-cluster suspension

You can suspend image automation directly in a cluster using the following command:

flux suspend image update helloenv-prod

Alternatively, you can suspend image automation by editing the ImageUpdateAutomation manifest in Git. Here's an example of the manifest:

kind: ImageUpdateAutomation
metadata:
  name: helloenv-prod
  namespace: flux-system
spec:
  suspend: true

Resuming automation

Once the incident is resolved, you can resume the automation using the following command:

flux resume image update helloenv-prod

Pausing automation for a specific image

If you want to pause automation for a particular image only, you can suspend and resume image scanning for that specific image. For example:

flux suspend image repository helloenv

Reverting image updates

Assuming you've configured Flux to update an application to its latest stable version and an incident occurs, you can instruct Flux to revert to a previous image version.

Reverting via command

For instance, to revert from version 1.0.1 to 1.0.0, you can use the following command:

flux create image policy helloenv-prod --image-ref=helloenv --select-semver=1.0.0

Reverting via Git manifest

You can also make this change by editing the ImagePolicy manifest in Git. Here's an example of the manifest:

kind: ImagePolicy
metadata:
  name: helloenv-prod
  namespace: flux-system
spec:
  policy:
    semver:
      range: 1.0.0

Updating the image policy

When a new version, e.g., 1.0.2, becomes available, you can update the policy again to consider only versions greater than 1.0.1. This can be achieved using the following command:

flux create image policy helloenv-prod --image-ref=helloenv --select-semver=">1.0.1"

This change will prompt Flux to update the podinfo deployment manifest in Git and roll out the specified image version in-cluster.

Conclusion

Flux's image automation and GitOps are powerful solutions for managing container image updates. By combining image automation and image reflector controllers, organizations can automate image version updates in their Git repositories. This not only simplifies the process but also ensures consistency and reliability in application deployment.

The practical example illustrated how GitOps with Flux can streamline the workflow from staging to production, providing a structured approach to managing deployments as code changes occur. This approach enhances efficiency and reliability in the development pipeline, making it a valuable asset in modern DevOps practices.

Thank you for taking the time to read our post. We hope you found it both informative and engaging. We highly value your feedback and would love to hear your thoughts on this topic. Let's kickstart a meaningful conversation on LinkedIn to exchange ideas and insights.

If you're seeking assistance in crafting a robust DevOps strategy or considering outsourcing your DevOps operations to seasoned experts, we invite you to discover why numerous startups and enterprises regard us as one of the top-tier DevOps consulting and services companies.

References

Unlocking Linux Host Performance Insights with Prometheus Monitoring (Part-1)

shashankpai — Fri, 14 Jul 2023 07:19:26 +0000

Introduction
Importance of monitoring Linux host metrics
How Prometheus can help?
What is Node Exporter and its role ?
Setting up Prometheus server to monitor Linux hosts
Install and Configure Node Exporter on the Server

Introduction:

Welcome to part 1 of our 2-part blog series on Linux host monitoring with Prometheus. This series is designed for DevOps professionals, monitoring enthusiasts, and Prometheus Certified Associate aspirants.

In this series, we'll explore the importance of monitoring Linux host metrics and how Prometheus can help. We'll cover topics such as exporting metrics through node exporter and how prometheus scrapes metrics.

Stay tuned for part 2, where we'll dive into practical implementation, configuration, and creating dashboards with grafana for monitoring Linux hosts with Prometheus.

Get ready to elevate your DevOps and monitoring skills as we unlock the power of Linux host monitoring with Prometheus. Let's begin this journey together.

Importance of monitoring Linux host metrics:

Monitoring Linux host metrics is crucial for optimizing performance, planning capacity, troubleshooting issues, ensuring security, and proactively maintaining systems. By tracking metrics like CPU, memory, disk, and network usage, you can identify problems, make data-driven decisions, and ensure the smooth operation of your Linux environment.

How Prometheus can help?

Prometheus is a powerful monitoring tool that simplifies the process of monitoring Linux host metrics. It collects data from various sources using lightweight exporters like Node Exporter and stores it in a time-series database. With PromQL, you can query and analyze the collected metrics, enabling you to retrieve specific Linux host metrics and gain insights into system performance.

What is Node Exporter and its role ?

Prometheus Server scraping metrics from Node Exporter service running on Linux hosts

Linux doesn't provide native Prometheus metrics out of the box. However, Node Exporter comes to the rescue by bridging this gap. Node Exporter is an exporter specifically designed for Linux systems, serving as a reliable source for collecting Linux host metrics.

Node Exporter runs as a service on the Linux host, exposing a wide range of metrics related to system resources, network, disk, CPU, memory, and more. It periodically collects these metrics from the Linux kernel and other system sources, making them available for scraping by Prometheus.

By installing and configuring Node Exporter on Linux hosts, you can unlock a wealth of Linux-specific metrics that would otherwise be inaccessible to Prometheus. This includes detailed information about CPU utilization, memory usage, disk I/O, network traffic, and other crucial indicators of system performance.

Node Exporter acts as a bridge between the Linux operating system and Prometheus, ensuring that Linux host metrics can be easily monitored and analyzed using Prometheus' powerful features. It provides a standardized and reliable way to expose Linux-specific metrics, enabling seamless integration with the Prometheus ecosystem.

With Node Exporter, system administrators and DevOps teams can gain valuable insights into the performance and health of their Linux hosts, allowing them to proactively optimize resource allocation, troubleshoot issues, and ensure the smooth operation of their Linux-based infrastructure.

Setting up Prometheus server to monitor Linux hosts

Download and install Prometheus

1.Create the `prometheus` user

sudo useradd -M -r -s /bin/false prometheus

2.Create prometheus directories

sudo mkdir /etc/prometheus /var/lib/prometheus

3. Then download latest binary archive for Prometheus.

mkdir -p /tmp/prometheus && cd /tmp/prometheus

curl -s https://api.github.com/repos/prometheus/prometheus/releases/latest | grep browser_download_url | grep linux-amd64 | cut -d '"' -f 4 | wget -qi -

4. Untar the the downloaded binary

tar xvf prometheus*.tar.gz
prometheus-2.45.0.linux-amd64/
prometheus-2.45.0.linux-amd64/LICENSE
prometheus-2.45.0.linux-amd64/prometheus.yml
prometheus-2.45.0.linux-amd64/console_libraries/
prometheus-2.45.0.linux-amd64/console_libraries/prom.lib
prometheus-2.45.0.linux-amd64/console_libraries/menu.lib
prometheus-2.45.0.linux-amd64/consoles/
prometheus-2.45.0.linux-amd64/consoles/node-overview.html
prometheus-2.45.0.linux-amd64/consoles/node-cpu.html
prometheus-2.45.0.linux-amd64/consoles/index.html.example
prometheus-2.45.0.linux-amd64/consoles/node.html
prometheus-2.45.0.linux-amd64/consoles/node-disk.html
prometheus-2.45.0.linux-amd64/consoles/prometheus-overview.html
prometheus-2.45.0.linux-amd64/consoles/prometheus.html
prometheus-2.45.0.linux-amd64/promtool
prometheus-2.45.0.linux-amd64/prometheus
prometheus-2.45.0.linux-amd64/NOTICE

5. Move the binary files to /usr/local/bin/ directory and set their ownership to `prometheus` user.

sudo mv prometheus promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/{prometheus,promtool}

6. Move the files from the tmp archive to the appropriate locations, and set ownership on these files and directories to the `prometheus` user

sudo cp -r prometheus-2.45.0.linux-amd64/{consoles,console_libraries} /etc/prometheus/

sudo cp prometheus-2.45.0.linux-amd64/prometheus.yml /etc/prometheus/prometheus.yml

sudo chown -R prometheus:prometheus /etc/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus

7. Run Prometheus in the foreground to make sure everything is set up correctly so far:

prometheus --config.file=/etc/prometheus/prometheus.yml

8. Configure Prometheus as a systemd Service

Create a systemd unit file for Prometheus:

sudo vim /etc/systemd/system/prometheus.service

[Unit] Description=Prometheus Time Series Collection and Processing Server Wants=network-online.target After=network-online.target

[Service] User=prometheus Group=prometheus Type=simple ExecStart=/usr/local/bin/prometheus \ --config.file /etc/prometheus/prometheus.yml \ --storage.tsdb.path /var/lib/prometheus/ \ --web.console.templates=/etc/prometheus/consoles \ --web.console.libraries=/etc/prometheus/console_libraries 

[Install] WantedBy=multi-user.target

9. Make sure systemd picks up the changes we made:

sudo systemctl daemon-reload

10. Start the Prometheus service:

sudo systemctl start prometheus

11. Enable the Prometheus service so it will automatically start at boot:

sudo systemctl enable prometheus

12. Verify the Prometheus service is healthy:

sudo systemctl status prometheus

We should see its state is active (running).

13. Make an HTTP request to Prometheus to verify it is able to respond:

curl localhost:9090

14. In a new browser tab, access Prometheus by navigating to http://:9090 (replacing with the IP listed on the lab page). We should then see the Prometheus expression browser.

Install and Configure Node Exporter on the Server

1. Create a user and group that will be used to run Node Exporter

sudo useradd -M -r -s /bin/false node_exporter

2. Get the node exporter binary

wget https://github.com/prometheus/node_exporter/releases/download/v1.6.0/node_exporter-1.6.0.linux-amd64.tar.gz

3. Extract the Node Exporter binary:

tar -xvzf node_exporter-1.6.0.linux-amd64.tar.gz

4. Copy the Node Exporter binary to the appropriate location

sudo cp node_exporter-1.6.0.linux-amd64/node_exporter /usr/local/bin/

5. Set ownership on the Node Exporter binary


sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

6. Create a systemd unit file for Node Exporter:

sudo vim /etc/systemd/system/node_exporter.service

7.Define the Node Exporter service in the unit file

[Unit]
Description=Prometheus Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

8. Make sure systemd picks up the changes we made:

sudo systemctl daemon-reload

9. Enable the node_exporter service so it will automatically start at boot:

sudo systemctl enable node_exporter

10. Test that your Node Exporter is working by making a request to it from localhost:

curl localhost:9100/metrics

11.Configure Prometheus to Scrape Metrics from the Acme Web Server

Edit the Prometheus config file:

sudo vim /etc/prometheus/prometheus.yml

Locate the scrape_configs section and add the following beneath it (ensuring it's indented to align with the existing job_name section):

...

  - job_name: 'LimeDrop Web Server'
    static_configs:
    - targets: ['10.0.1.102:9100']

...

11. Restart Prometheus to load the new config:

sudo systemctl restart prometheus

12. Navigate to the Prometheus expression browser in your web browser using the public IP address of your Prometheus server:

<PROMETHEUS_SERVER_PUBLIC_IP>:9090

13.In the expression field (the box at the top of the page), paste in the following query to verify you are able to get some metric data from the LimeDrop web server:

node_filesystem_avail_bytes{job="Acme Web Server"}

In the second part of our blog series on Linux host monitoring with Prometheus, we will delve into more advanced concepts of Node Exporter and explore how to create insightful dashboards and queries.

Demystifying Prometheus and PromQL: A Beginner's Guide to Monitoring and Querying Metrics

shashankpai — Fri, 07 Jul 2023 09:02:03 +0000

1. Introduction

Monitoring and alerting play a critical role in ensuring the reliability and performance of software systems. In today's complex and distributed environments, it is essential to collect, analyze, and visualize metrics to identify and resolve issues proactively. Prometheus, an open-source monitoring and alerting toolkit, offers powerful features to meet these needs effectively.

2. Importance of Monitoring and Alerting:

Monitoring and alerting provide several benefits for software systems:

Proactive issue detection and resolution.
Performance optimization and resource allocation.
Capacity planning and scalability.
Troubleshooting and root cause analysis.

3. What is Prometheus?

3.1 Origin and History:
Prometheus was initially developed at SoundCloud in 2012 to monitor their dynamic, containerized infrastructure. It was later donated to the CNCF in 2016, gaining popularity and an active community.

3.2 Key Features and Benefits:

1. Time-series Data Model: Prometheus stores metrics with labels and timestamps, enabling efficient storage and analysis over time.
2. PromQL: A flexible querying language for extracting insights and performing operations on metrics.
3. Service Discovery and Dynamic Configuration: Built-in support for monitoring dynamic environments like Kubernetes.
4. Alerting and Notifications: Define alert rules and receive notifications via various channels.
5. Data Export and Integration: Export metrics to external systems and integrate with tools like Grafana.
6. Scalability and Performance: Designed to handle large-scale deployments with real-time monitoring capabilities.
7. Active Community and Ecosystem: Supported by a vibrant community, ensuring ongoing development and availability of extensions.

4. Prometheus Architecture

4.1 Components of Prometheus:

Prometheus Server: The central component responsible for data ingestion, storage, querying, and alerting. It scrapes metrics from configured targets and stores them in a time-series database. The server exposes an HTTP API for querying and retrieving metrics.
Exporters: Specialized components or libraries that expose metrics from various systems in a format that Prometheus can scrape. Exporters allow Prometheus to collect metrics from different sources such as web servers, databases, operating systems, and cloud platforms.
Pushgateway: A tool that allows pushing metrics from short-lived or batch jobs into Prometheus. This is useful for cases where scraping metrics periodically from these jobs is not feasible, such as cron jobs or ephemeral tasks.
Alertmanager: Handles alerts generated by Prometheus based on predefined rules. It allows operators to define alert rules and configure notification channels. Alertmanager coordinates the sending of alert notifications through email, Slack, PagerDuty, or other supported channels.

4.2 Pull-based Model Explained:
Prometheus follows a pull-based model for data collection. The Prometheus Server periodically scrapes metrics from configured targets by making HTTP requests to their endpoints. It retrieves the metrics in the Prometheus exposition format, which includes metric names, labels, values, and timestamps. The server then stores the scraped data in its time-series database for querying and analysis.
The pull-based model provides flexibility and resilience. Prometheus determines the frequency of scraping for each target, allowing customization based on the importance and stability of the metrics. Additionally, it handles cases where targets may have different scrape intervals or temporary unavailability without losing data.

4.3 Role of Exporters and Service Discovery:
Exporters play a crucial role in the Prometheus ecosystem. They act as adapters between Prometheus and various systems, providing the capability to expose metrics in a format Prometheus can understand. Exporters can be official integrations, community-contributed projects, or custom-built solutions specific to the system being monitored. They enable Prometheus to collect metrics from a wide range of sources without requiring modifications to the source systems.

4.4 Service Discovery
Service discovery is another essential aspect of Prometheus architecture. It simplifies the process of dynamically identifying and monitoring targets in a dynamic environment like Kubernetes. Prometheus supports multiple service discovery mechanisms, such as DNS-based service discovery, Kubernetes service discovery, file-based discovery, and more. These mechanisms automate the process of target discovery and ensure that Prometheus can adapt to changes in the environment, allowing for seamless monitoring of dynamic infrastructures.

5. Why Prometheus is worth considering

Dimensional data model
Powerful query language
Simple architecture and efficient server
Service discovery integration

5.1 Data model

The data model in Prometheus revolves around time series, which are sequences of data points representing the values of a specific metric over time. Each data point consists of a timestamp and a corresponding value. Time series in Prometheus are uniquely identified by metric names and a set of key-value pairs called labels.

What is timeseries?

Identifier:
The identifier for a time series is formed by the metric name and its associated labels. It is the combination of these two elements that distinguishes one time series from another. For example, if we have a metric named cpu_usage_percent and two labels, host and region, the identifier for a specific time series would be the metric name along with the label values. These identifiers enable Prometheus to differentiate and query specific subsets of time series.

Timestamp:
The timestamp in a time series represents the point in time at which a data point was recorded. In Prometheus, timestamps are typically represented as integers, often using Unix timestamps, which are the number of seconds or milliseconds since January 1, 1970. The timestamp indicates when a specific value in the time series was measured, allowing for the analysis of time-based patterns and trends.

Values:
The values in a time series correspond to the recorded measurements or observations of the metric at specific timestamps. In Prometheus, these values are typically floating-point numbers or decimal values, representing the metric's magnitude or measurement quantity. For example, in the case of cpu_usage_percent, the values could be decimals indicating the percentage of CPU utilization.

Example:

Let's consider an example time series for the metric cpu_usage_percent with labels host="server1" and region="us-east". Here's how it might look:

cpu_usage_percent{host="server1", region="us-east"}

In this example, the PromQL query selects the time series for the cpu_usage_percent metric with the label values host="server1" and region="us-east". It retrieves the CPU usage data specifically for "server1" in the "us-east" region.

cpu_usage_percent{host="server1", region="us-east"}

Output:
Timestamp: 1626937200
Value: 75.6


Timestamp: 1626938100
Value: 81.2


Timestamp: 1626939000
Value: 78.9

In this example, the output shows multiple data points representing the cpu_usage_percent metric for the time series with labels host="server1" and region="us-east". Each data point consists of a timestamp (e.g., 1626937200) and the corresponding value (e.g., 75.6). The timestamps represent specific points in time when the CPU usage percentage values were recorded for the given label combination.

** 5.2 Querying**

PromQL

PromQL is a functional query language designed for time series data in Prometheus. It excels at performing computations and transformations on time series, making it great for analyzing monitoring data. Unlike SQL-style languages, PromQL focuses on time series computations rather than structured tabular data. With PromQL, you can aggregate, filter, and perform mathematical calculations on time series, enabling you to derive meaningful insights and perform advanced analysis on your monitoring data efficiently. Its intuitive syntax and functional approach make it a preferred choice for working with time series data in Prometheus.

Lets try out few PromQL queries

Example:

List all partitions in my infrastructure with more than 100 GB capacity that are not mounted on root

node_filesystem_size_bytes{mountpoint!="/"} / 1e9 > 100

Explanation:
node_filesystem_size_bytes: This metric represents the size of the filesystem in bytes.
{mountpoint!="/"}: This selector filters out the root filesystem, as indicated by the mountpoint label not equal to "/".
/ 1e9: This division converts the size from bytes to gigabytes.
> 100: This condition filters the time series based on a capacity threshold of 100GB.
The query selects all the time series that satisfy the condition of having a filesystem size greater than 100GB (> 100) and are not mounted on the root (mountpoint!="/") in your infrastructure.

Output

node_filesystem_size_bytes{mountpoint="/home"}: 150
node_filesystem_size_bytes{mountpoint="/data"}: 250

2nd example , we will create a promql query to get the ratio of request errors across all service instances

To calculate the ratio of request errors across all service instances, you can use the following PromQL query:


sum(rate(http_requests_total{status_code=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))

Explanation:

http_requests_total: This metric represents the total number of HTTP requests made to the service.
{status_code=~"5.."}: This selector filters the metric to include only requests with status codes starting with "5", indicating server errors.
rate(http_requests_total{status_code=~"5.."}[5m]): This calculates the per-second rate of HTTP requests with status codes indicating server errors over the past 5 minutes.
rate(http_requests_total[5m]): This calculates the per-second rate of all HTTP requests over the past 5 minutes.

The query divides the rate of HTTP requests with status codes indicating server errors by the rate of all HTTP requests to calculate the ratio of request errors across all service instances.
The result of this query will be a decimal value representing the ratio of request errors. For example, a value of 0.05 indicates that 5% of the requests across all service instances resulted in server errors.

Forem: shashankpai

Modern Infrastructure as Code: OpenTofu vs. Crossplane vs. Pulumi

Table of Contents

Introduction

IaC Evolution Timeline

DevOps → SRE → Platform Engineering

IaC Comparison At-a-Glance

OpenTofu: The Open Source Terraform Alternative

Project Structure Example

Code Example: AWS VPC Configuration

Key Architectural Concepts

When to Choose OpenTofu

Crossplane: Kubernetes-Native Infrastructure

Project Structure Example

Code Example: PostgreSQL Database in AWS

Key Architectural Concepts

When to Choose Crossplane

Pulumi: Infrastructure as Actual Code

Project Structure Example

Code Example: AWS VPC in Python

Key Architectural Concepts

When to Choose Pulumi

Choosing the Right Tool

Decision Flowchart

Quick Start Guide

OpenTofu Quick Start

Crossplane Quick Start

Pulumi Quick Start

Common Pitfalls to Avoid

OpenTofu Pitfalls

Crossplane Pitfalls

Pulumi Pitfalls

Conclusion

From Zero to Hosted: Building a Static Website Platform with Pulumi and MinIO

Introduction

What You'll Learn

Prerequisites

What is Pulumi?

Installing Pulumi

Setting Up Our MiniCloud Infrastructure

1. Creating the Project Directory

2. Initializing a Pulumi Project

3. Installing Dependencies

4. Installing the Pulumi Docker Provider

5. Defining Our MinIO Infrastructure

6. Deploying MinIO

7. Accessing MinIO

Enhancing Our Setup with NGINX and SSL

Generating a Self-Signed Certificate with SAN

Setting Up NGINX as a Reverse Proxy

Option 1: Subpath Routing

Option 2: Subdomain Split

Deploying a Static Website to MinIO

Setting Up the MinIO Client (mc)

Handling Self-Signed Certificate Issues

Option 1: Using the --insecure Flag

Option 2: Trusting the Self-Signed Certificate

Configuring MinIO for Website Hosting

Troubleshooting Common Issues

Issue: ModuleNotFoundError: No module named 'pulumi_docker'

Issue: Pulumi asks for a passphrase

Cleaning Up Resources

Conclusion

A Beginner’s Guide to Distributed Tracing with OpenTelemetry and Jaeger

Why Distributed Tracing?

Overview of the Setup

1. Setting Up Jaeger

Command to Run Jaeger

2. Instrumenting a Python App with OpenTelemetry

Install OpenTelemetry Libraries

Code: A Simple Flask Application

3. Visualizing Traces in Jaeger

4. Dockerizing the Application

Docker Compose File

Dockerfile for Flask App

5. Running the Demo

Code Explanation

Conclusion

Building a Simple CDN with NGINX and Docker: A Step-by-Step Guide

Introduction: Why Use a CDN?

Option 1: Using the `--insecure` Flag

Step 4: Configuring DNS Simulation with `/etc/hosts`

Understanding `flux bootstrap` command

1.Create the `prometheus` user

5. Move the binary files to /usr/local/bin/ directory and set their ownership to `prometheus` user.

6. Move the files from the tmp archive to the appropriate locations, and set ownership on these files and directories to the `prometheus` user