Forem: markbosire

GitOps Bridge Pattern On A Local Kind Cluster

markbosire — Mon, 17 Mar 2025 08:02:14 +0000

Last week, I had the opportunity to attend the "Bridging Worlds: The Magic of GitOps in Kubernetes" virtual event organized by Cloud Native Nairobi. The session was eye-opening, particularly the deep dive into the GitOps Bridge Pattern , a powerful approach that combines Infrastructure as Code (IaC) and GitOps methodologies for managing Kubernetes resources.

The Problem the Pattern Solves

During the event, the speaker highlighted a common challenge in Kubernetes environments: while we use IaC tools like Terraform to provision cloud resources, and GitOps tools like ArgoCD to manage Kubernetes workloads, there's often a disconnect between these two worlds.

The crux of the issue lies in Kubernetes addons that depend on external cloud resources. These addons need configuration metadata from those external resources, but traditionally, there hasn't been a clean way to bridge the gap between the IaC-provisioned resources and the GitOps-managed deployments.

My Local Implementation

Inspired by the event, I decided to implement the GitOps Bridge Pattern myself using a local Kind cluster. This would allow me to test the pattern without incurring cloud costs while gaining hands-on experience with this architectural approach.

I set up a project that utilizes Terraform to configure the overall environment and ArgoCD to manage the actual Kubernetes resources. My approach involved:

Creating a local Kind cluster as the foundation
Using Terraform with the gitops-bridge Helm module to establish the bridge
Configuring ArgoCD to monitor a Git repository for changes
Deploying both a simple application (Nginx) and addons(helm charts)- Prometheus monitoring stack

The Implementation Details

The full project and setup instructions can be found in : https://github.com/markbosire/gitops-repo

The heart of my implementation is the Terraform configuration that establishes the GitOps bridge. In my main.tf file, I defined the necessary providers for Kubernetes and Helm, then configured the locals block with my cluster details and addon preferences:

locals {
  cluster_name = "gitops-demo"
  environment  = "dev"
  gitops_repo  = "https://github.com/markbosire/gitops-repo"

  # OSS addons configuration
  oss_addons = {
    enable_kube_prometheus_stack = true  
    enable_prometheus_adapter    = true  
  }

  # Merge all addon categories
  addons = merge(
    local.oss_addons,
    {
      kubernetes_version = "1.26"
    }
  )
}

The magic happens in the gitops_bridge module, which takes these configurations installs ArgoCD and creates the necessary ArgoCD applications:

module "gitops_bridge" {
  source = "gitops-bridge-dev/gitops-bridge/helm"

  cluster = {
    cluster_name = local.cluster_name
    environment  = local.environment
    metadata     = merge(
      {
        repo_url  = local.gitops_repo
        repo_path = "apps"
      },
      local.addons_metadata
    )
    addons = local.addons
  }

  apps = local.argocd_apps
}

For the applications side, I created ArgoCD application manifests like nginx.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nginx
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/markbosire/gitops-repo
    path: nginx-manifests
    targetRevision: HEAD
  destination:
    server: https://kubernetes.default.svc
    namespace: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

And for the addons, I utilized an ApplicationSet (addons.yaml) that dynamically creates helm charts found in the addons folder in the repo:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-addons
  namespace: argocd
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            environment: dev
  template:
    metadata:
      name: 'bootstrap-addons'
    spec:
      project: default
      source:
        repoURL: 'https://github.com/markbosire/gitops-repo'
        path: 'addons'
        targetRevision: 'main'
        directory:
          recurse: true
      destination:
        namespace: 'argocd'
        name: '{{name}}'
      syncPolicy:
        automated: {}

The "Aha" Moment

Running terraform apply brought everything together. I watched as Terraform handled the initial setup and then ArgoCD took over, continuously ensuring my cluster state matched what was defined in my Git repository. When I accessed the ArgoCD UI, I could see my Nginx application and the Prometheus stack being deployed and maintained automatically.

The most satisfying part was seeing how cleanly the pattern handled the Prometheus stack deployment. The ApplicationSet for kube-prometheus-stack was particularly elegant:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: kube-prometheus-stack
  namespace: argocd
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            environment: dev
  template:
    metadata:
      name: 'kube-prometheus-stack'
    spec:
      project: default
      source:
        chart: kube-prometheus-stack
        repoURL: https://prometheus-community.github.io/helm-charts
        targetRevision: 55.4.0
        helm:
          valueFiles:
            - values.yaml
      destination:
        server: https://kubernetes.default.svc
        namespace: monitoring
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true
          - ServerSideApply=true

What I Learned

Implementing the GitOps Bridge Pattern in my local environment taught me several valuable lessons:

The bridge is real: The pattern effectively connects the IaC world with the GitOps world, allowing for seamless management of resources across both domains.
Local testing is powerful: By using Kind, I could experiment freely without worrying about cloud costs or affecting cloud environments.
Automation is transformative: Once set up, the automatic synchronization between my Git repository and cluster state eliminated manual deployment steps and configuration drift.
Monitoring is integrated: Including the Prometheus stack demonstrated how the pattern can handle complex addons with multiple components.

Next Steps

Having successfully implemented the pattern locally, my next steps are to:

Deploy this pattern in a cloud environment where I can work with real external resources
Expand the addons to include other common tools like Ingress controllers and certificate managers
Implement a full CI/CD pipeline that updates the Git repository, triggering the GitOps process

The Cloud Native Nairobi event opened my eyes to the GitOps Bridge Pattern, but actually implementing it myself solidified my understanding. If you're working with Kubernetes and struggling with the gap between IaC and GitOps tools, I highly recommend giving this pattern a try , even if just in a local Kind cluster to start.

Effortlessly Navigate and Search Large-Scale Repositories with fzf and Ripgrep

markbosire — Wed, 12 Mar 2025 17:27:21 +0000

In a fast-paced development environment, efficiency is key. Searching for files, commits, or code snippets in a Git repository can be time-consuming if you're relying solely on built-in Git commands. Fortunately, tools like Fuzzy Finder (fzf) and Ripgrep (rg) can significantly enhance your Git workflow, making navigation and searching seamless.

Let’s explore how to use these tools with your simple-repo example, starting with setting up local branches for feature-branch and bugfix-branch.

Installing fzf and ripgrep on Ubuntu

To install both tools on Ubuntu, use the following commands:

sudo apt update
sudo apt install fzf ripgrep -y

Verify the installations:

fzf --version
rg --version

What Are Fuzzy Finder and Ripgrep?

Fuzzy Finder (fzf): A command-line fuzzy finder that helps you quickly search and navigate through lists, including files, Git commits, branches, and more.
Ripgrep (rg): A blazing-fast search tool for finding text within files. It is significantly faster than traditional tools like grep and respects .gitignore by default.

Using fzf and rg in Your Git Workflow

Let’s use your simple-repo as an example to demonstrate how these tools can enhance your workflow.

1. Cloning the Repository

Clone your repository from GitHub:

git clone https://github.com/markbosire/simple-repo.git
cd simple-repo

2. Setting Up Local Branches

After cloning, you’ll only have the master branch locally. To create local branches for feature-branch and bugfix-branch that track their remote counterparts, use the following commands:

git checkout -b feature-branch origin/feature-branch
git checkout -b bugfix-branch origin/bugfix-branch

Now, when you run git branch, you’ll see:

  bugfix-branch
  feature-branch
* master

3. Finding and Checking Out Branches

Instead of manually typing out branch names, use fzf to interactively search and switch branches.

List and Checkout Local Branches

git branch | fzf | xargs git checkout

Example Usage

If you type feature, fzf will filter branches like feature-branch. Selecting it will check out that branch.

4. Searching Through Git History

Searching through commit messages can be tedious with git log. Use fzf to search through your commit history and checkout a commit:

git log --oneline --graph --decorate --color --all | fzf | cut -d' ' -f1 | xargs git checkout

Example Usage

If you type feature, fzf will filter commits related to the feature-branch. Selecting a commit will check it out.

5. Searching for a File in the Repository

If you need to find a file quickly within your simple-repo:

git ls-files | fzf

To open the selected file in VS Code:

git ls-files | fzf | xargs code

Example Usage

If you type file, fzf will filter files like file1.txt and file2.txt. Selecting one will open it in VS Code.

6. Finding Specific Code Snippets with Ripgrep

When searching for code snippets within your repository, ripgrep (rg) is much faster than grep. For example, to find occurrences of Hello in your simple-repo:

rg Hello

Example Usage

If you type Hello, ripgrep will list occurrences across files.

Expected Output:

file1.txt:1:Hello, World!

To integrate ripgrep with fzf for interactive searching:

rg --files | fzf --preview='rg --color=always --line-number {}'

7. Searching for Changed Files in Git

To search within modified or staged files:

git diff --name-only | fzf

For searching unstaged changes:

git status -s | awk '{print $2}' | fzf

Example Usage

If you modify file1.txt and type file, fzf will show the modified file.

8. Grepping Through Git-Tracked Files Only

By default, ripgrep searches all files, including ignored ones. To limit it to only Git-tracked files:

rg --files | fzf --preview='rg --color=always --line-number {}'

Alternatively, to search for a specific pattern in your simple-repo:

rg "simple" $(git ls-files)

Example Usage

If you type simple, ripgrep will show occurrences in file2.txt.

Expected Output:

file2.txt:1:This is a simple repository.

9. Interactive Staging

Staging selected files interactively:

git status -s | fzf | awk '{print $2}' | xargs git add

Example Usage

If you modify file1.txt and type file, fzf will show the modified file. Selecting it will stage the file.

Conclusion

Using fzf and ripgrep in your Git workflow can greatly improve your efficiency, making it easier to navigate repositories, find relevant information, and streamline daily tasks. By integrating these tools into your workflow with your simple-repo, you can reduce friction and focus more on coding rather than searching for files or commits.

Practicing Terraform Locally Without a Cloud Provider

markbosire — Tue, 11 Mar 2025 11:44:38 +0000

Practicing Terraform locally offers several advantages, especially for those who are new to infrastructure as code (IaC) or want to experiment without incurring cloud costs. Here are some key benefits:

Cost-Effective Learning: You can experiment and learn Terraform without the need for cloud resources, which can be costly.
Rapid Iteration: Local environments allow for quick testing and iteration, speeding up the learning process.
Isolated Environment: A local setup provides a sandboxed environment where you can make mistakes without affecting production systems.
No Cloud Dependencies: You can practice Terraform without needing access to cloud providers, making it accessible to everyone.
Realistic Environment: Using tools like kind (Kubernetes IN Docker) allows you to simulate a real Kubernetes cluster locally.

Before diving into Terraform configurations, let's start by setting up a local Kubernetes cluster using kind (Kubernetes IN Docker). This will provide the foundation for our Terraform practice environment.

Creating a Kind Cluster

Kind is a lightweight tool for running local Kubernetes clusters using Docker containers as nodes. Here's how to get started:

1. Install Kind

If you haven't already installed kind, you'll need to do so first:

For macOS (using Homebrew):

brew install kind

For Linux:

curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind

For Windows (using Chocolatey):

choco install kind

2. Create a Basic Kind Configuration File

Create a file named kind-config.yaml with the following content:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
  - role: worker
  - role: worker

3. Create the Cluster

Now, create your kind cluster using the configuration file:

kind create cluster --name terraform-practice --config kind-config.yaml

You should see output similar to:

Creating cluster "terraform-practice" ...
 ✓ Ensuring node image (kindest/node:v1.27.3) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-terraform-practice"
You can now use your cluster with:

kubectl cluster-info --context kind-terraform-practice

Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂

4. Verify Your Cluster

Ensure that your cluster is running and kubectl is properly configured to use it:

kubectl cluster-info --context kind-terraform-practice

You should see information about your Kubernetes control plane and CoreDNS.

Also, check that your nodes are ready:

kubectl get nodes

The output should show your node with status "Ready".

Terraform Configuration Files

Now that your kind cluster is up and running, you can proceed with your Terraform configurations. Here are the files you need:

1. main.tf - Provider Configuration

terraform {
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.0"
    }
  }
}

provider "kubernetes" {
  config_path = "~/.kube/config"
}

2. nginx-deployment.tf - NGINX Deployment Configuration

resource "kubernetes_deployment" "nginx" {
  metadata {
    name = "nginx-deployment"
    labels = {
      app = "nginx"
    }
  }

  spec {
    replicas = 3

    selector {
      match_labels = {
        app = "nginx"
      }
    }

    template {
      metadata {
        labels = {
          app = "nginx"
        }
      }

      spec {
        container {
          image = "nginx:latest"
          name  = "nginx"
          port {
            container_port = 80
          }
        }
      }
    }
  }
}

3. nginx-service.tf - Service Configuration for Exposing NGINX

resource "kubernetes_service" "nginx" {
  metadata {
    name = "nginx-service"
  }

  spec {
    selector = {
      app = "nginx"
    }

    port {
      port        = 80
      target_port = 80
      node_port   = 30080  
    }

    type = "NodePort"
  }
}

Deploying with Terraform

With your kind cluster ready, you can now follow these steps to deploy your infrastructure:

Initialize Terraform:

   terraform init

Plan your deployment:

   terraform plan

Apply the configuration:

   terraform apply

Verify the deployment:

   kubectl get deployments
   kubectl get pods
   kubectl get services

Accessing Your NGINX Service

You can manually forward the port by running:

kubectl port-forward svc/nginx-service 8080:80

Then, visit:

➡ http://localhost:8080

You should see the default NGINX welcome page.

Cleaning Up

When you're done practicing, you can:

Destroy your Terraform resources:

   terraform destroy

Delete your kind cluster:

   kind delete cluster --name terraform-practice

This complete workflow gives you a practical way to experiment with Terraform locally using a kind Kubernetes cluster, without any cloud provider costs or dependencies.