Forem: Hima

Streamlining AWS CloudFormation Template Tag Management with Tagion

Hima — Sun, 06 Apr 2025 03:33:30 +0000

Managing tags across multiple AWS CloudFormation templates can be a tedious and error-prone process. Enter Tagion, a command-line tool that simplifies the process of adding and managing tags across your CloudFormation resources.

The Challenge

AWS tags are essential for:

Cost allocation
Resource organization
Access control
Automation
Compliance requirements

However, manually adding tags to CloudFormation templates is:

Time-consuming
Error-prone
Often overlooked during resource creation
Inconsistent across templates

Introducing Tagion

Tagion is a Go-based CLI tool that automatically analyzes and adds tags to AWS CloudFormation templates. It supports both YAML and JSON formats and can process single templates or entire directories.

Key Features

Bulk Processing: Process multiple templates in a directory with a single command
Smart Analysis: Only modifies resources that support tags
Preview Changes: Shows which templates will be modified before making changes
Preserve Existing Tags: Merges new tags with existing ones without duplicates
Multiple Format Support: Works with both YAML and JSON templates

Usage Example

Define your tags in a JSON configuration file

  {
    "tags": {
      "Environment": "Production",
      "Project": "TagionCFN",
      "Owner": "DevOps",
      "ManagedBy": "Tagion"
    }
  }

Run Tagion

  tagion -tags tags.json -path templates/

Review the proposed changes

╭────────────────────────────────────────────┬───────────┬────────────────────╮
│ TEMPLATE                                   │ RESOURCES │ STATUS             │
├────────────────────────────────────────────┼───────────┼────────────────────┤
│ templates/ec2-no-tags.yaml                 │         1 │ Will be modified   │
│ templates/s3-with-tags.yaml                │         1 │ Has tags           │
│ templates/multi-resources.json             │         2 │ Will be modified   │
│ templates/unsupported.yaml                 │         2 │ No changes needed  │
├────────────────────────────────────────────┼───────────┼────────────────────┤
│ Total                                      │         6 │ To modify: 2       │
╰────────────────────────────────────────────┴───────────┴────────────────────╯

Confirm and apply changes

  Do you want to proceed with these changes? [y/N]:

Supported AWS Resources

Currently supports tags for common AWS resources including:

EC2 instances
S3 buckets
RDS databases
DynamoDB tables
Lambda functions

Benefits

Time Savings: Automate tag addition across multiple templates
Consistency: Ensure uniform tagging across your infrastructure
Compliance: Easily implement tagging policies
Error Prevention: Avoid manual tagging mistakes
Non-Destructive: Preserves existing tags and only adds missing ones

Getting Started

Install Tagion using Go:

go install github.com/himaatluri/tagion@latest

Future Enhancements

Support for more AWS resource types
Custom tag validation rules
Integration with AWS Organizations tag policies
Tag removal and modification features
CI/CD pipeline integration

Conclusion

Tagion simplifies CloudFormation template tag management, making it easier to maintain consistent tagging across your AWS infrastructure. It's an essential tool for DevOps teams managing multiple CloudFormation templates and needing to ensure proper resource tagging.

The project is open source and available on GitHub under the Apache License 2.0. Contributions are welcome!

This blog post was written for the Tagion project. For more information, visit our GitHub repository.

GCP Workload Identity - Kubernetes Federation

Hima — Wed, 29 Jan 2025 04:04:05 +0000

How to Authorize Kubernetes Pods in Rancher Desktop to Google Cloud APIs Using Workload Identity Federation (WIF)

In this post, I’ll show you how I used Workload Identity Federation (WIF) to authorize a Kubernetes pod running in Rancher Desktop to access Google Cloud APIs—without needing service account keys. This approach simplifies authentication by using federated identities for workload access, making it more secure and manageable.

Why Use Workload Identity Federation?

Workload Identity Federation (WIF) eliminates the need to handle service account keys. Instead of relying on long-lived keys, you use identity federation to authenticate workloads. This improves security, as there are no keys to rotate or expose, and is ideal for cloud-native environments like Kubernetes.

Steps to Set Up Workload Identity Federation with Rancher Desktop

1. Ensure prerequisites are met

Ensure that your Rancher Desktop cluster meets the following:

Kubernetes version 1.20 or higher.
ServiceAccount token volume projections enabled in kube-apiserver. In Rancher Desktop, this is typically already enabled since it's based on K3s, which supports projected volumes.

2. Set up the Kubernetes cluster issuer

To get the cert and key:

$ kubectl config view -o=jsonpath="{.users[?(@.name == 'rancher-desktop')].user.client-certificate-data}" --flatten | base64 -d > rancher.crt

$ kubectl config view -o=jsonpath="{.users[?(@.name == 'rancher-desktop')].user.client-key-data}" --flatten | base64 -d > rancher.key

Retrieve the cluster's issuer URL:

   $ curl -k --cert rancher.crt --key rancher.key https://127.0.0.1:6443/.well-known/openid-configuration

The expected format is something like https://kubernetes.default.svc.

Download the JWKS for the cluster:

   $ curl -k --cert rancher.crt --key rancher.key https://127.0.0.1:6443/openid/v1/jwks > cluster-jwks.json

Rancher Desktop will expose this endpoint since it's based on standard Kubernetes configurations.

3. Create the workload identity pool and provider in GCP

Create a Workload Identity Pool:

   gcloud iam workload-identity-pools create rancher-cowboy \
     --location="global" \
     --description="Workload identity for Rancher Desktop" \
     --display-name="Rancher Desktop Pool"

Add the Rancher Desktop Kubernetes cluster as a provider:

   gcloud iam workload-identity-pools providers create-oidc rancher-desktop\
     --location="global" \
     --workload-identity-pool="rancher-cowboy" \
     --issuer-uri="https://kubernetes.default.svc" \
     --attribute-mapping="google.subject=assertion.sub" \
     --jwk-json-path="cluster-jwks.json"

4. Grant IAM access

Grant access to the Kubernetes ServiceAccount:

$ gcloud projects add-iam-policy-binding PROJECT_ID \
    --role=roles/viewer \
    --member=principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/rancher-cowboy/subject/system:serviceaccount:default:default

5. Deploy the Kubernetes workload

Sample workload with alpine image and gcloud installed to test the access

apiVersion: v1
kind: Pod
metadata:
  name: horse
  namespace: default
spec:
  containers:
  - name: rhorse
    image: google/cloud-sdk:alpine
    command: ["/bin/sh", "-c", "gcloud auth login --cred-file $GOOGLE_APPLICATION_CREDENTIALS && gcloud auth list && sleep 600"]
    volumeMounts:
    - name: token
      mountPath: "/var/run/service-account"
      readOnly: true
    - name: workload-identity-credential-configuration
      mountPath: "/etc/workload-identity"
      readOnly: true
    env:
    - name: GOOGLE_APPLICATION_CREDENTIALS
      value: "/etc/workload-identity/credential-configuration.json"
  serviceAccountName: default
  volumes:
  - name: token
    projected:
      sources:
      - serviceAccountToken:
          audience: https://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/rancher-cowboy/providers/rancher-desktop
          expirationSeconds: 3600
          path: token
  - name: workload-identity-credential-configuration
    configMap:
      name: wif-cm

6. Authentication and verification

Verify authentication:

$ kubectl exec horse --namespace default -- gcloud auth print-access-token

$ kubectl exec horse --namespace default -- gcloud iam service-accounts list --project PROJECT_ID

This setup ensures that Rancher Desktop running locally on your laptop can securely authenticate workloads with GCP using Workload Identity Federation.

Wait what happened? How did this work?

Workload Requests GCP Access, A K8s Pod in Rancher Desktop runs with a ServiceAccount, which is configured to receive a projected OIDC token.
Kubernetes Issues OIDC Token, The K8s API server generates an OIDC-compatible JWT for the Pod based on the ServiceAccount and its audience.
- Token Details, The token contains claims like:
- Issuer iss: The Kubernetes API server's OIDC URL.
- Audience aud: The target GCP Workload Identity Pool URL.
- Subject sub: The identity of the ServiceAccount system:serviceaccount:<namespace>:<service-account-name>.
- The token is signed by the K8s cluster.
Pod Passes Token to GCP: The token is presented to GCP during API requests via the Google Cloud SDK or client libraries in the workload.
GCP verifies the OIDC token by
1. Checking the token signature against the trusted Kubernetes OIDC issuer's public keys (usually from a .well-known/openid-configuration endpoint).
2. Ensuring that the token audience matches the configured audience for the GCP WIF provider.
3. Validating claims (like expiration and ServiceAccount identity).

In conclusion, I successfully authorized a Kubernetes pod running in Rancher Desktop to interact with Google Cloud APIs using Workload Identity Federation. This method eliminated the need for managing service account keys and simplified the authentication process using federated identities. It’s a more secure and scalable solution for accessing Google Cloud resources from Kubernetes workloads.

If you're looking to improve the security of your Kubernetes workloads, I highly recommend exploring Workload Identity Federation. It’s a great way to manage cloud access without the overhead of key management.

Introducing NetIrk: A Lightweight CLI Tool for High-Level Network Insights

Hima — Thu, 28 Nov 2024 07:33:14 +0000

Netirk is a lightweight and efficient CLI tool designed to perform high-level network analysis. Built with simplicity in mind, it offers essential features like connectivity checks, DNS tracing, and a lightweight server to test local networking setups. Whether you're troubleshooting or verifying connectivity, Netirk is here to streamline the process.

Installation

To get started with Netirk, install it via Go:

go install github.com/himaatluri/netirk

Usage

Basic Commands

Netirk offers a clean and easy-to-use CLI interface. Use the following commands for different operations:

Check Connectivity

Verify the response from a host and inspect SSL details if required.

netirk check --target <hostname> --verify-ssl

Example

netirk check --target google.com --verify-ssl

Output:

Getting server certs...

➥ Cert: 0 
 ￫ CA: false
 ￫ Issuer: WR2
 ￫ Expiry: Monday, 13-Jan-25 08:36:56 UTC
 ￫ PublicKey: 
   -----BEGIN CERTIFICATE-----

Trace

Diagnose connection details, including DNS resolution time, connection latency, and TLS handshake information.

netirk trace --host <hostname> --port <port>

Example 1

netirk trace --host https://amazon.com --port 8080

Output:

DNS Resolution done: 7.618718ms
Connect Done: 26.686553ms
Request failed: dial tcp 54.239.28.85:8080: connect: connection refused

Example 2

netirk trace --host https://amazon.com --port 443

Output:

DNS Resolution done: 7.553307ms
Connect Done: 27.578134ms
TLS Handshake Done: 83.745488ms
Time to first byte: 147.988115ms

Running a Lightweight Server

Netirk also includes a basic HTTP server to test local networking setups.

netirk server

Example:

➜ netirk server                                   
Starting a simple HTTP server on port: 8080
2024/11/25 23:46:37 request: GET /host
2024/11/25 23:46:44 request: GET /health

You can use curl to interact with the server:

curl localhost:8080/host
curl localhost:8080/health

Output:

hostname-prints/returned
healthy

Help Menu

To explore additional options or flags, use the help command:

➜  netirk git:(main) ./netirk help

               _    _        _    
  _ __    ___ | |_ (_) _ __ | | __
 | '_ \  / _ \| __|| || '__|| |/ /
 | | | ||  __/| |_ | || |   |   < 
 |_| |_| \___| \__||_||_|   |_|\_\


A portable network utility to check system reachability,
this utility can also be used to run a small http server when figuring out how to deploy a small
http server in a dynamic network landscape such as cloud platforms.

Usage:
  netirk [command]

Available Commands:
  check       Verify if host is reachable
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  server      Run local server for quick testing
  trace       Run local server for quick testing
  version     Print the version number of Netirk CLI

Flags:
  -h, --help   help for netirk

Use "netirk [command] --help" for more information about a command.

Why Choose Netirk?

Netirk is a versatile tool that focuses on providing network insights with minimal setup. Its intuitive commands and clean output make it a valuable addition to any SRE or platform engineer’s toolkit.

Netirk combines sleek performance with practical functionality. From SSL validation to DNS resolution timing, it empowers you to quickly debug and analyze network connections.

🛠️ How You Can Help
Code Contributions:
Whether you're a seasoned developer or just starting, there’s always a place for your skills. From bug fixes and new features to optimizing performance, your code matters.

Documentation & Tutorials:
Help others get started by improving our documentation or creating tutorials for beginners.

Testing & Feedback:
Test new features and report issues to help us refine the tool.

🌐 How to Get Started
Check Out Our GitHub Repository:
Visit our GitHub repo to explore the project, review open issues, and find contribution guidelines.

Join the Discussion:
Engage with us through GitHub Issues and Pull Requests

💡 Submit a Pull Request:
Found a bug or have a new feature idea? Submit a PR and let’s collaborate!

🙌 Thank You for Your Support!
Every contribution, big or small, brings us closer to making this tool an essential resource for network engineers everywhere. Let’s build something amazing together!

Kubernetes Deployment with Ansible: A Comprehensive Guide to Bootstraping Kubernetes Clusters

Hima — Sun, 21 Apr 2024 02:33:16 +0000

Kubernetes has emerged as the de facto standard for container orchestration, enabling organizations to manage and scale containerized applications seamlessly. However, setting up and managing Kubernetes clusters can be complex and time-consuming. Automation tools like Ansible can significantly streamline this process by providing a consistent and repeatable way to provision, configure, and manage Kubernetes clusters. In this blog post, we will explore how to use Ansible roles to bootstrap Kubernetes clusters, simplifying the deployment process and ensuring consistency across environments.

Before diving into the specifics of bootstrapping Kubernetes clusters with Ansible, let's briefly review what Ansible is and how it works. Ansible is an open-source automation tool that simplifies IT orchestration, configuration management, and application deployment. It uses a declarative language called YAML to define the desired state of systems, which it then applies through SSH connections.

Ansible Roles

Ansible roles are a way to organize and package automation content in a reusable and modular format. Roles encapsulate tasks, variables, handlers, and other Ansible components into a directory structure, making it easy to share and reuse automation logic across different projects. Leveraging roles promotes consistency, scalability, and maintainability in Ansible playbooks.

Now, let's discuss how to leverage Ansible roles to bootstrap Kubernetes clusters. The process involves several key steps:

Prerequisites

Before getting started, ensure that Ansible is installed on the control machine from which you will orchestrate the Kubernetes deployment. Additionally, you'll need SSH access to the target nodes where Kubernetes will be deployed.

Ansible Inventory:
Create an Ansible inventory file containing the details of the nodes in your Kubernetes cluster. This includes IP addresses, hostnames, and any necessary SSH configuration.

Variables and Templates:
Utilize Ansible variables and Jinja2 templates to parameterize the playbook and roles, allowing for flexibility and customization based on different environments or requirements. This includes specifying Kubernetes version, networking options, and other configuration parameters.

Execute the Ansible playbook against the inventory of nodes using the ansible-playbook command. Ansible will connect to each node via SSH and apply the defined tasks and roles, orchestrating the entire Kubernetes deployment process automatically.

Benefits of Using Ansible for Kubernetes Bootstraping:

Automation: Ansible automates the deployment process, reducing manual intervention and human error.
Consistency: Ansible roles ensure consistency across deployments, eliminating configuration drift.
Reusability: Roles can be reused and shared across projects, saving time and effort in development.
Flexibility: Ansible's modular structure allows for customization and adaptation to different environments and requirements.
Scalability: Ansible can scale to manage Kubernetes clusters of any size, from small development environments to large-scale production deployments.

Bootstraping Kubernetes clusters with Ansible offers a streamlined and efficient approach to infrastructure automation, enabling organizations to deploy and manage Kubernetes clusters with ease. By leveraging Ansible roles, teams can achieve consistency, reliability, and scalability in their Kubernetes deployments, ultimately accelerating time-to-market and improving operational efficiency.

This repository contains reference roles for Kubeadm and k3s setup, PRs are welcome they are work-in-progress

CD pipeline in GO

Hima — Sun, 28 Jan 2024 05:51:09 +0000

The IDEA

I often pondered the necessity of acquiring proficiency in a new DSL (Domain-Specific Language) for orchestrating releases and executing CD (Continuous Delivery) steps such as simply building a container image and running shell commands etc.. It seemed counterintuitive to me, as I questioned why I couldn't achieve my objectives using the language(s) I'm already proficient in (Python/Go).

Analysis

Analyzing the above idea a bit two key benefits stood out:

Time Efficiency: The potential to save time and redirect it towards enhancing my proficiency in the core language, exploring concepts like Go routines, packaging, and more.

Singular Focus: The ability to concentrate solely on my primary project, minimizing the need to delve into and comprehend additional DSLs.

The solution

While looking at this ideal scenario, I recently came across dagger.io, a brilliant tool crafted by the minds behind Docker. Intrigued, I decided to put it to the test by implementing CICD as code in a small project, specifically focusing on "Building a container image from a Dockerfile."

Project

To kickstart the process, I began by installing the dagger.io/dagger Go package and most importantly dagger CLI.

For testing this you need to have the Container Runtime Environment (CRE) which is Docker in my case(you can try with containerd which is the current CRE for K8s)

Just a recap, I have one basic test in my mind, build my image and ship it in my choice of code and I have no interest in learning a new configuration other than what I already knew.

Breaking that goal into steps, the following are the tasks we need to code.

Creation of a Dockerfile.
Building Image.
Ship it.

This experience with dagger.io marked a noteworthy exploration into simplifying CICD workflows, aligning seamlessly with my aspiration to articulate deployment tasks using the language integral to my primary project.

Action

Dagger Definition:

package main

import (
    "context"

    "dagger.io/dagger"
)

var DockerTag string = "123123123213.dkr.ecr.us-east-1.amazonaws.com"

func main() {
    ctx := context.Background()
    client, err := dagger.Connect(ctx)
    if err != nil {
        panic(err)
    }

    defer client.Close()

    thisImage := client.Container().Build(
        client.Host().Directory("./"),
        dagger.ContainerBuildOpts{
            Dockerfile: "../Dockerfile",
        })

    thisImage.Publish(ctx, DockerTag)
}

What just happened?

In the above snippet, you are importing the dagger package and defining the steps that the pipeline should do, basically converting our tasks from the goal to instructions.

When you submit this to the dagger command line,

It opens a new session to the dagger engine and submits the above API requests.
The dagger engine calculates the DAG(Directed Acyclic Graph) on how to achieve those tasks, simply computing the steps to produce the result.
when the steps are executed the responses are sent back to our program.
This is where we can trigger other APIs or may even trigger a different pipeline.

Conclusion

This is just a basic project, I just wanted to see how easy it is to create a pipeline in Go and additionally test the pipeline locally without needing a CICD instance.

what I like is that you can test your pipeline locally without needing to spin a local Jenkins or any other tool as a container and configure it and add build agents or install plugins etc.. with dagger I can eliminate all that extra stuff.

Dagger

Gitlab CLI

Hima — Mon, 03 Apr 2023 21:34:01 +0000

Use Gitlab CLI for watching pipeline status remotely

$ glab ci view

This will show the gitlab CI view in your terminal, you don't have to switch tabs and your focus. very helpful for focusing and avoiding distractions.

Verify Merge Request details, without hopping into a web browser
$ glab mr view

Download and configure today!
https://docs.gitlab.com/ee/integration/glab/