Forem: KALPESH

Running Production-Grade Databases on K8s

KALPESH — Sun, 03 May 2026 19:23:22 +0000

Who this is for: Developers and DevOps engineers who want to understand how to run databases reliably on Kubernetes — from the basics of StatefulSets, to replication consistency, to choosing between self-managed and Operator-based approaches.

Why Databases on Kubernetes Are Tricky
Your Three Options
Understanding StatefulSets
How Replication Works
Avoiding Data Inconsistency
Self-Managed vs Kubernetes Operator
Detailed Task Comparison
When to Choose What
Summary

1. Why Databases on Kubernetes Are Tricky

Kubernetes was originally designed for stateless workloads — apps where any pod can be replaced at any time without data loss. A web server is stateless. A database is not.

Databases are stateful — they hold your data on disk, they have a concept of a primary (the one that accepts writes) and replicas (copies), and if you restart them carelessly, you risk data corruption or split-brain scenarios.

Over time, the Kubernetes community built proper support for stateful workloads in the form of StatefulSets (stable since Kubernetes v1.9). But even with StatefulSets, running a database in production requires deep knowledge and careful planning.

2. Your Three Options

When you need a database for your app running in Kubernetes, you have three broad options:

Option 1 — Cloud Provider Managed Database (AWS RDS, GCP Cloud SQL, Azure Database)

Pros	Cons
Easy to get started	Not your DBA — slow queries are your problem
Managed backups	Vendor lock-in
High availability built-in	Limited customization (can't add extensions freely)
	Expensive at scale (usage-based pricing)
	No support for air-gapped / data-sovereignty requirements

Option 2 — Database Vendor Hosted Service (MongoDB Atlas, Elastic Cloud, PlanetScale)

Pros	Cons
Optimized for that specific database	Same vendor lock-in issues as cloud providers
Deep expertise from the vendor	Only offers their one database engine
	Can get expensive at scale

Option 3 — Self-hosted Inside Kubernetes

Pros	Cons
Full control	Requires deep Kubernetes + DB knowledge
No vendor lock-in	All operational tasks fall on you
Works on-premises or any cloud	High risk if done carelessly
Most flexible	Time-consuming to maintain

The good news: Option 3 can be made dramatically safer and simpler by using a Kubernetes Operator — covered in depth later in this guide.

3. Understanding StatefulSets

What Makes StatefulSets Different from Deployments

A regular Kubernetes Deployment treats all pods as interchangeable. Pod names are random (app-7d9f4b-xkqjp), and they can be created or destroyed in any order.

A StatefulSet gives each pod a stable, predictable identity:

myapp-0   ← always the first pod (usually the primary)
myapp-1   ← always the second pod (replica)
myapp-2   ← always the third pod (replica)

These names are permanent. If myapp-1 crashes and restarts, it comes back as myapp-1 — not a new random name.

Three Guarantees StatefulSets Provide

1. Ordered startup — Pods start one at a time, in order. myapp-1 will not start until myapp-0 is Running and Ready. This is critical because replicas need the primary to exist before they can sync from it.

2. Stable network identity — Each pod gets a predictable DNS name via a headless service:

myapp-0.myapp-svc.default.svc.cluster.local
myapp-1.myapp-svc.default.svc.cluster.local

This lets replicas always know exactly where to find the primary.

3. Stable storage (PersistentVolumeClaim per pod) — Each pod gets its own dedicated disk. If myapp-1 dies and is rescheduled on a different node, it reattaches to the same PVC and picks up exactly where it left off — no data loss.

# Simplified StatefulSet example
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: myapp
spec:
  serviceName: "myapp-svc"
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: mysql
        image: mysql:8.0
        ports:
        - containerPort: 3306
  volumeClaimTemplates:           # ← Each pod gets its own PVC
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

4. How Replication Works

The Primary-Replica Model

In a typical database StatefulSet with 3 replicas:

Client App
   │
   ├──── WRITE ──► myapp-0 (Primary)   ← Only pod that accepts writes
   │                    │
   │              replication
   │                    │
   └──── READ  ──► myapp-1 (Replica)   ← Read-only, synced from primary
                   myapp-2 (Replica)   ← Read-only, synced from primary

Rule #1: All writes go to the primary only.

The primary pod (myapp-0) is the single source of truth. You connect to it using its stable DNS name:

myapp-0.myapp-svc.default.svc.cluster.local:3306

Replicas will reject write operations at the database level (MySQL, PostgreSQL, and MongoDB all enforce this automatically).

Rule #2: Reads can be distributed across replicas.

This improves read throughput and reduces load on the primary. You connect to replicas using:

myapp-1.myapp-svc.default.svc.cluster.local:3306
myapp-2.myapp-svc.default.svc.cluster.local:3306

Or use the headless service DNS to load-balance across all replicas.

Ordered Startup in Detail

Time 0: myapp-0 starts → initializes as primary
Time 1: myapp-0 is Running + Ready
Time 2: myapp-1 starts → connects to myapp-0, begins sync
Time 3: myapp-1 is Running + Ready
Time 4: myapp-2 starts → connects to myapp-0, begins sync
Time 5: myapp-2 is Running + Ready

If myapp-0 takes too long to start, Kubernetes waits. It will never start myapp-1 until myapp-0 passes its readiness probe.

5. Avoiding Data Inconsistency

This is the most important section. Replication introduces a window where replicas may not have the latest data from the primary. Here's how to handle it.

Problem: Replication Lag

Asynchronous replication (the default in most databases) means:

Client writes to primary → primary commits → returns success to client
Primary sends the change to replicas in the background
Replicas apply the change a few milliseconds (or more) later

If a client writes data and then immediately reads from a replica, they might get stale data — the replica hasn't caught up yet.

Client:  writes  "balance = 1000"  to primary
Client:  reads   "balance"         from replica  →  gets "500"  ← STALE!
         (replica hasn't received the update yet)

Synchronous replication solves this but at a cost — the primary waits for the replica to confirm before returning success to the client. Writes are slower, but every replica is always up to date.

Solution 1 — Route critical reads to the primary

For operations where you cannot tolerate stale data (payment confirmations, inventory checks), always read from the primary:

# Critical read → primary
SELECT balance FROM accounts WHERE id=123;
→ connect to myapp-0.myapp-svc (primary)

# Non-critical read → replica (dashboards, reports)
SELECT COUNT(*) FROM orders WHERE date > '2024-01-01';
→ connect to myapp-1.myapp-svc (replica)

Solution 2 — Use readiness probes to block traffic until synced

A pod's readiness probe tells Kubernetes whether the pod is ready to receive traffic. Add a custom check that verifies the replica's replication lag before marking it ready:

readinessProbe:
  exec:
    command:
    - /bin/sh
    - -c
    - |
      # Only mark ready if replication lag < 5 seconds
      mysql -e "SHOW SLAVE STATUS\G" | grep "Seconds_Behind_Master: 0"
  initialDelaySeconds: 30
  periodSeconds: 10

Until this probe passes, Kubernetes routes zero traffic to the pod. This prevents dirty reads from a partially synced replica.

Solution 3 — Use PodDisruptionBudgets to prevent unsafe scaling

A PodDisruptionBudget ensures that at least N pods remain available during voluntary disruptions (node upgrades, pod evictions):

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: myapp-pdb
spec:
  minAvailable: 2        # Always keep at least 2 pods running
  selector:
    matchLabels:
      app: myapp

This prevents a scenario where all replicas go down at the same time, leaving only the primary — which then has no failover if it crashes.

Solution 4 — Never write to replicas

Enforce this at the application level. Use two separate connection pools:

# Python example (pseudocode)
write_db = connect("myapp-0.myapp-svc:3306")   # Primary only
read_db  = connect("myapp-svc:3306")            # Headless service → replicas

def transfer_funds(from_id, to_id, amount):
    write_db.execute("UPDATE accounts SET balance=... WHERE id=?", from_id)
    write_db.execute("UPDATE accounts SET balance=... WHERE id=?", to_id)
    # Read-back the new balance from the PRIMARY, not a replica
    return write_db.fetchone("SELECT balance FROM accounts WHERE id=?", from_id)

Summary: Consistency Rules

Scenario	Read from	Why
Payment confirmed, show balance	Primary	Cannot tolerate stale data
Dashboard: orders last 30 days	Replica	Small lag is acceptable
After a write, confirm the value	Primary	Replica might not have it yet
Search / reporting queries	Replica	Heavy query, offload from primary

6. Self-Managed vs Kubernetes Operator

Once you decide to run your database inside Kubernetes, you have two approaches:

Self-Managed

You write and maintain all the Kubernetes resources yourself: StatefulSets, Services, ConfigMaps, init containers for replication setup, CronJobs for backups, shell scripts for failover, certificate management for TLS, and custom monitoring configuration.

You are the DBA.

Kubernetes Operator

A Kubernetes Operator is an application that runs inside your cluster and extends Kubernetes for a specific workload. It encodes the operational knowledge of a human DBA into automation.

You declare what you want using a Custom Resource Definition (CRD):

# With a MySQL Operator (e.g. KubeDB)
apiVersion: kubedb.com/v1alpha2
kind: MySQL
metadata:
  name: myapp
  namespace: default
spec:
  version: "8.0.27"
  replicas: 3
  topology:
    mode: GroupReplication
  storage:
    storageClassName: standard
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 10Gi

The Operator reads this and automatically creates the StatefulSet, Services, ConfigMaps, sets up replication, configures TLS, and wires up monitoring. You never write any of that YAML yourself.

The Operator is your automated DBA.

7. Detailed Task Comparison

Here is a task-by-task breakdown of what you do yourself vs what the Operator handles:

Provisioning

Self-managed:
You write the full StatefulSet YAML, a headless Service, a regular Service for reads, ConfigMaps for database config, and init containers for first-time setup scripts. This is typically 200–400 lines of YAML for a production-grade setup.

Operator:
You apply a single CRD (10–30 lines). The Operator generates all the underlying resources automatically and keeps them reconciled — if you accidentally delete a Service, the Operator recreates it.

Replication Setup

Self-managed:
You write init container scripts that detect whether the pod is myapp-0 (primary) or a replica, configure the database accordingly, and run the CHANGE MASTER TO ... (MySQL) or pg_basebackup (PostgreSQL) equivalent. This is fragile and database-version-specific.

Operator:
The Operator knows the internals of the specific database it manages. It configures primary-replica topology automatically, using the correct commands for that database version.

Failover

Self-managed:
When myapp-0 crashes, nothing happens automatically. You must:

Detect the failure (monitoring alert, manual check)
Identify which replica is most up-to-date (check replication lag)
Run the promotion command on that replica
Update all connection strings pointing to the old primary
Reconfigure remaining replicas to sync from the new primary

This can take 5–30 minutes manually and causes downtime.

Operator:
The Operator continuously monitors pod health using Kubernetes watches. When it detects the primary is down, it automatically elects the most up-to-date replica as the new primary, reconfigures all other replicas to sync from it, and updates the Service endpoint — typically within 30–60 seconds, with minimal or no downtime.

Backups

Self-managed:
You write a Kubernetes CronJob that runs a backup container on a schedule, runs mysqldump or pg_dump or a snapshot tool, uploads the result to S3, and handles retention (deleting old backups). You also need to periodically test restores manually.

Operator:
Operators like KubeDB provide a BackupConfiguration CRD:

apiVersion: stash.appscode.com/v1beta1
kind: BackupConfiguration
metadata:
  name: myapp-backup
spec:
  schedule: "0 2 * * *"        # 2 AM daily
  repository:
    name: s3-repo
  target:
    ref:
      apiVersion: appcatalog.appscode.com/v1alpha1
      kind: AppBinding
      name: myapp
  retentionPolicy:
    name: keep-last-7
    keepLast: 7

The Operator handles scheduling, execution, upload, and retention automatically.

Scaling

Self-managed:
Running kubectl scale statefulset myapp --replicas=4 adds a new pod, but you still need to verify it has fully synced before it receives read traffic. If you forget to check and route reads to an unsynced replica, users see stale data.

Operator:
Updating the replicas field in your CRD triggers the Operator to spin up the new pod, wait for it to fully sync (by polling replication lag), and only then mark it ready for traffic. The entire process is automated and safe.

Version Upgrades

Self-managed:
Changing the image tag in a StatefulSet (e.g., mysql:5.7 → mysql:8.0) applies a rolling update that is not database-aware. Pods may restart in the wrong order, causing replication breaks or data format incompatibility. This is one of the most common causes of production database incidents.

Operator:
The Operator performs an ordered, validated upgrade:

Upgrades replicas first, one by one, verifying each before proceeding
Once all replicas are upgraded, performs a controlled failover
Upgrades the old primary last
Validates the entire cluster health before declaring success

TLS / Security

Self-managed:
You set up cert-manager, create Issuer and Certificate resources, mount the resulting secret into the StatefulSet as a volume, configure the database to use those certs, and write a renewal process before the certs expire (typically 90 days for Let's Encrypt).

Operator:
The Operator integrates with cert-manager automatically, issues TLS certs for all pods, mounts them correctly, and rotates them before expiry — all without manual intervention.

Monitoring

Self-managed:
You add a Prometheus exporter sidecar container to your StatefulSet (e.g., prom/mysqld-exporter), create a ServiceMonitor resource so Prometheus discovers it, and configure alerting rules for replication lag, disk usage, connection count, and query performance.

Operator:
Operators expose Prometheus metrics from day one. The exporter is baked in, the ServiceMonitor is created automatically, and many Operators ship default Grafana dashboards for their managed database.

8. When to Choose What

Choose Self-Managed When:

You are learning Kubernetes and want to understand how everything works under the hood
You are running a niche or custom database that has no Operator available
You have a very specific operational requirement that no Operator supports
You have a dedicated DBA or SRE team with deep Kubernetes expertise

Choose a Kubernetes Operator When:

You are running in production with real users and data
You want automated failover, backups, and upgrades
Your team is primarily developers, not infrastructure specialists
You need to run the same database setup across multiple clusters or environments
You want GitOps-friendly database management (declare state in Git, Operator reconciles)

Recommended Operators by Database

Database	Operator Options
MySQL / MariaDB	KubeDB, MySQL Operator (Oracle), Percona Operator
PostgreSQL	KubeDB, CloudNativePG, Crunchy Postgres Operator
MongoDB	KubeDB, MongoDB Community Operator, Percona Operator
Elasticsearch	KubeDB, Elastic Cloud on Kubernetes (ECK)
Redis	KubeDB, Redis Operator

9. Summary

Here is everything in one place:

Key Concepts

Concept	What it means
StatefulSet	Kubernetes object that gives pods stable names, stable DNS, and stable storage — essential for databases
PVC per pod	Each pod gets its own dedicated disk that survives pod restarts and rescheduling
Ordered startup	Pods start one at a time; next pod only starts when previous is Running + Ready
Primary pod	The only pod that accepts writes (`myapp-0`)
Replica pod	Read-only copy, synced from primary (`myapp-1`, `myapp-2`, ...)
Replication lag	The delay between a write on the primary and it appearing on a replica
Readiness probe	Kubernetes check that prevents traffic to a pod until it is ready (used to block reads until replica is synced)
Kubernetes Operator	An application that automates all operational database tasks, acting as your automated DBA
CRD	Custom Resource Definition — the YAML spec you write when using an Operator

The Golden Rules

Always write to the primary only — never send writes to a replica
For critical reads, read from the primary — replicas may lag
Use readiness probes — don't send traffic to a replica until it is fully synced
Use a PodDisruptionBudget — always keep at least 2 pods available
For production, use an Operator — manual database management does not scale

Architecture at a Glance

                    ┌─────────────┐
                    │  App / Client│
                    └──────┬──────┘
                           │
              ┌────────────┴────────────┐
              │ WRITE                   │ READ
              ▼                         ▼
   ┌─────────────────┐      ┌──────────────────┐
   │  myapp-0        │      │  myapp-1         │
   │  (Primary)      │──────│  (Replica)       │
   │  Accepts writes │ repl │  Read only       │
   └────────┬────────┘      └──────────────────┘
            │                ┌──────────────────┐
            │                │  myapp-2         │
            └────────────────│  (Replica)       │
                       repl  │  Read only       │
                             └──────────────────┘

   PVC-myapp-0          PVC-myapp-1          PVC-myapp-2
   (dedicated disk)     (dedicated disk)     (dedicated disk)

CCR + Kilo Gateway — Full Setup Guide

KALPESH — Sun, 26 Apr 2026 12:40:32 +0000

Use Claude Code (terminal + VS Code extension) free via Kilo Gateway's kilo-auto/free model, routed through Claude Code Router (CCR).

Prerequisites

Node.js installed (node -v to verify)
npm available
A terminal

Step 1 — Get Kilo Gateway API Key

Go to app.kilo.ai → sign up / sign in
Navigate to API Keys → generate a new key
Copy it — needed in Step 4

Step 2 — Install Claude Code

npm install -g @anthropic-ai/claude-code

Step 3 — Install CCR

npm install -g @musistudio/claude-code-router

Verify:

ccr --version

Step 4 — Create CCR config

CCR always reads from ~/.claude-code-router/config.json. No env var or flag overrides this.

mkdir -p ~/.claude-code-router

cat > ~/.claude-code-router/config.json << 'EOF'
{
  "LOG": true,
  "Providers": [
    {
      "name": "kilogateway",
      "api_base_url": "https://api.kilo.ai/api/gateway/chat/completions",
      "api_key": "YOUR_KILO_API_KEY_HERE",
      "models": ["kilo-auto/free"]
    }
  ],
  "Router": {
    "default": "kilogateway,kilo-auto/free",
    "background": "kilogateway,kilo-auto/free",
    "think": "kilogateway,kilo-auto/free",
    "longContext": "kilogateway,kilo-auto/free"
  }
}
EOF

Replace YOUR_KILO_API_KEY_HERE with your actual Kilo API key.

Step 5 — Configure VS Code extension (Optional)

The VS Code extension reads from ~/.claude/settings.json — this is separate from CCR's config.

mkdir -p ~/.claude

cat > ~/.claude/settings.json << 'EOF'
{
  "env": {
    "ANTHROPIC_BASE_URL": "http://127.0.0.1:3456",
    "ANTHROPIC_AUTH_TOKEN": "dummy"
  }
}
EOF

Then disable the login prompt in VS Code:

Open VS Code → Ctrl+, (Settings)
Search: claude code login
Check ✅ Disable Login Prompt

Or add directly to your VS Code settings.json (Ctrl+Shift+P → Open User Settings JSON):

"claudeCode.disableLoginPrompt": true

Step 6 — Start CCR manually (every time)

Every time you open a new terminal, run this once before using Claude Code:

ccr start

Expected output:

Loaded JSON config from: /home/<user>/.claude-code-router/config.json
Providers configured: kilogateway
Listening on 127.0.0.1:3456

ccr start runs as a background daemon — it will continue running after the command exits. You only need to start it once per session (until you reboot or stop it manually with ccr stop).

If you see "No providers configured" → your config.json is missing or has invalid JSON. Re-check Step 4.

When is `ccr start` required?

Scenario	Need to run `ccr start`?	Notes
Fresh terminal session (new shell)	✅ Yes	Before using `claude` or `ccr code`
After system reboot	✅ Yes	Daemon doesn't persist across reboots
After `ccr stop`	✅ Yes	Must restart to use Claude Code again
After `ccr restart`	❌ No	Already handled by the command
Same terminal after previous `ccr start`	❌ No	Daemon is still running
VS Code reload (after initial setup)	❌ No	Daemon persists independently
After editing `config.json`	⚠️ No	Use `ccr restart` instead

Step 7 — Use it

Terminal:

ccr code

VS Code extension:

Reload VS Code: Ctrl+Shift+P → Developer: Reload Window

The extension will skip the login screen and route through CCR → Kilo Gateway automatically.

Verify inside Claude Code:

/status

Look for:

API Base URL: http://127.0.0.1:3456

Step 8 — Persist CCR start via `.bashrc` (optional)

Don't want to run ccr start manually every session? Add it to ~/.bashrc so it auto-starts on every new terminal.

cat >> ~/.bashrc << 'EOF'

# Auto-start CCR and activate routing on new terminal
ccr start 2>/dev/null
eval "$(ccr activate)"
EOF

Apply immediately:

source ~/.bashrc

From now on, every new terminal will auto-start CCR and the claude command routes through CCR — no manual steps needed.

After editing config — always restart

ccr restart

Free models on Kilo Gateway

Model ID	Description
`kilo-auto/free`	Auto-picks best free model per session ✅ recommended
`openrouter/free`	Best available free model via OpenRouter
`x-ai/grok-code-fast-1:optimized:free`	xAI Grok, code-focused
`bytedance-seed/dola-seed-2.0-pro:free`	ByteDance Dola Seed 2.0 Pro

Switch model mid-session:

/model kilogateway,openrouter/free

Useful CCR commands

Command	What it does
`ccr start`	Start the router
`ccr stop`	Stop the router
`ccr restart`	Restart after config changes
`ccr status`	Check if running
`ccr code`	Launch Claude Code via router
`ccr activate`	Print env vars for shell integration
`ccr ui`	Open Web UI

File reference

File	Path
CCR config	`~/.claude-code-router/config.json`
Desktop copy (if symlinked)	`~/Desktop/ccr/config.json`
Claude Code / VS Code settings	`~/.claude/settings.json`
Shell config	`~/.bashrc`
CCR logs	`~/.claude-code-router/logs/`
Router endpoint	`http://127.0.0.1:3456`

Troubleshooting

VS Code shows login screen
→ Confirm ~/.claude/settings.json has ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN set, and claudeCode.disableLoginPrompt is true in VS Code settings. Then reload: Ctrl+Shift+P → Developer: Reload Window.

No providers configured on ccr start
→ Check ~/.claude-code-router/config.json exists and has valid JSON with a Providers array containing your Kilo API key.

Model not responding
→ Confirm your Kilo API key is valid at app.kilo.ai. Free tier is rate-limited to 200 req/hr per IP.

Config changes not taking effect
→ Always run ccr restart after editing config.json.

Auth conflict warning in Claude Code
→ Run claude /logout to clear stored Anthropic credentials, then use ccr code only.

Reference: CCR GitHub Link

🤖 AI Dev Tools

KALPESH — Fri, 24 Apr 2026 18:49:12 +0000

AI Dev Tools — Comparison

graphify vs claude-context vs axon vs Lynkr

🧠 The Simple 1-Line Summary

Repo	What it is in plain English
graphify	Researchers, architects, anyone onboarding to a new codebase, mixed content (not just code)
claude-context	Large codebases (millions of lines), teams that want fast semantic search without loading entire repos into context
axon	Engineers doing refactors, audits, or impact analysis; teams who want full structural understanding of their codebase locally
Lynkr	"Developers who want provider flexibility, want to use local/private models, or want to slash their AI API bills

📊 Side-by-Side Comparison Table

	graphify	claude-context	axon	Lynkr
🎯 Core job	Build knowledge graph from anything	Semantic search over codebase	Deep structural code analysis	Universal AI proxy + cost optimizer
🛠️ Type	AI skill (slash command)	MCP server	MCP server + CLI	HTTP proxy server
📦 Input	Code + docs + images + video	Code only	Code only	All AI tool requests
🔍 Search type	Graph topology (no embeddings)	Hybrid BM25 + dense vectors	BM25 + semantic + fuzzy (RRF)	N/A (it's a proxy)
🌐 External infra?	❌ None needed	✅ Zilliz Cloud / Milvus	❌ Fully local	❌ Self-hosted
💸 Saves money?	Indirectly (71x token reduction)	Yes (smart retrieval)	Yes (local, no API keys)	✅ 60–80% cost reduction
🔒 Privacy	✅ Local extraction	⚠️ Cloud DB by default	✅ 100% local	✅ Self-hosted
💥 Impact analysis	❌	❌	✅	❌
🧟 Dead code detection	❌	❌	✅	❌
🎥 Video/audio input	✅	❌	❌	❌
🗺️ Visual graph UI	✅ Interactive HTML	❌	✅ WebGL dashboard	❌
🔀 Provider switching	❌	❌	❌	✅ 12+ providers
⭐ GitHub stars	~28.6k 🔥	~8.7k	Smaller/newer	—
📝 Language	Python	TypeScript / Node.js	Python 3.11+	Node.js
📜 License	MIT	MIT	MIT	Apache 2.0
🏆 Best for	Researchers, architects, onboarding to new codebases, mixed content (code + docs + images + video)	Large codebases (millions of lines), teams wanting fast semantic search without loading full repos	Engineers doing refactors, audits, or impact analysis; teams needing full structural understanding locally	Devs wanting provider flexibility, local/private models (Ollama, Bedrock, Azure), or slashing API bills

🏁 Which One Should YOU Use?

If you want to…	Use
🗺️ Understand a new codebase fast, or map code + docs + images + videos	graphify
🔍 Search millions of lines of code instantly without loading everything	claude-context
🔬 Know exactly what breaks when you change something, find dead code, trace flows	axon
💰 Use Ollama / Bedrock / Azure instead of Anthropic, or cut API costs by 60–80%	Lynkr

⚡ Quick Install

# graphify
pip install graphifyy
# then type /graphify inside Claude Code, Cursor, Aider, Gemini CLI, etc.

# claude-context
# Follow setup at https://github.com/zilliztech/claude-context
# Requires Zilliz Cloud API key + Node.js < 24

# axon
pip install axoniq

# Lynkr
npm install -g lynkr && lynkr start

🔗 Links

Tool	GitHub	Install
graphify	safishamsi/graphify	`pip install graphifyy`
claude-context	zilliztech/claude-context	See repo
axon	harshkedia177/axon	`pip install axoniq`
Lynkr	Fast-Editor/Lynkr	`npm install -g lynkr`

💡 Pro Tip

These tools are not mutually exclusive!

Run Lynkr as your proxy → use graphify to map your codebase → plug in axon for impact analysis → use claude-context for fast retrieval.

All four together = the ultimate AI dev stack. 🚀

NVIDIA Driver Setup on Fedora 43 Workstation

KALPESH — Fri, 24 Apr 2026 18:16:03 +0000

RTX 3050 Mobile (Hybrid AMD + NVIDIA Laptop)

Reference Guide: Comprehensive-Wall28/Nvidia-Fedora-Guide

System Profile

Property	Value
OS	Fedora 43 Workstation
GPU (dGPU)	NVIDIA GeForce RTX 3050 Mobile
GPU (iGPU)	AMD Radeon Vega (Renoir)
LUKS Encryption	❌ None
Secure Boot	❌ Disabled
Driver Installed	580.142

Why the Freeze Happened

The freeze + screen dimming during use was caused by the default Nouveau open-source driver, which has poor power management on modern NVIDIA GPUs — especially on hybrid AMD+NVIDIA laptops.

The Fix: Install Proprietary NVIDIA Drivers

Step 1 — Update your system

sudo dnf update

Step 2 — Enable RPM Fusion (provides NVIDIA drivers)

sudo dnf install \
  https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \
  https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

Step 3 — Identify your GPU

lspci | grep -iE 'VGA|3D|nvidia'

Step 4 — Install the driver (RTX 3050 = current driver)

sudo dnf install akmod-nvidia
sudo dnf install xorg-x11-drv-nvidia-cuda

Step 5 — Watch the kernel module build in real time

⏳ This takes 5–10 minutes. Do NOT reboot until verified.

# Watch live build logs
journalctl --follow --grep=akmod

Then in a separate terminal, keep checking every 2–3 minutes:

modinfo -F version nvidia

✅ When you see a version number like 580.142 — the build is done. Safe to reboot.

❌ If it says Module nvidia not found — it's still building. Wait more.

Step 6 — Reboot

systemctl reboot

Step 7 — Verify after reboot

nvidia-smi

Expected output:

NVIDIA-SMI 580.142    Driver Version: 580.142    CUDA Version: 13.0
GeForce RTX 3050 ...

⚠️ Hybrid GPU Note (AMD + NVIDIA Laptops)

Since this is a hybrid laptop (AMD iGPU + NVIDIA dGPU), there are two separate components managing your GPUs. Understanding both is important.

`nvidia-powerd` — Power Manager

Its job is purely power management of the NVIDIA GPU:

Turns the NVIDIA GPU on/off automatically based on demand
Prevents the GPU from staying on when idle (saves battery)
Manages dynamic power states — P8 = idle/low power, P0 = full performance (you saw P8 in your nvidia-smi output, which is correct at idle)

💡 Think of it as the power switch manager — it decides when to give the GPU electricity.

Status in your case: ✅ Auto-installed and running. Verify with:

systemctl status nvidia-powerd

PRIME Offload — Rendering Router

Its job is deciding which GPU renders what:

Routes specific apps to the NVIDIA GPU for rendering
The __NV_PRIME_RENDER_OFFLOAD=1 prefix command is PRIME
Tells the system "run THIS specific app on NVIDIA"

💡 Think of it as the traffic director — it decides what work gets sent to which GPU.

On Linux, this is manual per-app — unlike Windows (NVIDIA Optimus) which switches automatically.

How They Work Together

You run: nvidia-run glmark2
         ↓
PRIME says → "send this to NVIDIA GPU"
         ↓
nvidia-powerd says → "NVIDIA GPU needed, power it ON"
         ↓
RTX 3050 renders glmark2
         ↓
App closes → nvidia-powerd powers NVIDIA back OFF

In short:

nvidia-powerd = controls when the GPU gets power
PRIME = controls what gets sent to the GPU

Both are needed and both are active on your system ✅

How Hybrid GPU works (normal behavior)

GPU	Role
AMD Vega (iGPU)	Drives the display, runs desktop + most apps — always active
NVIDIA RTX 3050 (dGPU)	Performance GPU — only activates when explicitly asked via PRIME

Apps running on AMD by default (like glmark2) is correct and expected — it saves battery.

Run any app on NVIDIA explicitly (PRIME)

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia <your-app>

Example:

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glmark2

Or add a convenient alias so you don't type that every time:

echo 'alias nvidia-run="__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia"' >> ~/.bashrc
source ~/.bashrc

# Then simply use:
nvidia-run glmark2
nvidia-run steam

Watch NVIDIA GPU usage live while app runs

watch -n 1 nvidia-smi

You should see GPU utilization spike on the RTX 3050 when PRIME offload is active.

Check which GPU is currently rendering

glxinfo | grep "OpenGL renderer"

When WOULD you need extra manual steps?

Only if you experience any of the following after the driver install:

1. Still freezing during use
The NVIDIA module may not be loading at boot. Force-load it:

echo 'options nvidia NVreg_DynamicPowerManagement=0x02' | sudo tee /etc/modprobe.d/nvidia-pm.conf
sudo dracut --force
systemctl reboot

2. Battery draining abnormally fast even at idle
nvidia-powerd may not be running. Re-enable it:

sudo systemctl enable --now nvidia-powerd
systemctl status nvidia-powerd

3. systemctl status nvidia-powerd shows inactive or failed
Reinstall the power management package:

sudo dnf reinstall xorg-x11-drv-nvidia-power
sudo systemctl enable --now nvidia-powerd
systemctl reboot

Quick Diagnostic Commands

# Check Fedora variant
cat /etc/os-release

# Check LUKS encryption
lsblk -o NAME,TYPE,FSTYPE,MOUNTPOINT

# Check Secure Boot status
mokutil --sb-state

# Check GPU(s)
lspci | grep -iE 'VGA|3D|nvidia'

# Check driver version (before reboot)
modinfo -F version nvidia

# Check driver after reboot
nvidia-smi

# Check nvidia-powerd status
systemctl status nvidia-powerd

# Watch akmod build live
journalctl --follow --grep=akmod

Common Problems

Problem	Cause	Fix
`Module nvidia not found`	Still building	Wait 5–10 min, retry `modinfo`
Black screen after reboot	Wrong driver version	Boot to TTY (`CTRL+ALT+F2`), run `sudo dnf remove "nvidia"`
`nvidia-smi: command not found`	CUDA not installed	`sudo dnf install xorg-x11-drv-nvidia-cuda`
"Nvidia modules failed to load"	Secure Boot issue	Disable Secure Boot or redo MOK enrollment
Still freezing after install	Hybrid GPU power issue	Check `nvidia-powerd` status, consider PRIME config

Guide based on: github.com/Comprehensive-Wall28/Nvidia-Fedora-Guide

End-To-End DevOps + AIOps Project- 2

KALPESH — Mon, 20 Apr 2026 19:03:55 +0000

The Application: A Microservices E-Commerce App

The project is built around a real-world microservices-based e-commerce application — seven independent services, each containerized and independently deployable.

  E-Commerce Microservices
  ┌────────────────────────────────────────┐
  │   - Frontend (UI)                      │
  │   - Cart Service                       │
  │   - Orders Service                     │
  │   - Checkout Service                   │
  │   - Payments Service                   │
  │   - Product Catalog Service            │
  │   - Recommendation Service             │
  └────────────────────────────────────────┘

Each service is isolated, owns its own responsibility, and communicates over well-defined APIs — mirroring how teams actually build and ship software at scale.

The Full Architecture: End-to-End Flow

  Developer pushes code
          ↓
  GitHub (GitOps — Source of Truth)
          ↓
  CI/CD Pipeline (GitHub Actions)
  ┌──────────────────────────────────────┐
  │  - Run tests                         │
  │  - Build Docker images               │
  │  - Push to container registry        │
  │  - Update Kubernetes manifests       │
  └──────────────────────────────────────┘
          ↓
  Argo CD (GitOps Continuous Delivery)
  Watches Git repo → syncs cluster state
          ↓
  AWS EKS Cluster (Terraform-provisioned)
  ┌──────────────────────────────────────┐
  │  Microservices on Kubernetes         │
  │  - Cart      - Orders                │
  │  - Checkout  - Payments              │
  │  - Catalog   - Frontend              │
  │  - Recommendations                   │
  └──────────────────────────────────────┘
          ↓
  Observability Stack
  ┌──────────────────────────────────────┐
  │  Prometheus  → Metrics collection    │
  │  Grafana     → Dashboards & alerts   │
  │  Loki        → Log aggregation       │
  └──────────────────────────────────────┘
          ↓
  AIOps Layer
  ┌──────────────────────────────────────┐
  │  - Anomaly Detection                 │
  │  - Intelligent Log Analysis          │
  │  - Auto-remediation                  │
  │  - Incident Response Automation      │
  └──────────────────────────────────────┘

Layer 1: Local Development with Docker Compose

All seven microservices run locally using Docker Compose — spin up the full app on any laptop with a single command, no cloud credentials required. This validates the application before any infrastructure costs are incurred.

Layer 2: Infrastructure as Code with Terraform

AWS infrastructure is never clicked together manually. Terraform declares it as code — repeatable, version-controlled, and auditable.

  Terraform provisions on AWS
  ┌──────────────────────────────────────┐
  │  EKS Cluster                         │
  │  VPC + Subnets + Security Groups     │
  │  IAM Roles & Policies                │
  │  Node Groups (EC2 worker nodes)      │
  │  Load Balancers                      │
  └──────────────────────────────────────┘

Layer 3: CI/CD Pipeline with GitHub Actions

Every code push triggers an automated pipeline:

  Code pushed to GitHub
          ↓
  ┌──────────────────────────────────────┐
  │  1. Run unit & integration tests     │
  │  2. Build Docker image               │
  │  3. Push image to container registry │
  │  4. Update image tag in K8s manifests│
  │  5. Commit updated manifests to Git  │
  └──────────────────────────────────────┘
          ↓
  Argo CD detects the change

Layer 4: GitOps with Argo CD

Git is the single source of truth. Argo CD continuously watches the repo and auto-syncs the live cluster to match the declared state — self-healing, auditable, and rollbacks are just a git revert away.

  Git Repository (Desired State)
          ↓  Argo CD watches for drift
  AWS EKS Cluster (Actual State)
          ↓
  Drift detected → Auto-sync to reconcile

Layer 5: Kubernetes on AWS EKS

Amazon EKS manages the Kubernetes control plane so the team focuses on workloads, not cluster maintenance.

  AWS EKS Cluster
  ┌──────────────────────────────────────────────┐
  │  Deployments   → Run & manage pods           │
  │  Services      → Internal/external routing   │
  │  Ingress       → External traffic entry      │
  │  ConfigMaps    → App configuration           │
  │  Secrets       → Sensitive credentials       │
  │  HPA           → Horizontal Pod Autoscaling  │
  └──────────────────────────────────────────────┘

Layer 6: Observability — Prometheus, Grafana & Loki

  Prometheus   → Scrapes & stores metrics (CPU, memory, req/s, errors)
       ↓
  Grafana      → Visualizes metrics (dashboards + alerting)
       ↓
  Loki         → Aggregates logs from all microservices

Together they provide full visibility into application health, resource usage, error rates, and logs — all in one place.

Layer 7: AIOps — Intelligent Operations

AIOps moves beyond passive monitoring toward autonomous operations using ML and LLMs.

  Raw Telemetry (Metrics + Logs + Traces)
          ↓
  AIOps Layer
  ┌──────────────────────────────────────────────┐
  │  Anomaly Detection                           │
  │  → Flags issues before users are impacted    │
  │                                              │
  │  Intelligent Log Analysis                    │
  │  → LLMs parse & summarize logs               │
  │  → Pinpoints root cause faster               │
  │                                              │
  │  Auto-Remediation                            │
  │  → Auto-scales pods, restarts containers     │
  │  → Triggers rollbacks on degraded deploys    │
  │                                              │
  │  Incident Response Automation                │
  │  → Notifies on-call with context, not noise  │
  └──────────────────────────────────────────────┘

Tools & Technologies

Category	Tool
Containerization	Docker, Docker Compose
Orchestration	Kubernetes (AWS EKS)
Infrastructure as Code	Terraform
CI/CD	GitHub Actions
GitOps	Argo CD
Metrics	Prometheus
Dashboards & Alerts	Grafana
Log Aggregation	Loki
Cloud Provider	AWS
AIOps	ML anomaly detection + LLM log analysis

End-To-End DevOps + AIOps Project- 1

KALPESH — Mon, 20 Apr 2026 18:27:33 +0000

Why System Design Matters for DevOps

1. Distributed Systems

A distributed system splits workloads across multiple machines. Instead of one powerful server doing everything, many smaller services collaborate — each handling a piece of the work.

Why it matters: No single point of failure. If one node goes down, others keep running. This is the foundation of all modern cloud architecture.

2. Monolith vs Microservices

         MONOLITH                        MICROSERVICES
  ┌──────────────────────┐        ┌────────┐  ┌────────┐
  │                      │        │  Cart  │  │ Orders │
  │  UI + Auth + Cart +  │        └────┬───┘  └────┬───┘
  │  Orders + Payments   │             │            │
  │  + Notifications...  │        ┌────┴───┐  ┌────┴──────┐
  │                      │        │Payments│  │  Notifs   │
  └──────────────────────┘        └────────┘  └───────────┘
    One giant deployable             Each service deploys
    unit — scale all or nothing      & scales independently

	Monolith	Microservices
Deploy	One unit	Independent services
Scale	Whole app	Per service
Failure	One bug = full outage	Isolated failures
Best for	Small teams, MVPs	Large, evolving systems

3. API Communication

Services talk to each other via APIs. Three key patterns:

REST — Stateless HTTP calls, great for client-server communication
gRPC — High-performance, binary protocol ideal for internal service-to-service calls
Event-driven (Kafka/SQS) — Async messaging that decouples services and absorbs traffic spikes > Rule of thumb: Use REST for external APIs, gRPC for internal performance-critical calls, and events for async workflows.

4. Service Discovery

  ┌─────────────┐     "Where is cart-service?"     ┌──────────────────┐
  │  Checkout   │ ────────────────────────────────► │  Service Registry│
  │  Service    │ ◄────────────────────────────────  │  (CoreDNS)       │
  └─────────────┘     "cart-service:3000"           └──────────────────┘
         │                                                    ▲
         │  connects to                              registers │
         ▼                                                    │
  ┌─────────────┐                                   ┌──────────────────┐
  │    Cart     │ ─────────────────────────────────► │  cart-service    │
  │   Service   │                                   │  pod (dynamic IP)│
  └─────────────┘                                   └──────────────────┘

When services scale dynamically, hardcoded IPs break. Service discovery lets services find each other automatically.

In Kubernetes, this happens natively via CoreDNS — every service gets a stable DNS name regardless of how many pods are running or where they live.

5. Load Balancing

          Incoming Traffic
               │
               ▼
   ┌───────────────────────┐
   │      Load Balancer    │
   │  (AWS ALB / Ingress)  │
   └───────┬───────┬───────┘
           │       │       │
           ▼       ▼       ▼
      ┌────────┐ ┌────────┐ ┌────────┐
      │  Pod 1 │ │  Pod 2 │ │  Pod 3 │
      └────────┘ └────────┘ └────────┘
       Layer 4: routes by IP/port
       Layer 7: routes by path/headers

Load balancers distribute traffic across instances so no single server gets overwhelmed.

Layer 4 — Routes by IP/port (fast, lower overhead)
Layer 7 — Routes by HTTP path, headers, or cookies (smart, flexible) On AWS EKS, the AWS Load Balancer Controller + Kubernetes Ingress handles this automatically.

6. High Availability

HA means the system stays up even when parts of it fail. Key techniques:

Multi-AZ deployments — Spread workloads across Availability Zones
Replication — Keep multiple copies of data and services
Circuit breakers — Stop cascading failures between services

- Kubernetes self-healing — Failed pods restart automatically

7. Autoscaling

  CPU: 80% 🔺 (threshold: 70%)
         │
         ▼
  ┌─────────────┐     scale out      ┌──────────────────────────┐
  │     HPA     │ ─────────────────► │  Pod 1 │ Pod 2 │ Pod 3   │
  │ (autoscaler)│                    │        + Pod 4 + Pod 5   │
  └─────────────┘                    └──────────────────────────┘

  CPU: 20% 🔻 (below threshold)
         │
         ▼
  ┌─────────────┐     scale in       ┌────────────────┐
  │     HPA     │ ─────────────────► │ Pod 1 │ Pod 2  │
  └─────────────┘                    └────────────────┘

Manual scaling doesn't work in production. Kubernetes offers multiple autoscaling tools:

Tool	What it does
HPA	Scales pod count based on CPU/memory
VPA	Adjusts resource requests per pod
KEDA	Event-driven scaling (e.g., queue depth)
Cluster Autoscaler	Adds/removes nodes from the cluster

8. Reliability with Kubernetes

Kubernetes has built-in reliability primitives every DevOps engineer should know:

Liveness probes — Restart containers that hang or crash
Readiness probes — Remove unhealthy pods from the load balancer
Pod Disruption Budgets — Guarantee minimum replicas during rolling updates

- Resource quotas — Prevent one service from starving others

9. Security by Design

Security isn't an afterthought — it's architecture. Core principles:

Least privilege — IAM roles + Kubernetes RBAC limit what each service can do
Network policies — Restrict pod-to-pod traffic
Secrets management — AWS Secrets Manager or Vault (never hardcode credentials)
Image scanning — Tools like Trivy scan containers before they deploy

- mTLS — Encrypt all service-to-service traffic (via Istio or a service mesh)

10. Observability

  Your System
  ┌──────────────────────────────────────────────────┐
  │  Microservice A  ──►  Microservice B  ──►  DB    │
  └──────────┬──────────────────┬────────────────────┘
             │                  │
      ┌──────▼──────┐   ┌───────▼────────┐   ┌──────────────┐
      │    LOGS     │   │    METRICS     │   │   TRACES     │
      │  (what      │   │  (how much /   │   │  (where did  │
      │  happened)  │   │   how fast)    │   │  it go?)     │
      │  Loki /     │   │  Prometheus /  │   │  Jaeger /    │
      │  CloudWatch │   │  Grafana       │   │  X-Ray       │
      └─────────────┘   └────────────────┘   └──────────────┘
                  └──────────────┬──────────────┘
                                 ▼
                         AIOps Dashboard
                    (Anomaly Detection + Alerts)

You can't fix what you can't see. Observability is built on three pillars:

Pillar	Tools	Purpose
Logs	Fluentd, CloudWatch, Loki	Detailed event records
Metrics	Prometheus, Grafana	Time-series measurements
Traces	Jaeger, AWS X-Ray	Request flow across services

11. Deployment Strategies

  ROLLING UPDATE          BLUE / GREEN              CANARY
  ┌───┬───┬───┐          ┌───────────┐           ┌────────────┐
  │v1 │v1 │v1 │  step 1  │  BLUE(v1) │◄─ 100%    │   v1       │◄─ 90%
  └───┴───┴───┘   ──►    └───────────┘            └────────────┘
  ┌───┬───┬───┐          ┌───────────┐           ┌────────────┐
  │v2 │v1 │v1 │  step 2  │ GREEN(v2) │◄─  0%     │   v2       │◄─ 10%
  └───┴───┴───┘          └───────────┘  switch!   └────────────┘
  ┌───┬───┬───┐               ↕                    gradually shift
  │v2 │v2 │v2 │  done    flip traffic              to 100% if ok
  └───┴───┴───┘

Deploying safely means choosing the right strategy:

Rolling update — Gradually replace old pods with new ones (Kubernetes default)
Blue/Green — Two identical environments; switch traffic instantly with zero downtime
Canary — Route a small % of traffic to the new version first, then roll out fully

- Feature flags — Toggle features without redeploying

12. GitOps

  Developer
      │
      │  git push / pull request
      ▼
  ┌──────────────┐
  │   Git Repo   │  ◄── single source of truth
  │  (GitHub)    │
  └──────┬───────┘
         │  watches for changes
         ▼
  ┌──────────────┐
  │   ArgoCD /   │  detects drift between
  │    Flux      │  Git state ↔ cluster state
  └──────┬───────┘
         │  syncs automatically
         ▼
  ┌──────────────────────┐
  │   Kubernetes Cluster │
  │   (AWS EKS)          │
  └──────────────────────┘
  Rollback = revert a commit ↩

GitOps makes Git the single source of truth for infrastructure and application state.

All changes go through pull requests — reviewed, audited, version-controlled
An operator like ArgoCD or Flux continuously syncs the cluster to match what's in Git
Rollback = revert a commit Benefits: Full audit trail, consistent environments, and deployments that are always reproducible.

The Big Picture: How It All Connects

Developer pushes code
        ↓
  GitHub (GitOps)
        ↓
  CI/CD Pipeline
        ↓
  AWS EKS Cluster
  ┌─────────────────────────────┐
  │  Microservices (Kubernetes) │
  │  - Cart  - Orders           │
  │  - Checkout  - Payments     │
  └─────────────────────────────┘
        ↓
  Observability Stack
  (Prometheus + Grafana + Loki)
        ↓
  AIOps Layer
  (Anomaly Detection + Auto-remediation)

Key Takeaways

Distributed systems and microservices enable independent scaling and fault isolation
Kubernetes provides built-in resilience, self-healing, and safe deployments
Security and observability must be designed in — not added later
GitOps brings auditability and consistency to infrastructure changes
AIOps closes the loop: observability data drives intelligent automation

Networking

KALPESH — Thu, 05 Mar 2026 20:08:09 +0000

🖧 Subnet

Subnet is a smaller chunk of a bigger network, created using a subnet mask or CIDR

(e.g., /27) to split IPs into network and host parts.

It improves efficiency, security, and traffic control
CIDR /27 means you're borrowing 5 bits from the host portion (since 32 - 27 = 5).
That gives you 2⁵ = 32 IP addresses in total.
Out of those, 30 are usable IPs (excluding 1 for network address and 1 for broadcast).

DNS resolver

It stores information about domain names and their corresponding IP addresses in structured records

Handshaking

TCP make 3 ways handshake in order to establish connection.

OSI Layer

Layer 7, 6 & 5 happens in local level. e.g.- Browser

AWS VPC

Essential Linux Commands- 2

KALPESH — Thu, 05 Mar 2026 20:05:02 +0000

Shortcuts:

Reverse Search

Ctrl + R then type the command you want to search then Tab

Commands:

Info of Command

# brief description of command
whatis cat

Real time process info

# Real time process
top

# Wrapper: Graphical Process
htop

# System stat for performance check
vmstat

# Amount of CPU available
nproc

List Process & Hierarchy

# List Process not a Real time
ps

# Detail all process list
ps aux

# No of process(Line No)
ps aux | ln

# Only shows no lines
ps aux | wc -l

# kill process
kill <PID>

# Force process delete
kill -9 <PID>

# Thread dumnp
kill -3 <PID>

# Stop process
kill -STOP <PID>

# Resume Stop process
kill -CONT <PID>

# Prioritize Process (-n [1-20], lower no means high prioritize)
renice -n 10 -p <PID>

# Process Hierarchy
pstree -p

# Port used by Process
lsof -i :8085

Inspect Network Connection

# Get active ports in Use
netstat -tuln

# Network Interface Info
ifconfig

# Network Troubleshoot (Can WireShark tool)
# enX0: Network Interface
sudo tcpdump -i enX0 port 80

# Test Connectivity
ping google.com

# Tarce the path packet to reach destination
traceroute google.com

Disk Space, Size & Memory

# Check Disk Space
df -h

# Size of dir or file
# opt: directory
du -sh opt

# Memory (RAM)
free -h

# List Blob(All type of formats- Raw state) attach to Instance
lsblk

# Format the Blob Storage to linux supperted file system (ext4)
mkfs -t ext4 /dev/xvdf

# Mount it in order to use
mount /dev/xvdf mnt/demo-volume/

Services

Systemd manages services

# Logs of Services
journalctl

# Particular Service
journalctl -u nginx

# Logs of services from Last Boot
journalctl -b

Logs Filter

# Last 10 line of logs
tail -n 10 /var/log/auth.log

# First 10 line of logs
head -n 10 /var/log/auth.log

Alias

alias detail_list='ls -la'

# Want to persist the Alias, add in '~/.bashrc'

Symbolic Link

Soft Link (Like Windows Shortcuts)- It can be broken

# ln: link
# -s: flag for soft link 
# myfile: file you want to crate soft link
# slink: name of soft link file created
ln -s myfile slink

Hard Link (Actual Copy)- Doesn’t break the other one

# ln: link
# myfile: file you want to crate soft link
# hlink: name of hard link file created
ln myfile hlink

Users

# Full setup of User
adduser tim

# Just Add user
useradd tim

# Delete user
userdel tim

# Login into User with sudo privilege
sudo su - tim

# Root user indication
"#"

# Standard user indication
"$"

Groups & Ownership

==================================
# Before you shoud be in root user
==================================
# Create Group
groupadd devops

# Add User into Group- (Adding user:tim into group:devops)
usermod -aG devops tim

# Remove user tim from group devops
deluser tim devops 

# To see how manu group user belong to
id tim

# Change ownership of dir
# change owenership of dir to nexus:nexus(user:group) -R recursive
chown -R nexus:nexus <dir-name>

SSH Server

# ssh server config- sshd
ls /etc/ssh/sshd_config.d/<50-cloud-init.conf>

File Management

# Overwriting the existing content
echo "Hello" > file.txt

# Appending the content
echo "Hello" >> file.txt

Services

# Re-read all service configuration files from /etc/systemd/system/
sudo systemctl daemon-reload

# Registers your service to start automatically at boot time
sudo systemctl enable myservice

# Starts your service right now
sudo systemctl start myservice

# Check status of your service
sudo systemctl status myservice

Refer Linux Journey

Linux Journey

Python Overview

KALPESH — Thu, 05 Mar 2026 20:03:09 +0000

Python Modules

A file with reusable Python code (functions, classes, variables).
Example: custom .py files or sys, math module

Python Package

A folder with related modules, including an __init__.py file.
Example: numpy or custom module directories.
pip is a tool in Python to install & manage Python packages or libraries from the Python Package Index (PyPI).

Python Virtual Environment

Isolated environment for project-specific dependencies.
Create: python -m venv <env_name>
Activate: source <env_name>/bin/activate

Command Line Args

Python in build sys module

import sys # python inbuild sys module, which is used for command line args

num1 = float(sys.argv[1])

Environment Variables

Env vars used for sensitive data, which we can’t hardcoded:

API keys
passwords
tokens
certificates

Declare Env vars in terminal:

export password=”pass@123”

Code:

import os

pass = os.getenv("password")

File Operation of Windows

Open: open() with modes (r, w, etc.), e.g., open("file.txt", "r").
Read: Use read(), readline(), or readlines() to fetch content.
Write: Use write() or writelines() in modes like w or a.
Close: Use close() or with for auto-closing.

Module

Requests

The requests module in Python simplifies HTTP requests to interact with web servers.

Purpose: Send HTTP methods (GET, POST, etc.) and handle responses via API.
Features: Manage headers, cookies, auth, and work with JSON or text.
Install: pip install requests

import requests  
response = requests.get("https://example.com")  
print(response.status_code)  # Status code  
print(response.text)         # Response body

Boto3

AWS SDK for Python to interact programmatically with AWS services.
Ideal for automating AWS workflows and managing resources efficiently
Building serverless applications with services like Lambda and DynamoDB.

import boto3

# Initialize S3 client
s3 = boto3.client('s3')

# Upload a file to S3
s3.upload_file('local_file.txt', 'my-bucket', 'remote_file.txt')

Flask

It’s a lightweight web framework in Python used to build web applications with added functionality.

Decorators:

It’s a special function in Python used to modify the behaviour of another function.
It written above function with @ symbol.

from flask import Flask

app = Flask(__name__)

@app.route("/greet")  # Flask decorator that connects a URL('/greet') route to a function
def greet():
    return "Greetings from Flask!"

if __name__ == "__main__":
    app.run()

Constructor

It’s special method in a class that runs automatically when you create an object.

__init__()

# __name__ special build-in variable
__name__ = __main__ # Runs code only when file is executed directly

Pytest

Tool/framework for testing
Basic Structure

my_project/
│
├── app.py              # Your actual code
├── test_app.py         # Your test file (MUST start with "test_")

pytest -v — Verbose output
pytest -s — Show print statements

Assert Statements

They check if a condition is True. If True, test passes ✅

Fixtures

A fixture in pytest is a reusable piece of setup code that prepares data or resources for your tests.

Conftest

Special file where you define fixtures that are automatically available to ALL test files in that directory and sub directories. No imports needed!

Mocking

Create fake versions of external dependencies (APIs, databases, files) so tests run fast, reliably.

Terraform Scenario Qus

KALPESH — Thu, 05 Mar 2026 19:58:48 +0000

Scenario 1 : Terraform Migration of AWS Resource

Step 1: Import Resource Configuration

Import Block: Add the import block in main.tf to specify the EC2 resource ID and target resource name:
```
import {
  id = "instance ID"
  to = aws_instance.example
}
```
Generate Resource Configuration: Run the command to fetch the resource configuration:
```
terraform plan -generate-config-out=generated_resources.tf
```
Copy Code: Copy the generated resource block from generated_resources.tf to main.tf.
Cleanup: Delete generated_resources.tf as it is no longer needed.

Step 2: Import Resource into Statefile

Run the import command to bring the resource into Terraform state:
```
terraform import aws_instance.example <instance ID>
```
Result: The resource data is imported into the Terraform state file, ensuring the state matches the actual resource.

Scenario 2 : Terraform Drift Detection

Terraform doesn't automatically detect manual changes made directly in the AWS cloud.

Solution 1: Use a Cron Job to Refresh Terraform State

Set up a cron job to periodically run terraform refresh, which updates the Terraform state with the latest changes in the cloud.

Solution 2: Use Audit Logs or Event Notifications

Audit Logs: Enable AWS CloudTrail to log and monitor all changes.
AWS Lambda/Notification: Use AWS Lambda functions or event notifications to alert or trigger Terraform updates when manual changes happen by IAM users.

AWS Strategies

KALPESH — Thu, 05 Mar 2026 19:54:23 +0000

AWS Cloud Migration

1- Preparation stage

Verify if the application follows a microservices architecture.
If not, refactor the monolithic application into a microservices architecture.

2- Planning Stage

Break down the microservices migration into phases based on their criticality for cloud migration.

Most used Migration Strategies:

Rehost (Lift and Shift): Move applications to the cloud with minimal changes, focusing on quick deployment.
Replatform: Optimize specific components for cloud use without significant code changes.
Refactor/Rearchitect: Redesign the application, e.g., transforming a monolithic architecture into microservices.
Relocate: Shift services, e.g., from Kubernetes to OpenShift or EKS.

Least Used Migration Strategies:

Retain: Keep certain applications on-premise.
Retire: Shutdown unused applications.
Repurchase: Replace with a cloud-based solution.

3- Migrate Stage

Conducted in phases, working concurrently with the monitor stage.

4- Monitor Stage

Runs alongside the migrate stage to ensure smooth transitions.

5- Optimize Stage

Enhance efficiency, refine processes, and implement improvements for better performance.

AWS Cost Optimization

1. AWS Resource Groups & Tag Editor

Use Tag Editor to track resources and identify those incurring charges.

2. Set Budgets in AWS Billing

Create Budgets to get alerts when spending hits thresholds.
Schedule budget reports for proactive cost management.

3. Reduce Console Access/UI, Use IaC

Use Terraform or similar tools for automated resource management.

4. Apply Least Privilege Access

Limit permissions to essential roles to prevent unnecessary resource usage.

5. Automate with AWS Lambda

Use a Lambda function triggered by cron jobs, AWS CLI, or CloudWatch events.
Write Python scripts to identify stale or unused resources based on specific conditions.
Define actions for stale resources:

1. Notify: Use SNS (Simple Notification Service) to send an email alert about stale resources.



Cleanup: Automatically delete stale resources using APIs.

Request AWS Support for Unexpected Charges

Contact AWS Support for waivers on unexpected charges.

AWS ELB (Elastic Load Balancer)

KALPESH — Thu, 05 Mar 2026 19:51:17 +0000

ELB distributes incoming network traffic across multiple servers.
Primary purpose:
- Optimize resource use
- Enhance application performance
- Ensure high availability by preventing any single server from becoming overwhelmed with too many requests.

OSI Model

3 Types of ELB

ELB Type	Protocols	Use Cases	Cost Aspect
Application Load Balancer (ALB) - Layer 7	HTTP/HTTPS	Web apps, microservices, content & path-based routing	Costlier
Network Load Balancer (NLB) - Layer 4	TCP/UDP/TLS	Low-latency apps, millions of requests/sec, static IP	Moderate
Gateway Load Balancer (GWLB) - Layer 3	IP Protocol	Network appliances, firewalls, security monitoring	Less costlier than ALB and NLB

Load Balancer vs API Gateway vs Kubernetes Ingress (Reverse Proxy)

Feature	Load Balancer	API Gateway	Kubernetes Ingress (Reverse Proxy)
Primary Purpose	Distributes traffic for high availability	Manages, secures, and scales APIs	Routes external traffic to internal services in Kubernetes
Layer of Operation	Layer 4 or Layer 7	Layer 7	Layer 7
Key Features	Traffic distribution, fault tolerance	Authentication, rate limiting, monitoring	URL rewriting, SSL termination, routing
Use Case	High-traffic web applications (e.g., Amazon)	Microservices architectures (e.g., Netflix)	Containerized apps in Kubernetes (e.g., GitHub)

API Gateway

API Gateway vs Load Balancer

Example: E-commerce Platform

Load Balancer: Distributes traffic across multiple API Gateway instances.
API Gateway: Manages and secures APIs for product catalog, user authentication, and payment processing.
Kubernetes Ingress: Routes traffic to backend services like inventory management, order processing, and recommendation engines.
Backend Services: Reside in a private subnet and handle the core business logic.

Feel free to share and spread the knowledge! 🌟😊 Enjoy Learning! 😊