Forem: Ashish

Git & GitHub for DevOps - The Complete Hands-On Guide (Week 4)

Ashish — Sun, 14 Dec 2025 13:17:35 +0000

Linkedin Profile: https://www.linkedin.com/in/ashish360/

Git and GitHub are the backbone of modern DevOps workflows. Every CI/CD pipeline, Infrastructure-as-Code repo, GitOps setup, and production deployment relies on Git for version control and collaboration.
This guide is written from a DevOps engineer’s perspective, not just “how Git works”, but how Git is actually used in real systems.

Table of Contents
What is Version Control & Why DevOps Needs It
Git vs GitHub (Clear Difference)
Git Core Concepts (Repo, Commit, Branch, Merge)
Git Installation & Configuration
Git Repository Lifecycle
Working Directory, Staging Area & Repository
Essential Git Commands (with definitions & examples)
Branching Strategies for DevOps
Merging & Conflict Resolution
Staging & Stashing
Undoing Changes (Reset, Revert, Amend, Reflog)
Tags & Releases
Git Ignore, Attributes & Cleanup
GitHub Basics (Remote, SSH, Push, Pull)
Forks, Pull Requests & Open-Source Flow
Advanced Git (Rebase, Cherry-Pick, Hooks, LFS)
Git in CI/CD & DevOps Pipelines
Best Practices for DevOps Engineers

What is Version Control?
Definition:
Version control is a system that tracks changes in files over time, allowing you to review history, collaborate safely, and roll back when needed.
Why DevOps Needs Git
Infrastructure code must be traceable
CI/CD pipelines rely on Git events
Rollbacks must be fast and safe
Teams work in parallel without conflicts
Git vs GitHub
Git
GitHub
Distributed version control system
Hosting platform for Git repositories
Runs locally
Cloud-hosted
Tracks changes
Enables collaboration
CLI-based
Web UI + APIs
Created in 2005
Launched in 2008

Important:
👉 Git works without internet
👉 GitHub is only used for push, pull, PRs, CI/CD

Core Git Concepts Repository A project folder tracked by Git. Commit A snapshot of changes with a unique hash. Branch An independent line of development. Merge Combining changes from one branch into another. 3.1 Understanding the Git Three-Stage Workflow (Easy Explanation) This diagram shows how files move step-by-step inside Git, from your computer to permanent history.

A. Working Directory

By Ashish - Learn-in-Public DevOps Journey (Week 4)

What is Version Control?
Definition:
Version control is a system that tracks changes in files over time, allowing you to review history, collaborate safely, and roll back when needed.
Why DevOps Needs Git
Infrastructure code must be traceable
CI/CD pipelines rely on Git events
Rollbacks must be fast and safe
Teams work in parallel without conflicts
Git vs GitHub
Git
GitHub
Distributed version control system
Hosting platform for Git repositories
Runs locally
Cloud-hosted
Tracks changes
Enables collaboration
CLI-based
Web UI + APIs
Created in 2005
Launched in 2008

Important:
👉 Git works without internet
👉 GitHub is only used for push, pull, PRs, CI/CD

Core Git Concepts Repository A project folder tracked by Git. Commit A snapshot of changes with a unique hash. Branch An independent line of development. Merge Combining changes from one branch into another. 3.1 Understanding the Git Three-Stage Workflow (Easy Explanation) This diagram shows how files move step-by-step inside Git, from your computer to permanent history.

A. Working Directory
This is your actual project folder on your system.
You write code here
You edit files here
Git is aware of changes but hasn’t saved them yet
Example:
vim app.py

At this stage, Git sees changes but nothing is recorded.

B. Staging Area
The staging area is like a preview or waiting room.
You tell Git which changes you want to save
Only selected files move forward
Command used:
git add app.py

Now Git knows:
“This file should be included in the next commit.”
This gives you control over what goes into version history.

C. Repository
The repository is where Git permanently stores snapshots of your project.
Once committed, changes become part of history
Each commit has a unique ID (hash)
Command used:
git commit -m "Add initial app logic"

Now the change is saved forever in Git history.

D. How the Flow Works Together
Working Directory → Staging Area → Repository
edit add commit

git add moves files from working directory → staging
git commit moves files from staging → repository
3.2 Understanding Local vs Remote Git Workflow (Easy Explanation)
This diagram shows how your code moves between your system (local) and GitHub (remote) using common Git commands.
Think of it as two worlds:
Local → Your laptop / system
Remote → GitHub repository

🔹 Local Side (Your System)
a. Working Directory
This is where you actually write and edit code.
Files are modified here
Git is aware of changes but hasn’t saved anything yet
Example:
vim app.py

b. Staging Area
The staging area is where you prepare changes for saving.
Command:
git add app.py

This tells Git:
“I want this file to be part of the next commit.”

c. Local Repository
This is Git’s local database of commits.
Command:
git commit -m "Add app logic"

Now the change is saved permanently on your system with a commit ID.

🔹 Remote Side (GitHub)
d. Remote Repository
This is the repository hosted on GitHub / GitLab / Bitbucket.
It allows:
Collaboration
Backup
CI/CD triggers
Code reviews

e. How Code Moves Between Local & Remote
👉 Push (Local → Remote)
git push origin main

Sends your local commits to GitHub
Makes your code visible to teammates
Triggers pipelines (CI/CD)

👈 Pull (Remote → Local)
git pull origin main

Brings latest changes from GitHub to your system
Keeps your local code updated

f. Checkout (Move Between States)
git checkout branch-name

Switches branches
Can restore files from commits
Moves code back into your working directory

🧠 Simple Flow Summary
Working Directory
↓ git add
Staging Area
↓ git commit
Local Repository
↓ git push
Remote Repository (GitHub)
↑ git pull

3.3 Understanding the Gitflow Workflow (Easy Explanation)
This diagram shows how professional teams manage code using multiple branches instead of pushing everything directly to main / master.

Gitflow is designed to:
Keep production code stable
Allow parallel development
Handle releases and hotfixes safely

🧠 Big Picture First
Think of Gitflow like a road system:
Master (Main) → Production highway
Develop → Integration road
Feature branches → Side streets (new work)
Release branch → Final inspection lane
Hotfix branch → Emergency lane

A. Master Branch (Production)
This branch always contains stable, production-ready code
Every commit here is usually tagged with a version (e.g., v1.0, v2.2)
No direct development happens here
In the diagram:
Master runs at the top
Tags like 0.1, 0.2, 1.0 represent production releases

B. Develop Branch (Main Development Line)
This is where all completed features come together
It represents the next release in progress
Developers do not push features directly to master
In the diagram:
Develop runs below master
Features are merged into develop first

C. Feature Branches (New Work)
Created from develop
Used to build new features
Once finished, merged back into develop
Example:
git checkout -b feature-login develop

In the diagram:
Feature branches are shown at the bottom
They merge back into develop using:
git merge feature-branch

After merging:
git branch -d feature-branch

D. Release Branch (Preparing for Production)
Created when the product is almost ready
Only bug fixes, version bumps, and final testing happen here
No new features allowed
Example:
git checkout -b release/1.0 develop

In the diagram:
Release branch sits between develop and master
After testing, it is merged into:
master (for production)
develop (to keep history consistent)

E. Hotfix Branch (Emergency Fixes)
Used when production is broken
Created directly from master
Fixes critical bugs quickly
Example:
git checkout -b hotfix/0.2 master

In the diagram:
Hotfix branch jumps directly from master
After fixing:
Merged back into master
Also merged into develop
This ensures:
Emergency fixes don’t get lost in future releases.

F. Merge Flow Summary
Feature → Develop
git merge feature-x

Release → Master + Develop
git merge release

Hotfix → Master + Develop
git merge hotfix

🏷️ Tags (Versions)
Tags mark release points
Example:
git tag -a v1.0 -m "Production release"

In the diagram:
Tags appear on the master branch
They represent what is deployed in production
3.4 Understanding This Gitflow Diagram (Very Easy Explanation)
This diagram shows how code moves over time when a team follows the Gitflow branching strategy.

👉 The vertical direction represents time (top = older, bottom = newer).
👉 Each vertical lane is a branch.
👉 Dots are commits.
👉 Arrows show merges between branches.

a. Master Branch (Rightmost – Production)
This is the production branch
Whatever is here is live for users
Every production release is tagged (e.g. 0.1, 0.2, 1.0)
In the diagram:
Blue dots on the right = production releases
Tags show version numbers
🔒 Rule:
Never develop directly on master

b. Develop Branch (Center – Integration Branch)
This is where all completed features come together
Represents the next upcoming release
Always ahead of master
In the diagram:
Yellow dots = commits on develop
Features and bugfixes merge into develop first
Think of develop as:
“What will go live in the next version”

c. Feature Branches (Left Side – New Work)
Created from develop
Used to build new features
Each feature is isolated so it doesn’t break others
In the diagram:
Pink dots = feature branch commits
Multiple feature branches can exist at the same time
Once a feature is done → merged into develop
Example:
git checkout -b feature-login develop

After completion:
git merge feature-login

d. Release Branch (Green – Final Preparation)
Created when features are complete
Only bug fixes, version updates, and final testing
No new features allowed here
In the diagram:
Green dots = release branch
Label “Start of release branch for 1.0” marks release creation
After testing, it is merged into:
master → production
develop → keep history consistent
This ensures:
✔ Production is stable
✔ Develop doesn’t miss fixes

e. Hotfix Branch (Emergency Fixes)
Created directly from master
Used when production breaks
Fixes are urgent and small
In the diagram:
Red dot = hotfix
Created after a production tag (0.1)
After fixing:
Merged into master
Also merged into develop
This avoids:
❌ Fix existing only in production
❌ Losing fixes in future releases

f. Why Merges Go Back to Develop
Notice arrows going back into develop.
This is important because:
Any fix done in release or hotfix must be included in future versions
Develop always stays complete and updated

g. Version Tags Explained
Tags like:
0.1
0.2
1.0
Represent:
Exact production snapshots
Used for rollback, auditing, deployments
Example:
git tag -a v1.0 -m "Production release"

h. Simple Flow Summary
Feature → Develop
Develop → Release
Release → Master
Hotfix → Master + Develop
Tag → Production version

Install & Configure Git
git --version
Purpose: Verify Git installation
git config --global user.name "Ashish"
git config --global user.email "ashish@email.com"
Purpose: Set author identity for commits
Initialize a Repository
git init
Purpose: Create a new Git repository
Creates a hidden .git/ directory
git status
Purpose: Shows current state of repo (tracked, staged, untracked)
Working Directory → Staging → Repository
Stage
Meaning
Working Directory
Files you edit
Staging Area
Files selected for commit
Repository
Permanent commit history
Essential Git Commands (With Definitions)
Create Files
touch file.txt
Creates a new file

Stage Files
git add file.txt
Stages a specific file
git add .
Stages all changes

Commit Changes
git commit -m "Initial commit"
Saves staged changes as a snapshot

View History
git log
Shows full commit history
git log --oneline
Compact commit view
git show
Shows changes of a specific commit

Branching (DevOps Critical)
git branch
Lists branches
git branch feature-login
Creates new branch
git checkout feature-login
Switches branch
git checkout -b feature-api
Create + switch branch
git branch -d feature-login
Deletes merged branch
git branch -D feature-login
Force delete branch
Merging Branches
git merge feature-login
Merges branch into current branch
Merge Types
Fast-forward → No divergence
No-ff → Always creates merge commit
Squash → Combines commits
git merge --abort
Cancels a failed merge
Staging & Stashing
Stash Changes
git stash
Temporarily saves uncommitted work
git stash list
Lists stashes
git stash apply
Restores stash
git stash pop
Apply + remove stash
Undoing Changes Safely
Git Reset (Local Only)
git reset --soft
Moves HEAD, keeps staged
git reset --mixed
Unstages changes (default)
git reset --hard
Deletes changes (dangerous)

Git Revert (Safe for Public Repos)
git revert
Creates a new commit that undoes changes

Amend Last Commit
git commit --amend
Edits last commit

Reflog (Recovery Tool)
git reflog
Shows all HEAD movements — lifesaver for recovery

Tags & Releases
git tag v1.0
Creates lightweight tag
git tag -a v1.1 -m "Release v1.1"
Creates annotated tag
git show v1.1
Shows tagged commit
Ignoring & Cleaning Files
.gitignore
*.log
node_modules/
.env
Prevents files from being tracked

Clean Untracked Files
git clean -n
Dry run
git clean -f
Deletes untracked files

GitHub Basics Clone Repo git clone Downloads full repository

Set Remote
git remote add origin
Links local repo to GitHub
git remote -v
Verifies remotes

Push & Pull
git push origin main
Uploads changes
git pull origin main
Fetch + merge changes

Forks & Pull Requests
Fork → Personal copy of repo
Clone → Local copy
Pull Request → Request to merge code
Standard Open-Source Flow:
Fork repo
Clone fork
Create feature branch
Push changes
Open PR
Advanced Git (DevOps Focus)
Rebase
git rebase main
Rewrites history (cleaner logs)

Cherry-Pick
git cherry-pick
Applies a specific commit

Hooks
Scripts triggered on Git events (pre-commit, post-merge)

Git LFS
Handles large files efficiently

Git in CI/CD
Git push triggers pipelines
Branch rules protect production
Tags trigger releases
GitOps tools watch Git state
Best Practices for DevOps Engineers
Small, meaningful commits
Never rewrite public history
Protect main branch
Use PR reviews
Tag releases
Automate via pipelines

Final Note
Git is not just a tool - it’s the control system of modern DevOps.
Mastering Git means mastering collaboration, automation, and safe deployments.
Thanks for following Week 4 of my Learn-in-Public DevOps journey.

Networking for DevOps and SDLC(Software Development Lifecycle) — The Complete Hands-On Beginner-to-Advanced Guide (Week 3)

Ashish — Mon, 08 Dec 2025 13:09:28 +0000

**Networking Foundations for DevOps — Part 1

**
Ultra-Detailed — Definitions, Why it matters, When to use, and Practical Examples

By Ashish — Learn-in-Public DevOps Journey (Week 3)

Overview

This part gives a rock-solid, practical foundation in networking for DevOps engineers. Every topic below contains:

Definition (what it is),

What it does / why it matters,

When a DevOps engineer uses it,

Practical command examples you can copy–paste and run,

Notes / common pitfalls to watch for.

Targets: system/network troubleshooting, cloud networking, container networking, observability, CI/CD connectivity issues, and on-call remediation.

Table of Contents

Network models: OSI vs TCP/IP

Network interfaces, MAC & link layer

IP addresses: IPv4 and IPv6 (definition + examples)

Subnetting & CIDR (step-by-step + worked examples)

Routing basics & default gateway

ARP (Address Resolution Protocol)

NAT (Network Address Translation) — definition & simple examples

TCP vs UDP — ports, sockets & connection states

MTU, fragmentation and common issues

DNS fundamentals (what, how, tools)

DHCP basics (how clients get addresses)

Linux network tools & commands (ip, ss, netstat, traceroute, ping, curl, dig, nslookup) — with examples

Firewalls basics (iptables/nftables, ufw/firewalld) — examples for DevOps

Virtual networking basics (VLANs, bonding/teaming, bridges)

Container & VM networking concepts (bridge, host, macvlan, overlay)

Quick troubleshooting workflows & checklist

1 — Network Models: OSI vs TCP/IP

Definition: conceptual frameworks that describe how network communication is layered.

OSI (7 layers): Physical → Data Link → Network → Transport → Session → Presentation → Application

TCP/IP (4 layers): Link (Network Interface), Internet (IP), Transport (TCP/UDP), Application (HTTP, DNS, etc.)

What it does / why it matters: Layers help you reason: is the problem physical (cable), link (MAC), routing (IP), transport (TCP), or application (HTTP)? DevOps troubleshooting uses layer thinking to isolate faults.

When DevOps uses it: Incident triage: packet loss (link layer), unreachable IP (network layer), TCP hangs (transport), 502 errors (application).

Example (diagnostic approach):

Layer 1: check ip a, ethtool eth0 — link up/down, speed, duplex.

Layer 2: check arp -a — MAC resolution.

Layer 3: check ip route, ping — IP routing.

Layer 4: check ss -tulnp or netstat — TCP/UDP sockets.

Layer 7: curl -I https://service or check application logs.

Notes: You’ll jump between layers during incidents — practice mapping symptoms to layers.

2 — Network Interfaces, MAC & Link Layer

Definition: A network interface (NIC) is the OS representation of a physical or virtual network adapter. A MAC (Media Access Control) address is the hardware address at the link layer.

What it does: NICs transmit/receive frames; the MAC address uniquely identifies an interface on a local link.

When DevOps uses it: Identifying physical/virtual NICs, bonding multiple NICs, diagnosing cable or switch port problems.

Commands / Examples:

Show interfaces and addresses

ip link show
ip a show eth0

Show MAC addresses

ip link show eth0 | grep link/ether

Get low-level link info

ethtool eth0 # (may need package install)

Notes:

Virtual interfaces (docker0, cni0, veth*) also show here.

MACs are used only inside a broadcast domain — routers do not forward MACs.

3 — IP Addresses: IPv4 & IPv6

Definition:

IPv4: 32-bit addresses shown as dotted decimal (e.g., 192.168.1.10).

IPv6: 128-bit addresses shown hex (e.g., 2001:db8::1).

What it does: Identify hosts at the network layer; routing uses IP addresses.

When DevOps uses it: Assigning static IPs, configuring cloud NICs, diagnosing reachability, setting firewall rules.

Examples:

Show IP addresses on interfaces

ip -4 addr show # IPv4
ip -6 addr show # IPv6

Add a secondary IP

sudo ip addr add 192.168.100.10/24 dev eth0

Remove

sudo ip addr del 192.168.100.10/24 dev eth0

Notes:

Use IPv6 in cloud where supported; be mindful of firewall differences.

Public vs private addresses: private ranges (RFC1918): 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16.

4 — Subnetting & CIDR (worked examples)

Definition: Subnetting divides an IP network into smaller networks. CIDR (Classless Inter-Domain Routing) uses suffix /N to denote network prefix length (e.g., /24).

What it does: Controls which hosts are in the same L3 network (subnet); determines routing and broadcast domains.

Why DevOps uses it: Planning VPC subnets, assigning service subnets, calculating IP ranges for clusters, isolating environments.

Quick rules:

/32 = 1 host (IPv4)

/31 = 2 hosts (special)

/30 = 4 addresses (2 usable)

/29 = 8 addresses (6 usable)

/24 = 256 addresses (254 usable)

Usable hosts = total addresses − network − broadcast (for IPv4)

Example 1 — /24 to /26 split (step-by-step)

Start: 192.168.1.0/24 (addresses 192.168.1.0–192.168.1.255)

Split into four /26 subnets (each has 64 addresses, 62 usable):

192.168.1.0/26 -> 192.168.1.0 - 192.168.1.63 (usable: .1 - .62)
192.168.1.64/26 -> 192.168.1.64 - 192.168.1.127 (usable: .65 - .126)
192.168.1.128/26 -> 192.168.1.128 - 192.168.1.191
192.168.1.192/26 -> 192.168.1.192 - 192.168.1.255

How to calculate binary quick trick:
/26 means 26 ones in netmask: 11111111.11111111.11111111.11000000 -> 255.255.255.192 -> block size 64.

Commands to inspect subnet info:

Show routing table and connected networks

ip route show

Calculate network info (using ipcalc if installed)

ipcalc 192.168.1.10/26

or use 'sipcalc' if available

Notes:

Always reserve addresses for gateway (.1 or .254) and avoid .0/.255 as hosts in /24.

Cloud consoles often reserve first/last IPs in a subnet — check provider docs (AWS, GCP, Azure).

5 — Routing Basics & Default Gateway

Definition: Routing chooses paths packets follow between networks. A default gateway is the router an IP host sends traffic to when the destination is not on the local subnet.

What it does: Routes forward packets between subnets and to the Internet.

When DevOps uses it: Troubleshooting unreachable hosts, peering VPCs, configuring NAT gateways.

Commands / Examples:

Show IP routing table

ip route show

Typical default route output

default via 10.0.0.1 dev eth0 proto dhcp metric 100

Add a route (persistency differs by distro/cloud)

sudo ip route add 10.10.20.0/24 via 192.168.1.1 dev eth0

Delete route

sudo ip route del 10.10.20.0/24

Notes:

Route priority uses metric; lower metric preferred.

Mistyped routes can blackhole traffic — be careful when scripting route changes.

6 — ARP (Address Resolution Protocol)

Definition: ARP maps IPv4 addresses to MAC addresses on the same broadcast domain.

What it does: When Host A wants to reach 192.168.1.10 in its subnet but only knows the IP, it broadcasts an ARP request; the owner replies with its MAC.

When DevOps uses it: Local network troubleshooting (duplicate IPs, stale ARP entries), debugging NIC problems, Docker overlay issues.

Commands / Examples:

Show ARP table

ip neigh show

or (older)

arp -a

Example: ping to populate ARP

ping -c 1 192.168.1.10
ip neigh show

Delete an ARP entry (if stale)

sudo ip neigh del 192.168.1.10 dev eth0

Pitfalls:

ARP spoofing can be a security issue.

Stale ARP entries can happen with VM migration — clear table if necessary.

7 — NAT (Network Address Translation)

Definition: NAT rewrites IP addresses and/or ports of packets crossing a router — common types: SNAT (source NAT) for outbound, DNAT (destination NAT) for inbound.

What it does: Allows multiple private hosts to share a public IP (masquerading), or forwards public traffic to internal hosts (port forwarding).

When DevOps uses it: Internet egress from private subnets, exposing services behind NAT, load balancer + reverse proxy setups.

Simple Linux example (iptables):

Masquerade outbound traffic on eth0 (simplified)

sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

DNAT: forward port 8080 on gateway to internal 10.0.1.10:80

sudo iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 10.0.1.10:80
sudo iptables -A FORWARD -p tcp -d 10.0.1.10 --dport 80 -j ACCEPT

Notes:

Cloud providers offer managed NAT gateways; prefer them to DIY NAT in production for reliability.

NAT breaks end-to-end IP visibility; use logging and port mapping carefully.

8 — TCP vs UDP — Ports, Sockets & Connection States

Definition:

TCP (Transmission Control Protocol): connection-oriented, reliable, ordered.

UDP (User Datagram Protocol): connectionless, lower overhead, no guarantees.

What it does: TCP for HTTP/HTTPS, SSH, database connections; UDP for DNS, syslog, some streaming.

When DevOps uses it: Choose protocol suitable for service; troubleshoot socket states (SYN, ESTABLISHED, TIME_WAIT).

Commands / Examples:

Show listening sockets

ss -tuln # t: tcp, u: udp, l: listening, n: numeric

Show connections

ss -tnp # active tcp connections with process info

Sample output interpretation:

LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=1234))

Common socket states to know:

LISTEN — waiting for incoming connections

SYN_SENT / SYN_RECV — connection handshake

ESTABLISHED — active connection

TIME_WAIT — waiting to ensure remote side received final ACK (normal after close)

Notes:

Excessive TIME_WAIT may indicate short-lived connections; tune tcp_tw_reuse with caution.

UDP is stateless — troubleshooting needs packet captures (tcpdump).

9 — MTU, Fragmentation & Common Issues

Definition: MTU (Maximum Transmission Unit) is the largest packet size that can be transmitted without fragmentation. Fragmentation occurs when a packet exceeds MTU and is split.

What it does: Correct MTU avoids fragmentation; mismatched MTU causes connectivity issues, especially with VPNs and tunnels.

When DevOps uses it: Troubleshooting slow connections, VPNs, overlay networks (VXLAN/Weave/Flannel), Docker overlay MTU issues.

Commands / Examples:

Show MTU

ip link show eth0

Ping with specific packet size to test MTU (Linux)

ping -M do -s 1472 8.8.8.8 # 1472 + 28 = 1500 (ICMP header = 28)

Change MTU (temporary)

sudo ip link set dev eth0 mtu 1400

Notes:

Encapsulation (GRE/VXLAN) reduces usable MTU; reduce interface MTU accordingly (e.g., 1450).

ICMP "Fragmentation needed" messages must reach the source for Path MTU Discovery to work — blocked ICMP breaks PMTUD.

10 — DNS Fundamentals

Definition: DNS maps human names (example.com) to IP addresses (A/AAAA records) and other records (CNAME, MX, TXT).

What it does: Allows services to be addressed by names rather than IPs; critical for service discovery.

When DevOps uses it: Configure app domains, validate DNS propagation, troubleshoot name resolution.

Commands / Examples:

Query A record

dig +short example.com A

Full trace

dig +trace example.com

nslookup interactive

nslookup

server 8.8.8.8
example.com

check TXT or MX

dig example.com TXT
dig example.com MX

Common issues:

Misconfigured TTL causes stale records.

Missing records or wrong zone files cause failures.

Split-horizon DNS (different answers internally vs externally) can confuse troubleshooting.

11 — DHCP Basics

Definition: DHCP dynamically assigns IP addresses and network configuration (gateway, DNS) to clients.

What it does: Simplifies host provisioning and IP management.

When DevOps uses it: Cloud VMs often use DHCP; on-prem hosts and containers may use DHCP; know how DHCP impacts bootstrapping.

Commands / Examples:

Check lease file (example for dhclient)

cat /var/lib/dhcp/dhclient.leases

Force renew (Linux)

sudo dhclient -r eth0
sudo dhclient eth0

Notes:

In cloud environments, metadata services provide more than DHCP (e.g., user data).

Static IPs are preferable for critical services.

12 — Linux Network Tools & Commands (practical)

This is your daily toolkit — copy these.

ip (modern replacement for ifconfig/route)
ip a # show addresses
ip link show # show interfaces & state
ip route show # routing table
ip neigh show # ARP table

ss / netstat
ss -tuln # listening ports
ss -tnp # tcp connections + processes
netstat -tulnp # older systems

ping / traceroute / mtr
ping -c 4 google.com
traceroute google.com
mtr google.com # real-time traceroute+ping (interactive)

curl / wget
curl -I https://example.com # show headers
curl -sS http://api/endpoint | jq '.'
wget https://example.com/file

DNS tools
dig +short example.com
nslookup example.com 8.8.8.8

Packet capture
sudo tcpdump -i eth0 port 80 -w capture.pcap

view live (text)

sudo tcpdump -i eth0 -n -vv

Inspect routing to a host
ip route get 8.8.8.8

L4 testing

test TCP connect to port

nc -vz 10.0.0.5 443 # (netcat)

test UDP (less reliable)

nc -vu 10.0.0.5 123

Notes:

Use sudo for privileged operations.

tcpdump outputs can be large — filter by host/port.

13 — Firewalls: iptables, nftables, ufw, firewalld

Definition: Firewalls enforce policies for traffic filtering (packet/connection level).

What it does: Allow/deny traffic based on IP, port, interface, state.

When DevOps uses it: Control service exposure, secure nodes, implement port-forwarding.

Simple examples (iptables):

Allow incoming SSH

sudo iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

Allow established/related

sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

Drop everything else

sudo iptables -P INPUT DROP

nftables (modern):

sudo nft list ruleset

add rules via nft syntax (recommended to read docs)

UFW (Ubuntu simple firewall):

sudo ufw allow 22/tcp
sudo ufw enable
sudo ufw status

firewalld (RHEL/CentOS):

sudo firewall-cmd --add-service=http --permanent
sudo firewall-cmd --reload

Notes:

Cloud security groups are separate (AWS/GCP/Azure) — ensure both cloud and host firewall rules align.

Incorrect firewall rules can lock you out of remote servers — always keep console access or temporary rules.

14 — Virtual Networking: VLANs, Bonding/Teaming, Bridges

Definition & Use:

VLAN (802.1Q): logical segmentation of a physical network — multiple L2 networks on same cable.

Bonding / Teaming: combine multiple NICs for redundancy or throughput aggregation.

Bridge: L2 device in Linux that forwards frames between interfaces — used for VM/container networking.

When DevOps uses it: Data center network segmentation, high availability NIC setup, container networks.

Examples:

create VLAN (example)

sudo ip link add link eth0 name eth0.100 type vlan id 100
sudo ip addr add 192.168.100.10/24 dev eth0.100
sudo ip link set dev eth0.100 up

show bridges

bridge link

create bridge (for VMs/containers)

sudo ip link add name br0 type bridge
sudo ip link set dev br0 up
sudo ip link set dev eth0 master br0

Notes:

Bonding modes matter (active-backup vs LACP) — coordinate with switch config.

Bridges are the basis for docker0 and many CNI plugins.

15 — Container & VM Networking (core concepts)

Definition: Containers and VMs use virtual networks — bridge, host, macvlan, overlay.

What it does:

bridge: containers get private NIC on host bridge (NAT to outside).

host: container uses host network namespace (no isolation).

macvlan: container appears as separate L2 device on network.

overlay (VXLAN/Weave): connect containers across hosts (used by Docker Swarm, older Kube CNI plugins).

Why DevOps cares: Troubleshooting pod-to-pod, node-to-node connectivity, Service/Ingress behavior, MTU issues on overlays.

Quick inspect examples:

list docker networks

docker network ls
docker network inspect bridge

check container IP

docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name

Kubernetes pointers (preview):

Pods have network namespace and virtual eth0; kubectl exec + ip a inside pod helps debug.

kubectl get svc, kubectl get endpoints, and kubectl describe svc are essential.

Notes:

Overlay networks often need reduced MTU due to encapsulation.

DNS inside containers: check /etc/resolv.conf in pod/container.

16 — Quick Troubleshooting Workflows & Checklist

When a service or host is unreachable, follow a layered checklist:

Is the interface up?

ip link show eth0
ip a show eth0

Does host have an IP & route to destination?

ip addr show
ip route
ip route get

Can you resolve DNS (if name used)?

dig +short service.example.com
nslookup service.example.com

Is the host reachable (ICMP)?

ping -c 4

If ping fails: try traceroute / mtr.

Is the port listening on the server?

ss -tuln | grep :80

Are firewall or security groups blocking?

Check iptables/nft/ufw/firewalld and cloud security groups.

Is NAT/Load Balancer translating correctly?

Review NAT/DNAT rules, ELB/NLB target health.

Packet level check

sudo tcpdump -i eth0 host and port

Check ARP / local broadcast domain

ip neigh show
arp -a

If in containers/k8s: check CNI plugin logs, node routes, kube-proxy status.

Bonus: Small Useful Scripts / One-Liners

Count open connections to a host:

ss -tn state established '( dport = :443 or sport = :443 )' | wc -l

Show top talkers by bytes (netstat variant):

sudo ss -tanp | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr | head

Check for duplicate IPs (ARP)

arp -a | awk '{print $2}' | sort | uniq -d

Networking for DevOps — Part 2

DNS • Routing • NAT • Tools • Firewalls • Cloud Networking
(With Definitions, Deep Explanations & Real DevOps Examples)
By Ashish — Learn-in-Public DevOps Journey (Week 3)

📘 Table of Contents

DNS — Definition + Deep Explanation + Examples
Hostnames, resolv.conf & Name Resolution (Definitions + Examples)
Routing — Definition, Linux Routing, Cloud Routing
NAT — Definition + SNAT + DNAT + Masquerade
Firewalls — Definition + Linux + Cloud Firewalls
Networking Tools — Definitions + How They Work
Real DevOps Debugging Case Studies
Cloud Networking Summary (AWS / Azure / GCP)

1️⃣ DNS — Definition, Working & Practical DevOps Usage
📌 Definition: What is DNS?

DNS (Domain Name System) is a global distributed system that translates human-readable names into IP addresses.

Example:

www.google.com → 142.250.70.14

Without DNS, you would have to type IPs for every site—impossible at scale.

📌 Why DNS Matters for DevOps?

Because every cloud service, microservice, Kubernetes service, API, Git repo, load balancer, and CI/CD tool depends on DNS.

If DNS fails:

Apps fail

API calls fail

Load balancer health checks fail

Containers can’t resolve internal services

CI/CD webhooks break

🔹 How DNS Works (Step-by-Step + Diagram)
[Client]
↓
DNS Resolver
↓
Root Servers
↓
TLD Servers
↓
Authoritative DNS
↓
Final IP returned

Try checking a domain:

dig google.com

🔹 Important DNS Records (with Definitions & Examples)
Record Definition Example
A Maps hostname → IPv4 api.app.com → 54.21.11.9
AAAA Maps hostname → IPv6 app → 2607:f8b0...
CNAME Alias pointing to another domain www → app.com
MX Mail routing Gmail mail servers
NS Nameserver for domain ns1.cloudflare.com
TXT Text records (SPF, DKIM, verification) google-verification
SRV Service discovery _sip._tcp.example.com
🔥 Real DevOps Example — ALB + CNAME

AWS ALB hostname:

myapp-alb-988.ap-south-1.elb.amazonaws.com

DNS record you create:

app.example.com → ALB CNAME above

If ALB changes, your app still works.

🔹 DNS Tools (Definitions + Examples)
✔ dig — Definition: DNS query tool

Examples:

dig linkedin.com
dig +short linkedin.com
dig +trace google.com # full DNS chain

✔ nslookup — Definition: legacy DNS lookup tool
nslookup api.service.com

✔ host — Definition: simple reverse & forward DNS lookup
host google.com

2️⃣ Hostnames, resolv.conf & Name Resolution
📌 Definition: Hostname

A hostname is the human-readable name of a system in a network.

Check:

hostname
hostnamectl

📌 Definition: /etc/hosts

Local static DNS mapping file.

Example entry:

10.0.1.40 backend.internal

Now you can run:

curl http://backend.internal:8080

📌 Definition: resolv.conf

File that tells Linux which DNS servers to use.

Check:

cat /etc/resolv.conf

Example:

nameserver 8.8.8.8
nameserver 1.1.1.1

If this file is wrong → DNS will fail.

3️⃣ Routing — Definitions + Linux Routing + Cloud Routing
📌 Definition: Routing

Routing is the process of selecting which path a packet should take to reach its destination.

Every Linux system has a routing table.

✔ View routing table:
ip route

Example:

default via 192.168.1.1 dev eth0
10.0.0.0/24 dev eth0 proto kernel

Definition: Default Route
The path used when no specific route exists.

✔ Add route manually (used in hybrid cloud)
sudo ip route add 172.16.0.0/16 via 10.0.0.1

🔥 Cloud Routing Example (AWS VPC)
10.0.0.0/16 local
0.0.0.0/0 igw-abc123 # internet access
10.0.2.0/24 nat-xyz987 # private → internet

Definitions:

IGW: Internet Gateway

NAT: Outbound internet for private subnets

4️⃣ NAT — Definitions + SNAT + DNAT + Masquerade
📌 Definition: NAT (Network Address Translation)

Technique to modify IP addresses in packets.

Used for:

internet access

load balancers

proxies

Kubernetes

home routers

🔹 SNAT — Source NAT

Definition: Change source IP before sending to destination.

Used by:

AWS NAT Gateway

GCP Cloud NAT

Example:

Private: 10.0.1.10 → Public: 44.11.22.33

🔹 DNAT — Destination NAT

Definition: Change destination IP of incoming packets.

Example:

Public 44.22.13.7:443 → Private 10.0.1.25:443

Used by:

Reverse proxies

Load balancers

Ingress controllers

🔹 Masquerading

Definition: Dynamic SNAT on Linux.

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

5️⃣ Firewalls — Definitions + Linux + Cloud
📌 Definition: Firewall

A firewall controls which traffic is allowed or blocked.

🔥 Linux Firewall Tools
✔ iptables — packet-level filtering
sudo iptables -L -n

✔ ufw — simple firewall
sudo ufw allow 8080

🔥 Cloud Firewalls
AWS

Security Groups (SG): Stateful

NACLs: Stateless

Azure

NSG

GCP

VPC firewall rules

Example rule:

Allow inbound TCP 22 from 103.94.x.x

6️⃣ Networking Tools — Definitions + Practical Examples
🔸 ping

Definition: ICMP echo tool for basic reachability.

ping google.com

Checks:

DNS

ICMP

packet loss

latency

🔸 traceroute

Definition: Shows path packets take.

traceroute youtube.com

🔸 curl

Definition: HTTP client to test APIs and servers.

curl -I https://example.com
curl -v http://backend:8080/health

🔸 wget

Definition: network downloader

wget https://example.com/file.zip

🔸 ss

Definition: Shows open ports & sockets (modern netstat).

ss -tulnp

🔸 ip

Definition: Modern replacement for ifconfig/route.

ip a
ip link
ip route

🔸 arp

Definition: Maps IP ↔ MAC on local network.

arp -a

7️⃣ Real DevOps Debugging Scenarios
✔ App cannot reach database

Steps:

1️⃣ DNS

nslookup db.internal

2️⃣ Connectivity

ping db.internal

3️⃣ Port

ss -tulnp | grep 5432

4️⃣ Firewall
Check SG/NSG rules.

✔ Port already in use
ss -tulnp | grep 8080
kill -9

✔ Pod cannot reach internet

Check node DNS:

cat /etc/resolv.conf

Check routing:

ip route

8️⃣ Cloud Networking Summary
AWS

VPC

Subnets

Route Tables

IGW

NAT Gateway

Security Groups

NACLs

ALB/NLB

PrivateLink / VPC Endpoints

Azure

VNet

Subnets

NSG

UDR

Application Gateway

Load Balancer

GCP

VPC

Global Subnets

Firewall Rules

Cloud NAT

Cloud Router

**Networking for DevOps — Part 3

**
Load Balancers, VPC Design, Subnets, Kubernetes Networking (Ultra-Detailed)

By Ashish — Learn-in-Public DevOps Journey (Week 3)

Overview

This part covers the backbone of modern cloud networking:

Load Balancers (L4 vs L7)

VPCs (AWS/GCP/Azure)

Subnet types (Public, Private, Database, DMZ)

Routing tables, NAT gateways, Internet gateways

Kubernetes cluster networking (Pod CIDR, Service CIDR, CNI, kube-proxy)

Ingress Controllers (Nginx, Traefik, AWS ALB)

NodePort, ClusterIP, LoadBalancer — everything explained simply

Real diagrams + real DevOps troubleshooting examples

This is one of the most critical parts for DevOps engineers—you’ll use this knowledge daily in cloud, containers, and microservices.

📘 Table of Contents

Load Balancers

Definition

L4 vs L7

Health checks

Sticky sessions

SSL termination

Example in AWS / GCP / Azure

VPC Design for DevOps

What is a VPC?

CIDR selection

Public, private, database subnets

Internet Gateway, NAT Gateway

Route tables

Network ACLs vs Security Groups

Subnets — Deep Dive

Definition

Subnet CIDR design

Public vs Private vs Isolated

Best practices

Kubernetes Networking

Pod CIDR

Service CIDR

CNI (Calico, Flannel, Weave, Cilium)

kube-proxy

Types of Services

ClusterIP

NodePort

LoadBalancer

ExternalName

Ingress

Network Policies

Real DevOps Scenarios

Troubleshooting Checklist

PART 3 — Detailed Content

Load Balancers (LB) — Foundation of Modern Distributed Systems Definition

A Load Balancer distributes incoming traffic across multiple backend servers to ensure:

High availability

Fault tolerance

Scalability

Better performance

Used heavily in microservices, cloud apps, and Kubernetes.

1.1 L4 Load Balancer (Transport Layer Load Balancer)

Definition:
Operates at Layer 4 (TCP/UDP) in the OSI model. It routes traffic only based on IP + Port.

What it does:
It doesn't inspect HTTP headers or URLs — only TCP/UDP ports.

Use Cases:

Database load balancing

TCP services

Game servers

High-speed low-latency systems

Examples:

AWS Network Load Balancer (NLB)

GCP Network TCP Load Balancer

Linux ipvs

Example — AWS NLB:

SSL termination NOT done here

Super low latency

Best for millions of requests per second

1.2 L7 Load Balancer (Application Layer Load Balancer)

Definition:
Operates at Layer 7 (HTTP/HTTPS). It routes traffic by:

Path

URL

Host header

HTTP method

What it does:
Understands the HTTP protocol completely.

Use Cases:

Microservices (ex: /api → service A)

Routing based on URL path

Blue/green deployments

Canary releases

Examples:

AWS ALB

GCP HTTP(S) Load Balancer

Nginx / Envoy / HAProxy

Example:

/ -> frontend
/api -> backend
/auth -> auth-service

1.3 Health Checks

Definition:
Periodic tests by LB to check if backend is healthy.

Types:

TCP health check

HTTP health check

Command-based (K8s probes)

Example (HTTP check):

GET /health
200 OK → healthy
500 / timeout → unhealthy

1.4 Sticky Sessions (Session Affinity)

Definition:
A feature where LB routes same user to same backend server.

Used for:

Stateful applications

Legacy monoliths

Disabled in modern microservices.

1.5 SSL Termination

Definition:
LB decrypts HTTPS → HTTP between LB & backend.

Benefits:

Offload CPU

Central certificate management

Simpler backend setup

VPC Design for DevOps (Cloud Networking Core) Definition — VPC

A Virtual Private Cloud (VPC) is an isolated virtual network you create inside a cloud provider.

Equivalent to:

Your own data center

Your own IP range

Your own routing, firewall, NAT, internet gateway

2.1 How VPC Works (Simplified Diagram)
+------------------ VPC (10.0.0.0/16) ------------------+
| |
| Public Subnet Private Subnet |
| 10.0.1.0/24 10.0.2.0/24 |
| |
| [ EC2 Web ] <--> IGW [ EC2 App ] <--> NAT GW |
| |
+-------------------------------------------------------+

2.2 VPC Components (Definitions + Examples)
✔ CIDR Block

IP range of the VPC
Example:

10.0.0.0/16 → 65,536 IPs

✔ Subnets

Divide VPC into smaller networks.

✔ Route Table

Decides where traffic flows.

Example:

0.0.0.0/0 → igw-1234 (public)
0.0.0.0/0 → nat-5678 (private)

✔ Internet Gateway (IGW)

Allows public internet access.

✔ NAT Gateway

Allows outbound internet traffic from private subnets.

✔ Security Groups

Stateful firewalls.

✔ NACLs

Stateless subnet-level firewalls.

Subnets — Deep Explanation Definition — Subnet

A subnet is a smaller network inside a VPC created from a larger CIDR.

Subnet Types

Public Subnet

Has route to internet via IGW
Used for:

ALB

Bastion host

Public-facing applications

Private Subnet

Outbound only via NAT gateway
Used for:

Application servers

Containers

Microservices

Database Subnet

Isolated, no internet
Used for:

RDS

MongoDB

Redis

Elasticsearch

DMZ Subnet

Used in highly secure architectures.

Subnet Allocation Example (AWS Best Practice)
Purpose Subnet Example CIDR
Public 2 subnets 10.0.1.0/24, 10.0.2.0/24
Private 2 subnets 10.0.3.0/24, 10.0.4.0/24
DB 2 subnets 10.0.5.0/24, 10.0.6.0/24

Kubernetes Networking (The Heart of Cloud-Native)

Kubernetes networking is where 90% DevOps engineers struggle.

Let’s make it simple.

4.1 Pod Networking (Pod CIDR)
Definition

Each pod gets its own IP address.

Requirement

Every pod must communicate with every other pod without NAT.

Example Pod CIDR
10.244.0.0/16 — Flannel
192.168.0.0/16 — Calico

Important Command:
kubectl exec -it pod -- ip a

4.2 CNI — Container Network Interface

Definition:
Plugin responsible for creating pod networks.

Popular CNIs:

Flannel (simple, overlay)

Calico (L3 routing + Network Policies)

Weave

Cilium (eBPF — fastest)

4.3 Service Networking (Service CIDR)

Definition:
Service abstracts pods behind stable virtual IPs.

Example CIDR:

10.96.0.0/12 (default in kubeadm)

4.4 kube-proxy

What it does:
Implements service load balancing via:

iptables (older)

ipvs (faster, production-grade)

4.5 Kubernetes Service Types

ClusterIP (default)

Only accessible inside the cluster.

Example:

type: ClusterIP

NodePort

Exposes service on each node’s port (30000–32767).

Example:

type: NodePort

LoadBalancer

Cloud LB creates automatically.

Used in AWS/GCP/Azure.

ExternalName

DNS CNAME mapping.

4.6 Ingress Controller

Definition:
L7 HTTP reverse proxy inside Kubernetes.

Used for:

Routing /api /auth

SSL termination

Multi-domain hosting

Examples:

NGINX Ingress

Traefik

HAProxy

AWS ALB Ingress Controller

4.7 Network Policies

Definition:
Firewall for pods.

Example:

Allow only app → db
Deny everything else

Real DevOps Scenarios Scenario 1 — Pod can't reach Internet

Checklist:

CNI → Node routing → NAT Gateway → Route Table → IGW

Scenario 2 — App behind ALB returning 502

Possible:

Target group health check failing

Wrong security group

Timeout mismatch

Wrong VPC subnet

Scenario 3 — Microservices unable to talk

Check:

Network policy
Service CIDR
DNS resolution
Pod-to-pod communication

Troubleshooting Checklist Basic:

kubectl get svc

kubectl get ep

kubectl describe svc

kubectl exec -it pod -- curl :

ss -tulnp

ip route

LB Troubleshooting:

Check target health

Check SG/NACLs

Remove stickiness

Compare timeout values

VPC Troubleshooting:

Validate route tables

Confirm NATGW/IGW attachment

Check overlapping CIDRs

**Networking for DevOps — Part 4

**
Firewalls, Security Groups, NACLs & Zero-Trust (Ultra Detailed, Beginner → Advanced)

By Ashish — Learn-in-Public DevOps Journey (Week 3)

📘 Overview

This part explains the security backbone of cloud networking:

What is a firewall? (simple definition + real examples)

Cloud firewalls vs Linux firewalls

Security Groups (AWS/GCP/Azure) & why they’re stateful

Network ACLs (NACLs) & why they’re stateless

SG vs NACL — clear comparison

Zero-Trust Networks

Bastion Hosts & Jumpboxes

Practical DevOps scenarios

Troubleshooting rules that break production

This section is designed so a complete beginner can understand, AND an advanced DevOps engineer can refine their mental model.

📘 Table of Contents

What is a Firewall? (Definition + Types + Examples)

Linux Firewalls

iptables

nftables

ufw

Cloud Security Groups (AWS/GCP/Azure)

Definition

How they work

Inbound/Outbound rules

Real examples

Network ACLs (NACLs)

Definition

Allow/Deny rules

Stateless behavior

SG vs NACL (Simple Comparison)

Zero-Trust Networking

Bastion Hosts / Jump Servers

Real DevOps Scenarios

Troubleshooting Security Issues Checklist

PART 4 — Full Breakdown

What is a Firewall? Definition:

A firewall is a security system that filters inbound and outbound traffic based on predefined rules.

Simple explanation:

A firewall decides who is allowed in, who can go out, and which ports/protocols are allowed.

1.1 Types of Firewalls
Type Meaning Example
Network Firewall Protects networks AWS NACLs, Cisco ASA
Host Firewall Protects a single machine UFW, iptables
Application Firewall Filters HTTP apps WAF, Cloudflare
Cloud Firewalls Built into cloud providers Security Groups
1.2 Firewall Example (Real Life)

Imagine a building:

Security Guard = Firewall

ID Check = Authentication

Permission Check = Rules

Servers work the same way.

Linux Firewalls (Local Machine Level)

These run inside the server itself.

2.1 iptables (Legacy but widely used)

Definition:
iptables is a Linux firewall tool that filters packets using chains and rules.

Example — Allow SSH
iptables -A INPUT -p tcp --dport 22 -j ACCEPT

Block everything else:
iptables -A INPUT -j DROP

View rules:
iptables -L -n

Used heavily in:

Bare-metal servers

Older Kubernetes nodes

Legacy deployments

2.2 nftables (Modern replacement for iptables)

Definition:
A newer, faster firewall framework that replaces iptables.

View rules:

nft list ruleset

2.3 UFW (Uncomplicated Firewall) — Ubuntu’s Easy Firewall

Enable:

sudo ufw enable

Allow SSH:

sudo ufw allow 22

Allow NGINX:

sudo ufw allow 'Nginx Full'

Disable:

sudo ufw disable

Used in Ubuntu servers for quick rule setup.

Cloud Security Groups (SGs) Definition (VERY IMPORTANT):

A Security Group (SG) is a stateful firewall attached to cloud resources like:

EC2 instances

Load Balancers

RDS databases

EKS nodes

Azure VMs

GCP VMs

Stateful = automatically allows return traffic

If inbound rule allows port 80:

Response traffic automatically allowed outbound

No need to create reverse rule

3.1 AWS Security Group Example
Allow inbound HTTP & SSH
Inbound:
80/tcp → 0.0.0.0/0

22/tcp → My-IP

Outbound (default allow)
0.0.0.0/0

3.2 Azure Network Security Group (NSG)

Same concept as AWS SGs, different naming.

3.3 GCP Firewall Rules

GCP uses project-level firewall rules, not instance-level.

3.4 Security Group Use Cases
✔ Allow ALB → EC2
SG-ALB → SG-EC2: 80

✔ Allow App → Database
SG-APP → SG-DB: 3306

✔ Allow Bastion → Private EC2
SG-BASTION → SG-PRIVATE: 22

✔ Block the world, allow private communication
10.0.0.0/16 only

Network ACLs (NACLs) — Subnet Level Firewall Definition:

A NACL is a stateless firewall that controls traffic at the subnet level.

Stateless = NO automatic return traffic

If you allow inbound 80:
➡ You must manually allow outbound 80.

4.1 NACL Example
Allow HTTP:
Inbound:
80 ALLOW

Outbound:
80 ALLOW

Deny everything else:
Inbound:

DENY Outbound:
DENY

Used for:

Extra protection

Secure subnets

Blocking malicious CIDRs

Security Group vs NACL (Simple Chart) Feature SG NACL Level Instance Subnet Stateful ✔ Yes ✖ No Default All deny All allow Best use App-level control Subnet-level restriction Supports deny? No Yes Complexity Simple More complex

Rule:

Use Security Groups for 90% of cases.
Use NACLs only when you must deny CIDRs or need subnet-wide rules.

Zero-Trust Networking — Cloud Security Standard Definition

Zero-Trust means:

Trust no one, not even internal networks

Every connection must be authenticated

Least privilege access always

No implicit trust even inside VPC

Real applications:

AWS IAM roles

Pod identity in Kubernetes

Service mesh (Istio, Linkerd)

mTLS (mutual TLS)

Zero trust is the future of cloud security.

Bastion Hosts (Jump Servers) Definition:

A server in public subnet used ONLY to SSH into private subnet servers.

Diagram:

Internet → Bastion Host → Private EC2

Why?

Secure 22/tcp access

No need to expose private instances to public

Can restrict SSH to your IP only

Real DevOps Scenarios ✔ Scenario 1 — App not accessible on port 8080

Troubleshooting:

SG inbound 8080 open?

SG outbound allowed?

NACL inbound/outbound both allowed?

App actually listening? (ss -tulnp)

Route table correct?

Service behind LB passing health checks?

✔ Scenario 2 — RDS cannot be accessed from EC2

Check:

SG-EC2 → SG-RDS : 3306
Same VPC?
Correct subnet routing?
No NACL deny rule?

✔ Scenario 3 — Kubernetes LoadBalancer stuck in “Pending”

Possible issues:

No public subnet tagged

Firewall blocking 30000–32767

Missing cloud controller

NACL blocking ports

✔ Scenario 4 — Private subnet EC2 has no internet

Check:

NAT gateway exists

Route table has 0.0.0.0/0 → NAT

NACL outbound allow

SG outbound allow

Troubleshooting Checklist 🔍 For Security Group Issues

Check explicit inbound allow

Check outbound (rare but important)

Check instance-level firewall (iptables)

Check if LB → target mapping exists

🔍 For NACL Issues

Must allow BOTH ways

Look for DENY rules

Confirm correct subnet association

🔍 For Linux host issues
sudo ss -tulnp
sudo ufw status
sudo iptables -L -n

🔍 For Kubernetes issues
kubectl describe svc
kubectl describe ingress
kubectl get endpoints

**Networking for DevOps — Part 5

**
Monitoring, Observability & Packet Captures (tcpdump, ss, iperf, Wireshark)

By Ashish — Learn-in-Public DevOps Journey (Week 3)

📘 Overview

In modern DevOps, “networking” isn’t just configuring subnets and IPs — it’s being able to observe, measure, debug, and trace what’s happening inside the network.

This chapter covers the real debugging tools used in Cloud + Linux + Containers + Kubernetes + Production SRE environments:

You will learn:

What observability means in networking

Key metrics: latency, throughput, jitter, RTT, packet loss

Network debugging tools:

ping, traceroute, mtr

ss, netstat

iftop, nload, iperf3

Packet capture tools (tcpdump, tshark, Wireshark)

Deep dive into tcpdump filters with examples

Capturing packets inside containers (Docker/K8s)

Real DevOps troubleshooting scenarios

When to use which tool (flow diagram)

This is a long chapter — but it will make you significantly better than an average DevOps engineer.

📘 Table of Contents

What is Observability in Networking?

Key Network Metrics You Must Understand

Basic Network Monitoring Tools

Real-Time Bandwidth Monitoring Tools

Connection & Socket Monitoring

Packet Captures with tcpdump

Wireshark & tshark (GUI + CLI packet analysis)

Packet Captures Inside Docker & Kubernetes

Real DevOps Troubleshooting Case Studies

Tool Selection Cheat-Sheet

What is Observability in Networking? Definition:

Network observability is the ability to see, measure, and understand network behavior in real-time and retroactively.

Why DevOps needs it:

Diagnose slow applications

Debug API failures

Fix DNS issues

Check load balancer routing

Investigate packet drops

Ensure firewall/NACL rules aren’t blocking traffic

Confirm microservices are communicating properly

Observability tools fall into three categories:

Category Tools Purpose
Monitoring ping, traceroute, netstat, ss Check status & health
Metrics iftop, nload, iperf3 Bandwidth, throughput
Packet Capture tcpdump, Wireshark Deep inspection

Key Network Metrics for DevOps/SRE

These are the fundamentals behind all networking analysis.

2.1 Latency

Time taken for a packet to reach the destination.

Measured using:

ping google.com

2.2 Packet Loss

% of packets that never reach the server.

In mtr:

Loss% column

2.3 Jitter

Variation in latency — extremely important for VoIP, video, real-time apps.

2.4 Throughput

Amount of data transferred per second.

Measured using:

iperf3 -s
iperf3 -c server-ip

2.5 Bandwidth

Maximum theoretical data rate of a network link.

2.6 RTT (Round Trip Time)

Time taken for a request to go and return.

Shown in ping:

rtt min/avg/max/mdev

Basic Network Monitoring Tools (Every DevOps Must Know) 3.1 ping — Latency + Reachability Test Definition:

Sends ICMP echo requests to test connection and latency.

Example:

ping google.com

Uses:

DNS test

Reachability test

Basic latency check

3.2 traceroute — Path Trace
Definition:

Shows each hop between you and the target.

traceroute google.com

3.3 mtr — ping + traceroute combined (best tool)
mtr google.com

Shows:

Packet loss

Latency per hop

Real-time route changes

Most useful tool for network debugging.

Real-Time Bandwidth Monitoring Tools 4.1 iftop — Real-time bandwidth “top” sudo iftop

Shows:

Live traffic between IPs

Highest bandwidth users

4.2 nload — Live incoming/outgoing traffic graph
nload

Great for:

Debugging sudden spikes

Monitoring server saturation

4.3 iperf3 — Network speed testing

Server:

iperf3 -s

Client:

iperf3 -c

Useful for:

Testing between cloud regions

Benchmarking VPNs

Validating network throughput

Connection & Socket Monitoring 5.1 ss — Modern socket investigation tool

Definition:
Replaces netstat, faster & more detailed.

Show listening ports:

ss -tulnp

Find process using port:

ss -tulnp | grep 8080

5.2 netstat — Legacy tool
netstat -tulnp

Still used in many old systems.

Packet Captures with tcpdump (MOST IMPORTANT)

Packet capture = the only way to see exactly what is happening on the wire.

Definition:

tcpdump captures and displays packet-level network traffic.

6.1 Basic Capture

Capture all traffic:

sudo tcpdump -i eth0

Write to file:

sudo tcpdump -i eth0 -w capture.pcap

Stop after 100 packets:

sudo tcpdump -c 100

6.2 Filters (Critical for DevOps)
Capture only HTTP traffic:
sudo tcpdump -i eth0 port 80

Capture only SSL/TLS (HTTPS):
sudo tcpdump port 443

Capture specific IP:
sudo tcpdump host 10.0.1.15

Capture traffic between two hosts:
sudo tcpdump src 10.0.1.15 and dst 10.0.1.20

Capture DNS traffic:
sudo tcpdump port 53

Capture only SYN packets (TCP handshake):
tcpdump 'tcp[tcpflags] & tcp-syn != 0'

6.3 Analyse the pcap file in Wireshark

Open file:

File → Open → capture.pcap

Wireshark allows inspection of:

HTTP requests

TLS handshakes

DNS queries

Retransmissions

Packet loss

TCP window issues

Wireshark & tshark (GUI & CLI) 7.1 Wireshark (GUI)

Used for:

Deep packet inspection

Identifying slow backend services

Seeing encrypted vs unencrypted traffic

Troubleshooting TLS failures

7.2 tshark (CLI version of Wireshark)

Capture DNS traffic:

tshark -f "port 53"

List available interfaces:

tshark -D

Filter HTTP requests:

tshark -Y http

Packet Captures in Docker & Kubernetes 8.1 Capture packets in a Docker container

Find container PID:

pid=$(docker inspect -f '{{.State.Pid}}' container-name)

Capture:

sudo nsenter -t $pid -n tcpdump -i eth0 -w container.pcap

8.2 Capture packets in Kubernetes Pod

Get pod:

kubectl get po -A

Exec tcpdump:

kubectl exec -it pod-name -- tcpdump -i eth0 -w pod.pcap

Real DevOps Troubleshooting Case Studies Case 1: Application is slow

Tools:

mtr
ss -tulnp
iftop

Check:

Packet loss?

High bandwidth consumption?

Port conflict?

Case 2: DNS Issues

Tools:

tcpdump port 53
dig +trace domain.com

Symptoms:

Slow API response

Curl fails randomly

Case 3: API unreachable from Kubernetes

Tools:

kubectl exec -- curl
tcpdump from node
ss -tulnp

Look for:

Firewall rules

Service endpoints missing

Wrong DNS names

Case 4: Load Balancer Health Checks Failing

Tools:

tcpdump port 80
curl -I localhost
ss -tulnp

Tool Selection Cheat-Sheet Problem Tool Slow network mtr, iftop Port blocked ss, tcpdump API unreachable curl, tcpdump DNS issues dig, tcpdump Bandwidth high iftop, nload TCP handshake failing tcpdump Kubernetes network down kubectl exec, tcpdump

**Networking for DevOps — Part 6

SDLC + DevOps Architecture (Ultra-Detailed Week 3 Final Notes)**

By Ashish — Learn-in-Public DevOps Journey

📘 Why This Part Matters

As a DevOps engineer, your entire job sits between:

SDLC (Software Development Life Cycle)

→ The complete process of building software from idea → production → maintenance.

DevOps Architecture

→ The tools, pipelines, environments, networks, and automation that turn SDLC into real deployments.

To build, deploy, scale, troubleshoot, and monitor modern cloud systems, you must deeply understand:

What happens in each SDLC stage

Where DevOps fits

How CI/CD automates the flow

How networking connects everything

How cloud-native architecture changes SDLC

How security integrates (DevSecOps)

How SRE extends DevOps in production

This part connects everything from Week 1 (Linux), Week 2 (Shell Scripting), and Week 3 (Networking) into one complete architecture understanding.

📘 Table of Contents

What is SDLC — DevOps Perspective

Waterfall SDLC vs DevOps SDLC

Detailed Breakdown of Each SDLC Phase

DevOps Architecture: Fully Explained

CI/CD Pipeline Architecture (Deep Dive)

Multi-Environment Flow (Dev → Test → Stage → Prod)

GitOps, IaC & Cloud-Native DevOps

DevSecOps (Security in Every Stage)

SRE vs DevOps

End-to-End DevOps Architecture Diagram

Real-World DevOps Pipeline Example

Week 3 Summary + Completion

SDLC (Software Development Life Cycle) — DevOps View 📌 Definition

SDLC is the complete roadmap for building and maintaining software.
It defines how software is:

Planned

Developed

Tested

Deployed

Released

Maintained

Traditional SDLC was designed for older systems where deployments happened once every few months.

⚠️ But DevOps changed SDLC completely.

Today companies like Netflix, Amazon, Meta ship hundreds of deployments per day — possible only because SDLC evolved through DevOps.

Classical SDLC vs DevOps SDLC Waterfall SDLC (Old Model) Requirements → Design → Coding → Testing → Deployment → Maintenance

Problems:

Dev & Ops are separate

Testing happens too late

Deployments are manual

Feedback comes after weeks/months

No automation

High risk, slow releases

DevOps SDLC (Modern Model)
PLAN → CODE → BUILD → TEST → RELEASE → DEPLOY → OPERATE → MONITOR → FEEDBACK → PLAN

All stages run continuously and automatically.

Key upgrades:

CI/CD automates build, test, deploy

Cloud infra makes deployments scalable

IaC (Terraform/Ansible) automates infra

Monitoring gives real-time feedback

Dev & Ops collaborate closely

Deep Dive: Each SDLC Stage with DevOps Context

Let’s break down each phase the way DevOps teams work in real companies.

3.1 PLAN — Product Requirements + Architecture
Definition:

The stage where teams define what to build and how to design the system.

In DevOps:

DevOps teams participate to:

Define infra needs

Plan environments (Dev/Test/Staging/Prod)

Estimate cloud resources (cost optimization)

Decide CI/CD tools

Decide branching strategy

Plan monitoring & logging

Tools:

Jira

Notion

Confluence

Lucidchart

Example:

A team plans a microservices-based eCommerce backend on AWS using:

EC2 / ECS / Kubernetes

RDS database

S3 storage

CloudFront CDN

Terraform for IaC

Jenkins + GitHub Actions for CI/CD

3.2 CODE — Version Control + Collaboration
Definition:

Writing the source code + storing it in version control.

DevOps Responsibilities:

Set up Git repo

Enforce branch protection

Implement Git branching strategy

GitFlow

Trunk-based development

Code scanning for vulnerabilities

Pre-commit hooks

Tools:

Git

GitHub / GitLab / Bitbucket

Real example:
feature/login-api → pull request → code review → merge → CI pipeline starts

3.3 BUILD — Compilation + Packaging + Containerization
Definition:

Build takes raw code → converts into executable artifact (binary, jar, image).

DevOps Tasks:

Create Dockerfiles

Optimize build caching

Automate builds in CI

Create repeatable builds

Tools:

Maven, Gradle (Java)

npm/yarn (Node)

Docker

Example:
docker build -t webshop/auth-service:v2 .

3.4 TEST — Automated Quality Gates
Definition:

Run automated tests on every code change.

DevOps Tasks:

Add test stages to CI

Fail pipeline if tests fail

Run parallel tests

Add security scans

Add code-quality analysis (SonarQube)

Example test types:

Unit Tests

Integration Tests

API Tests

Load testing

Static analysis

3.5 RELEASE — Versioning + Packaging
Definition:

Preparing artifacts to be stored or deployed.

Tools:

Docker Registry

Github Releases

JFrog Artifactory

AWS ECR / GCR / ACR

Example:
docker push /auth-service:v2

3.6 DEPLOY — Delivering Software to Cloud/Kubernetes
Definition:

Deploying the artifact to a target environment.

DevOps Responsibilities:

Manage zero-downtime rollouts

Implement deployment strategies

Rolling

Blue/Green

Canary

Database migrations

Infrastructure provisioning

Tools:

Terraform

Ansible

ArgoCD

Jenkins

Kubernetes

Example:
helm upgrade --install auth-service ./helm-chart

3.7 OPERATE — Running the System in Production

DevOps ensures:

Server uptime

Container orchestration (K8s)

Load balancers

Firewall rules

Auto-scaling

Backup and DR

Tools:

AWS EC2 / ECS / EKS

Azure AKS

GCP GKE

3.8 MONITOR — Observability + Insights
Definition:

Collect performance data + logs + alerts.

DevOps Tasks:

Set SLIs/SLOs

Configure dashboards

Create alert rules

Analyze logs

Find root causes

Tools:

Prometheus

Grafana

Loki

ELK Stack

CloudWatch

Example:

Alert if CPU > 85%

Alert if API latency > 200ms

Alert if pods crash repeatedly

DevOps Architecture — Detailed Breakdown

A complete DevOps architecture includes:

Developer → Git → CI Server → Artifact Registry → CD → Cloud Infra → Monitoring → Feedback

Let's break each:

4.1 Source Code Management (SCM)

GitHub, GitLab

Branching rules

Webhooks

Commit checks

4.2 Continuous Integration (CI)

Triggered on every commit or pull request.
Runs:

Linting

Unit tests

Build

Code scanning

Security scanning

4.3 Artifact Repository

Stores build outputs.

Examples:

ECR (AWS)

GCR (Google)

ACR (Azure)

Nexus

DockerHub

4.4 Continuous Delivery (CD)

Automatically deploys the artifact to environments.

Tools:

Jenkins

ArgoCD

Spinnaker

GitHub Actions

4.5 Observability Layer

Logs

Metrics

Traces

Tools:

ELK

Loki

Prometheus

Jaeger

4.6 Cloud Infrastructure

Compute (EC2, K8s, GKE)

Network (VPC, Subnets, SG, NACL)

Storage (S3/EBS)

LB (ALB/NLB)

CI/CD Pipeline Architecture (Deep Explanation)

Diagram:

GitHub → Jenkins → Unit Tests → Build Docker Image → Push to Registry → Deploy to K8s → Monitor

Detailed Flow:

Developer pushes code → main branch

Webhook triggers Jenkins

Jenkins pipeline starts

Pipeline stages:

Checkout code

Install dependencies

Unit tests

Build

Security scan

Docker build

Push to ECR

Deploy to K8s via Helm

Notify Slack

Environments Lifecycle: Dev → Test → Staging → Prod

Each environment serves a different purpose.

Dev → Test → Staging → Production

Dev:

Developers experiment.

Test:

QA tests feature behavior.

Staging:

Production replica.

Production:

Real users.

DevOps Responsibilities:

Maintain infra differences

Manage secrets per environment

Control release promotion

Enable rollback

GitOps + IaC + Cloud-Native DevOps 7.1 GitOps

Git is the single source of truth.
Tools:

ArgoCD

Flux

7.2 IaC (Infrastructure as Code)

Everything is code:

VPC

Subnets

EC2

Security groups

K8s clusters

Tools:

Terraform

CloudFormation

Pulumi

7.3 Cloud-Native DevOps

Built around:

Containers

Kubernetes

Service Mesh

Observability

Auto-scaling

DevSecOps — Security Integrated Everywhere

Security now runs inside CI/CD.

SAST — Static Code Scans
DAST — Runtime Testing
Container Image Scanning
Dependency Scanning
Secrets Scanning

Example:

trivy image backend:v1

SRE (Site Reliability Engineering) vs DevOps DevOps SRE Builds automation Ensures reliability Pipelines, IaC Uptime, Error Budgets Improves speed Improves stability Deployments Incident response

SRE = advanced operational reliability layer on top of DevOps.

End-to-End DevOps Architecture Diagram
PLAN
+---------------+
| Jira/Confluence|
+---------------+
|
v
CODE
+---------------+
| GitHub/GitLab |
+---------------+
|
v
BUILD
+---------------+
| Jenkins/GHA |
+---------------+
|
v
TEST
+----------------+
| Unit/Integration|
+----------------+
|
v
RELEASE
+-----------------+
| Docker Registry |
+-----------------+
|
v
DEPLOY
+----------------+
| Terraform/Helm |
+----------------+
|
v
OPERATE
+-----------------+
| AWS/K8s/LoadBal |
+-----------------+
|
v
MONITOR
+-----------------+
| Grafana/ELK |
+-----------------+
|
v
FEEDBACK → PLAN
Real-World DevOps Pipeline Example (Production Grade)
E-Commerce Backend Deployment (AWS Example)
GitHub → Jenkins Pipeline → Docker Build → Push to ECR →
Terraform deploys infra → ECS Fargate → ALB → RDS →
CloudWatch Monitoring → PagerDuty Alerts

What DevOps Would Debug:

AWS networking issues → mtr, ss, dig

Container crashes → docker logs, kubectl logs

High latency → check ALB target health, CloudWatch metrics

Autoscaling → HPA or ASG mismatches

DNS failures → Route53 health checks

Shell Scripting for DevOps (Week 2)

Ashish — Sun, 23 Nov 2025 06:09:24 +0000

Part 1 - The Essentials: Shebang, Shell Types, Basic Syntax, Variables & User Input

By Ashish — Learn in Public DevOps Journey (Week 2)
🔗 LinkedIn: https://www.linkedin.com/in/ashish360/
Github: https://github.com/ashish0360/devops-learn-in-public/tree/main/shell-scripting-for-devops

📘 Table of Contents

Why Shell Scripting Matters in DevOps
What is a Shell? (bash, sh, dash)
The Shebang (#!)
Running a Shell Script
Basic Syntax: echo, comments, variables
Reading User Input (read)
Script Arguments ($0, $1, $2)
Debugging & Error Handling (set -x, set -e, set -o pipefail)
Essential Commands You’ll Use Daily
Summary & What’s Next (Part 2 Preview)
🚀 Why Shell Scripting Matters in DevOps
Shell scripting is the foundation of DevOps automation.
Every time you:
Deploy an application
Configure a server
Build a container
Parse logs
Create CI/CD pipelines
Automate infrastructure
Trigger AWS CLI, Azure CLI, or gcloud
Write Kubernetes helpers

…you are essentially writing or using shell scripts.
If you master shell scripting → you unlock real automation power.

🧠 What Is a Shell? (bash vs sh vs dash) A shell executes your commands. Common shells: Shell Path Notes bash /bin/bash Default for Linux servers, DevOps standard sh /bin/sh Lightweight shell, usually a link to dash dash /bin/dash Very fast, used in system-level scripts zsh/fish optional For interactive use

➡️ DevOps uses bash the most.

🔥 The Shebang (#!) The first line of every script defines which shell should run it. Most common: #!/bin/bash

System-level (mapped to dash on many systems):

!/bin/sh

Why it matters:
bash has more features than sh/dash
Some expressions don’t work in dash
CI pipelines may break if the wrong shebang is used

▶️ How to Run a Shell Script Step 1 — Create a script touch script.sh

Step 2 — Add a shebang

!/bin/bash

Step 3 — Give execute permission
chmod +x script.sh

Step 4 — Run
./script.sh

Or:
bash script.sh
sh script.sh # if POSIX compatible

📝 Basic Syntax Every DevOps Engineer Must Know echo — print output echo "Hello DevOps"

Comments

single-line comment

Block comments:
<<comment
This is a block comment
comment

Variables
User-defined:
name="Ashish"
echo "Hello $name"

System variables:
echo $HOME
echo $PATH
echo $USER

🔡 Reading User Input (read) read -p "Enter your name: " username echo "Welcome $username"

read -p "Enter your age: " age
echo "You entered $age"

Used for:
interactive scripts

validation

server setup tools

menu-based scripts

🧩 Script Arguments ($0, $1, $2 …) When running: ./deploy.sh prod v2

Inside script:
echo "Script name: $0"
echo "Environment: $1"
echo "Version: $2"

Used for:
dynamic deployments
folder creation
automation scripts
CI/CD parameter passing

🔍 Debugging & Error Handling Essentials set -x — debug mode (prints commands) set -x

set -e — exit when a command fails
set -e

set -o pipefail — stop when any pipe command fails
set -o pipefail

Most common DevOps combo
set -exo pipefail

🛠 Essential Shell Commands DevOps Engineers Use Daily Process management ps -ef ps -ef | grep nginx ps -ef | grep python | wc -l ps -ef | grep python | awk -F" " '{print $2}'

curl vs wget
curl https://api.com/data # fetch data
wget https://example.com/file # download file

find
find / -name app.log

These commands become the backbone of automation scripts.

🎯 Part 1 Completed You now understand: What shells DevOps engineers use bash vs sh differences The importance of the shebang How to create and run scripts Variables, input, arguments Debugging with set flags Essential shell commands This is the foundation needed to automate real systems.

=================================================================

Part 2 - Conditions, Expressions, If/Else, Case, Loops (for/while/until) — The Logic Layer of Automation

By Ashish — Learn in Public DevOps Journey (Week 2)
🔗 LinkedIn: https://www.linkedin.com/in/ashish360/

📘 Table of Contents

Why Logic Matters in DevOps Automation
Expressions & Operators (-gt, -eq, -lt…)
If/Else — The Backbone of Script Decisions
Nested Conditions
Case Statement (Menu-Based Scripts)
For Loops — Iteration for Automation
While Loops — Event-Based Automation
Until Loops — Run Until Condition Becomes True
Practical DevOps Examples
Summary & Next Part Preview
🚀 Why Logic Matters in DevOps Automation
Every meaningful DevOps script makes decisions.
For example:
“Is Nginx running? If not, restart it.”
“If the age is >= 18, allow access.”
“If the directory already exists, exit the script.”
“Loop through 100 log files and clean them.”
“Run this until the server becomes healthy.”

Without if, loops, conditions, comparisons, shell scripting is basically just a list of commands.

This section gives you the logic power needed for real automation.

🧠 Understanding Expressions & Operators

Shell scripts don't use normal symbols like > or <.
They use operators inside [[ ]].

🔹 Common numeric operators:
Operator Meaning
-gt greater than
-lt less than
-eq equal
-ne not equal
-ge greater or equal
-le less or equal

Example:

if [[ $age -ge 18 ]]; then
echo "Adult"
fi

🧩 If / Else — Decision Making in Scripts Basic structure: if [[ condition ]]; then # code elif [[ another_condition ]]; then # code else # fallback fi

Example:
a=10
b=5

if [[ $a -gt $b ]]; then
echo "a is greater than b"
else
echo "a is smaller than b"
fi

🎯 Practical Example — Voting Eligibility Script #!/bin/bash

read -p "Enter your age: " age
read -p "Are you Indian? yes/no: " nation

if [[ $age -ge 18 ]]; then
echo "You can vote because you are $age"
elif [[ $nation == "yes" ]]; then
echo "You are Indian, you can vote"
else
echo "You cannot vote because you are $age"
fi

✔ Demonstrates numeric + string evaluations.

🔄 Case Statement — Build Menu-Based Tools

Case is perfect for DevOps automation menus.

read -p "Choose option: " choice

case $choice in
1) echo "Start service" ;;
2) echo "Stop service" ;;
3) echo "Check status" ;;
*) echo "Invalid option" ;;
esac

Used for:
deployment menus
CI/CD utility scripts
service management tools

🔁 For Loop — Iterate Through Numbers, Files, Commands Example 1 — Simple counter: for ((i=1; i<=10; i++)); do echo "Number: $i" done

Example 2 — Create multiple folders:
for (( num=$1 ; num<=$2 ; num++ )); do
mkdir "$3$num"
done

Run:

./folder.sh 1 50 project

Creates:
project1, project2, ..., project50
Perfect for:
log directories
batch processing
repeating commands over files

⏳ While Loop — Run While Condition Is True Example: read -p "Enter a number between 1-8: " num

while [[ $(( num%2 )) -eq 0 && $num -le 10 ]]; do
echo "$num"
num=$(( num+2 ))
done

Useful for:
monitors
waiting loops
retry logic

🔁 Until Loop — Opposite of While

Runs until the condition becomes true.

Example:

n=1

until [[ $n -gt 5 ]]; do
echo "n = $n"
n=$((n+1))
done

🧰 FUNCTIONS (Logic + Reusability)
function vote() {
read -p "Enter your age: " age
read -p "Are you Indian? yes/no: " nation

if [[ $age -ge 18 ]]; then
echo "You can vote"
elif [[ $nation == "yes" ]]; then
echo "You can vote"
else
echo "You cannot vote"
fi
}
vote

Functions are essential in DevOps for:
deployments
checks
re-used scripts
AWS CLI automation

🛠 Real DevOps Use Case — Create Folder via Script #!/bin/bash

create_directory(){
mkdir test
}

if ! create_directory; then
echo "Directory already exists — exiting."
exit 1
fi
echo "Directory created successfully"

✔ Demonstrates if-condition
✔ Demonstrates error handling
✔ Demonstrates function use

🎯 Real DevOps Logic Example — Process Counting ps -ef | grep python | wc -l

Add logic:

count=$(ps -ef | grep python | wc -l)

if [[ $count -gt 1 ]]; then
echo "Python service is running"
else
echo "Python service is DOWN!"
fi

These checks form the basis of:
monitoring
health checks
auto-restarts

📌 Part 2 Complete

You now understand the full logic system in shell scripting:

✔ Numeric & string comparisons
✔ if / else / elif
✔ case (menu-driven automation)
✔ for, while, until loops
✔ functions for reusable automation

You now have the building blocks for real-world DevOps scripts.

=================================================================

**Part 3 - Error Handling, Debugging Flags, Exit Codes & Real DevOps Automation Scripts

**
By Ashish — Learn in Public DevOps Journey (Week 2)
🔗 LinkedIn: https://www.linkedin.com/in/ashish360/

📘 Table of Contents

Why Error Handling Matters in DevOps
Exit Codes (0, non-zero, $? usage)
set -e, set -x, set -o pipefail, set -exo
Error Handling Patterns
Debugging Shell Scripts
Writing Safe & Reliable DevOps Scripts
Practical DevOps Examples
Directory creation with error handling
Process validation
Automated deployment script
Django Notes App deployment
AWS EC2 creation through script
Summary & Next Part Preview
🚨 Why Error Handling Is Critical in DevOps

DevOps engineers write scripts that automate:
deployments
backups
AWS/GCP/Azure infra creation
CI/CD steps
service monitoring
log processing
A single silent error can:
break deployments
corrupt environments
delete production data
take servers offline
So DevOps scripts must never fail silently.
That’s why error handling and debugging flags matter.

🧯 Understanding Exit Codes Every command in Linux returns an exit code.

✔ Exit Code Meaning:
Code Meaning
0 Success
1-255 Error / Failure
Check exit code:
echo $?

Example:
ls /not_found
echo $? # 2 → means failure

As a DevOps engineer, you must use exit codes to stop scripts when something breaks.

🛑 set Flags — Your Safety Net in Shell Scripts

These are critical for production-grade scripts.

3.1 set -e → Exit Immediately on Error

If any command fails, the script stops.
set -e
mkdir test
cp abc.txt xyz.txt # if this fails → script stops
echo "Will NOT run"

When DevOps uses it:
CI/CD pipelines
AWS provisioning
Database backup scripts
Deployment scripts

3.2 set -x → Show Commands as They Execute (Debugging)
set -x
echo "Hello"
ls -l

Output looks like:

echo Hello
ls -l Helps debug scripts line-by-line.

3.3 set -o pipefail → Fail If Any Command in a Pipeline Fails
Without this:
command1 | command2 | command3

❌ Only command3 exit code is checked
✔ Earlier failures are ignored silently

With pipefail:

set -o pipefail

Pipeline fails if any command fails — perfect for DevOps pipelines.

3.4 set -exo — The DevOps Gold Standard
set -euo pipefail

Meaning:
-e → Exit on errors
-u → Undefined variables cause errors
-o pipefail → Pipelines must succeed
This is the correct way to write most DevOps automation scripts.

⚙️ Error Handling Patterns (Must Learn) Pattern 1 — Use || for Manual Error Catching mkdir demo || { echo "Failed to create directory" exit 1 }

Pattern 2 — Use a Function and Catch Its Error
create_directory() {
mkdir demo
}

if ! create_directory; then
echo "Directory already exists"
exit 1
fi

Pattern 3 — Use Combined Flags
set -euo pipefail

deploy() {
echo "Deploying..."
cp app.conf /etc/app/
}

deploy || {
echo "Deployment failed"
exit 1
}

🐞 Debugging Your Script (DevOps Way) Use set -x during debugging: set -x

Or wrap only the part you want to debug:
set -x
command1
command2
set +x

Print variables to trace execution:
echo "User: $USER"
echo "Directory: $PWD"

Debugging is a non-negotiable DevOps skill.

🔥 Real DevOps Scripts with Error Handling Below are real scripts you wrote with me—now rewritten cleanly, documented, and production-ready.

✔ Example 1 — Safe Directory Creation Script

!/bin/bash

create_directory() {
mkdir test
}

if ! create_directory; then
echo "Directory already exists — exiting."
exit 1
fi

echo "Directory created successfully."

Used for:
log directories
deployment folders

✔ Example 2 — Count Running Processes
count=$(ps -ef | grep python | wc -l)

if [[ $count -gt 1 ]]; then
echo "Python service is running"
else
echo "Python service is DOWN!"
fi

Used in:
service monitors
cron jobs
health checks

✔ Example 3 — Django Notes App Deployment (Full Script)
Your complete script, now cleaned and formatted:

!/bin/bash

echo "********** DEPLOYMENT STARTED *********"
code_clone() {
echo "Cloning Django app..."
if [ -d "django-notes-app" ]; then
echo "Directory exists, skipping clone."
else
git clone https://github.com/LondheShubham153/django-notes-app.git || {
echo "Clone failed"; return 1;
}
fi
}

install_requirements() {
echo "Installing dependencies..."
sudo apt-get update && sudo apt-get install -y docker.io nginx docker-compose || {
echo "Dependency installation failed"; return 1;
}
}

required_restarts() {
echo "Performing restarts..."
sudo chown "$USER" /var/run/docker.sock || {
echo "Failed to change docker.sock ownership"; return 1;
}
}

deploy() {
echo "Building and deploying the app..."
docker build -t notes-app . && docker-compose up -d || {
echo "Deployment failed"; return 1;
}
}

if ! code_clone; then cd django-notes-app || exit 1; fi
if ! install_requirements; then exit 1; fi
if ! required_restarts; then exit 1; fi
if ! deploy; then
echo "Deployment failed — notifying admin..."
exit 1
fi

echo "********** DEPLOYMENT DONE *********"

This is real DevOps-grade code.

✔ Example 4 — AWS EC2 Creation Script (Professional Version)

Your script, polished for publication:

!/bin/bash

set -euo pipefail

check_awscli() {
if ! command -v aws &>/dev/null; then
echo "AWS CLI is not installed."
exit 1
fi
}

install_awscli() {
echo "Installing AWS CLI v2..."
curl -s "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o awscliv2.zip
sudo apt-get install -y unzip
unzip awscliv2.zip
sudo ./aws/install
rm -rf aws awscliv2.zip
aws --version
}

wait_for_instance() {
local instance_id="$1"

echo "Waiting for EC2 instance..."
while true; do
    state=$(aws ec2 describe-instances \
        --instance-ids "$instance_id" \
        --query 'Reservations[0].Instances[0].State.Name' \
        --output text)

    [[ "$state" == "running" ]] && break
    sleep 10
done

}

create_ec2_instance() {
instance_id=$(aws ec2 run-instances \
--image-id "$1" \
--instance-type "$2" \
--key-name "$3" \
--subnet-id "$4" \
--security-group-ids "$5" \
--tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=$6}]" \
--query 'Instances[0].InstanceId' \
--output text)

echo "Created instance: $instance_id"
wait_for_instance "$instance_id"

}

main() {
check_awscli

AMI_ID=""
INSTANCE_TYPE="t2.micro"
KEY_NAME=""
SUBNET_ID=""
SECURITY_GROUP_IDS=""
INSTANCE_NAME="Shell-Script-EC2-Demo"

create_ec2_instance "$AMI_ID" "$INSTANCE_TYPE" "$KEY_NAME" "$SUBNET_ID" "$SECURITY_GROUP_IDS" "$INSTANCE_NAME"

}

main "$@"

Used in:
DevOps automation
Infrastructure creation
AWS provisioning
Demo/testing environments

🧾 Part 3 Complete You now understand: ✔ Exit codes ✔ Error handling (||, functions, manual checks) ✔ set -e, set -x, set -o pipefail, set -exo ✔ Debugging shell scripts ✔ Crafting production-grade automation ✔ Real-world scripts for EC2, deployments, directories, processes

This is the level of scripting expected from a DevOps engineer.

=================================================================

Trap, Signals, Cron Jobs & Background Automation (Professional DevOps Guide)

By Ashish — Learn in Public DevOps Journey (Week 2)
🔗 LinkedIn: https://www.linkedin.com/in/ashish360/

📘 Table of Contents

Understanding Signals in Linux
trap — The DevOps Lifesaver
Common Signals Every DevOps Engineer Must Know
Automating Cleanup with trap
Running Background Processes
nohup for long-running jobs
Scheduling with Cron (Beginner → Advanced)
Practical DevOps Automation Examples
Production-Grade Script Templates
Summary & What’s Coming in Part 5
⚠️ Understanding Linux Signals (Critical DevOps Concept)
Linux processes constantly interact with signals — notifications sent by users, other programs, the kernel, or system events.

Most commonly used signals:
Signal Code Meaning
SIGINT 2 Interrupt (Ctrl + C)
SIGTERM 15 Request to terminate
SIGKILL 9 Force kill (cannot be trapped)
SIGHUP 1 Hangup (restart daemons)
SIGQUIT 3 Quit & dump core
SIGUSR1/2 — Custom signals

As a DevOps engineer, you MUST handle signals safely because:
Deployments shouldn’t quit mid-way
Cleanup must always run
Temporary files must be removed
Background tasks must be reliable
That’s where… 👇

🔗 trap — The Most Underrated DevOps Tool trap executes a command when a signal is received.

Syntax:
trap "commands" SIGNALS

🧹 Example 1 — Cleanup Temporary Files on Exit #!/bin/bash

tmp_file="/tmp/myapp.log"

trap "echo 'Cleaning...'; rm -f $tmp_file" EXIT

echo "Working..."
touch $tmp_file
sleep 5

echo "Done."

What it does:
Creates temp file
Removes it even if you press Ctrl+C
Ensures the script never leaves junk behind
This is production-safe scripting.

❗ Example 2 — Trap SIGINT (Ctrl + C) trap "echo 'Ctrl+C detected. Stopping safely...'; exit" SIGINT

while true; do
echo "Running..."
sleep 2
done

Useful for:
long-running operations
loops
automation tasks

🛑 Example 3 — Prevent Script from Dying Unexpectedly trap "echo 'Script terminated unexpectedly'; exit 1" SIGTERM

Used in:
CI/CD pipelines
systemd scripts
deployment hooks

🪝 Example 4 — Multiple Signals at Once trap "cleanup" SIGINT SIGTERM EXIT

cleanup() {
echo "Cleaning up resources..."
rm -rf /tmp/deploy
}

⚙️ Background Jobs (Essential for DevOps) You learned loops, conditions, debugging… Now let’s add background jobs.

7.1 Run Command in Background
command &

Example:

python server.py &

Check jobs:
jobs

Resume background job:
bg %1

Bring to foreground:
fg %1

7.2 nohup — Keep Process Alive After Logout

Used heavily in servers, SSH sessions, and EC2 instances.
nohup python app.py &

You close the terminal → the process still runs.
Output saved to:
nohup.out

⏰ Cron Jobs — Scheduling in Linux (DevOps Mandatory Skill) Cron = Scheduler for Linux automation. Open cron: crontab -e

8.1 Cron Syntax

* * * * command | | | | | | | | | └── day of week (0–6) | | | └──── month (1–12) | | └────── day of month (1–31) | └──────── hour (0–23) └────────── minute (0–59)

8.2 Examples DevOps Engineers Use Daily
Run backup every night:
0 0 * * * /home/ashish/backup.sh

Run a health check every 5 minutes:
*/5 * * * * /home/ashish/check_service.sh

Clear logs weekly:
0 0 * * 0 rm -rf /var/log/*.gz

8.3 Example — Cron + Error Handling

!/bin/bash

set -euo pipefail

logfile="/var/log/health.log"

if ! systemctl is-active --quiet nginx; then
echo "$(date) — Nginx DOWN" >> $logfile
systemctl restart nginx
fi

Cron entry:
*/2 * * * * /home/ashish/health.sh

⚡ Real-World DevOps Automation Examples ✔ Example: Kill Hanging Process & Log the Event #!/bin/bash trap "echo 'Terminated!' >> /var/log/proc.log" SIGTERM

pid=$(pgrep python)

if [[ -z "$pid" ]]; then
echo "No python process running"
else
kill -9 $pid
echo "Killed python process $pid"
fi

✔ Example: Auto-Restart Docker on Failure

!/bin/bash

trap "echo 'Docker restarted due to failure'" EXIT

if ! docker ps >/dev/null; then
systemctl restart docker
fi

✔ Example: Cleanup Docker Resources Automatically

!/bin/bash

set -e

trap "docker system prune -f" EXIT

echo "Running build..."
docker build -t webapp .

📦 Production Template — “Safe DevOps Script” Use this for all future scripts:

!/bin/bash

set -euo pipefail

trap "echo 'Script exited unexpectedly'; cleanup" EXIT

cleanup() {
echo "Performing cleanup..."
}

main() {
echo "Starting task..."
}

main "$@"

This template = best practice.

🎉 Part 4 Complete
You now understand advanced automation fundamentals:
trap
Signal handling
Background tasks
nohup
Cron jobs
Cleanup automation
Production-safe scripting patterns
This is the professional level expected in DevOps interviews & real jobs.

=================================================================

**File Operations, find, grep, awk, sed, Automation Scripts

**
By Ashish — Learn in Public DevOps Journey (Week 2)
🔗 LinkedIn: https://www.linkedin.com/in/ashish360/

📘 Table of Contents

File Operations in Shell Scripting (Beginner → Pro)
Reading/Writing Files in Bash
Using find like a DevOps Engineer
DevOps-Grade grep, sed, awk Integration
File Permissions Automation
Bulk Folder Creation & Organization
Building Real DevOps Automation Scripts
Log Automation, Cleanup Scripts, CI/CD Helpers
Production Template — File-Handling Script
Summary & Next Steps
📁 File Operations in Shell Scripting (Practical Guide)
File handling is at the core of DevOps:
Editing configs
Cleaning logs
Managing backups
Processing monitoring data
Updating YAML/JSON
Generating environment files
Here are the must-know techniques.

1.1 Create a File
touch filename.txt

Inside script:

!/bin/bash

touch report.log

1.2 Write to a File
echo "Hello DevOps" > file.txt

Overwrite? → yes.

Append instead:
echo "New entry" >> file.txt

1.3 Read a File Line-by-Line
while read line; do
echo "Line: $line"
done < file.txt

Use-case:
processing logs, configs, CSVs, access logs

1.4 Read a specific line number
sed -n '5p' file.txt

1.5 Replace text in a file
sed -i 's/old/new/g' config.yaml

Used in:
CI/CD variable changes
Environment promotion (dev → prod)
Auto-fixing broken configs

🔍 find — The Most Important DevOps File Tool find is used everywhere: troubleshoot disk usage locate configs find failing scripts cleanup logs manage permissions

2.1 Find file by name
find / -name "nginx.conf"

2.2 Find directories
find /var -type d -name "log*"

2.3 Find files modified recently

Last 24 hours:
find /var/log -mtime -1

2.4 Delete files older than X days

Critical DevOps script:
find /var/log -type f -mtime +7 -delete

Use this for:
disk pressure issues
log rotation

CI runners storing artifacts

2.5 Find files > 100MB
find / -type f -size +100M

2.6 Find and execute a command (🔥 powerful)
find /tmp -type f -name "*.log" -exec rm -f {} \;

🎯 Integrating grep + sed + awk (DevOps-Level)

These 3 tools together =
90% of DevOps text processing automation.

3.1 grep → find matching lines

Extract errors from logs:
grep -i "error" app.log

3.2 sed → modify lines

Bulk edit config values:
sed -i 's/ENV=dev/ENV=prod/g' .env

3.3 awk → extract fields/reports

Extract IP & status:
awk '{print $1, $9}' access.log

🗄 Automating File Permissions

Fix “permission denied” errors:
sudo chown -R $USER:$USER /path

Change permission in script:
chmod +x deploy.sh

🧬 Bulk Folder & File Creation 5.1 Create 100 folders automatically for i in {1..100}; do mkdir "folder$i" done

5.2 Create folders using arguments

!/bin/bash

for (( i=$1 ; i<=$2 ; i++ ))
do
mkdir "$3$i"
done

Run:

./create.sh 1 50 week

5.3 Create multiple files based on list
for f in $(cat list.txt); do
touch "$f.txt"
done

⚡Real DevOps Automation Scripts

These are the scripts DevOps engineers actually use daily.

6.1 Cleanup Logs Older Than X Days

!/bin/bash

LOG_DIR="/var/log"
DAYS=7

find $LOG_DIR -type f -mtime +$DAYS -delete

echo "Old logs cleaned."

6.2 Disk Pressure Emergency Script

!/bin/bash

du -sh /* | sort -h | tail
df -h

6.3 Find top 10 IPs hitting your server
awk '{print $1}' access.log | sort | uniq -c | sort -nr | head

6.4 Auto-Restart Service When It Goes Down

!/bin/bash

if ! systemctl is-active --quiet nginx; then
echo "Nginx down! Restarting..."
systemctl restart nginx
fi

6.5 Check all Python Processes
ps -ef | grep python | awk '{print $2}'

6.6 Delete specific type of files safely
find . -name "*.tmp" -exec rm -f {} \;

6.7 Backup a directory
tar -cvzf backup.tar.gz /etc

6.8 Parse AWS EC2 instance data
aws ec2 describe-instances | jq '.Reservations[].Instances[].InstanceId'

🏗 Production-Grade File Automation Script Template #!/bin/bash set -euo pipefail

trap "echo 'Exiting safely'; cleanup" EXIT

cleanup() {
echo "Performing cleanup..."
}

log() {
echo "$(date) — $1"
}

process_files() {
for file in *.log; do
log "Processing $file"
grep -i "error" "$file" >> errors.txt
done
}

main() {
log "Script started"
process_files
log "Script completed"
}

main "$@"

This is the structure companies expect.

🎉 Part 5 Complete - Advanced Automation Unlocked
You now know:
Full file operations
find for DevOps scenarios
Combining grep + awk + sed
Bulk file & folder creation
Cleanup, backup, monitoring scripts
Production-safe file automation templates
This is professional-grade shell scripting, not beginner-level theory.

=================================================================

**Part 6 - DevOps Project Scripts, AWS Automation, Error Handling & Deployment

**
By Ashish — Learn in Public DevOps Journey (Week 2)
🔗 LinkedIn: https://www.linkedin.com/in/ashish360/

📘 Table of Contents

Why DevOps Engineers Automate Cloud Tasks
Understanding AWS CLI for Automation
Installing AWS CLI via Shell Script
AWS EC2 Creation Script (Deep-Dive)
AWS Instance Waiters (Production-Grade)
Error Handling, Logging & Exit Codes
Automating Docker + Nginx Deployment
Django App Deployment Script (Your Notes Included)
Real DevOps Cloud Automation Examples
CI/CD Integration Patterns
Summary & Next Part

🚀 Why DevOps Engineers Automate Cloud Tasks

Modern DevOps is automation-first.
A DevOps engineer must be able to:
spin up servers
install dependencies
run deployments
debug errors
restart services
scale infra
perform health checks
all using shell scripting, not just clicking on AWS Console.

Your Week-2 work (EC2 script, error handling, Django deploy script) reflects exactly what real engineers do.

⚙️ Understanding AWS CLI for Automation To automate AWS from your shell scripts, AWS CLI must be: installed configured authenticated tested with simple commands

Check if installed:
aws --version

If output is missing → install inside the script.

🧰 Script: Install AWS CLI Automatically

This is the improved, production-level version of your script.

!/bin/bash

set -euo pipefail

install_aws_cli() {
echo "Installing AWS CLI v2..."

curl -s "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt-get install -y unzip &>/dev/null
unzip -q awscliv2.zip

sudo ./aws/install
aws --version

rm -rf aws awscliv2.zip

}

check_aws() {
if ! command -v aws &>/dev/null; then
install_aws_cli
else
echo "AWS CLI already installed."
fi
}

set -euo pipefail is essential in DevOps:
-e → exit on error
-u → undefined variables error
-o pipefail → detect pipeline failures

🖥️ EC2 Instance Creation Script (Full Breakdown) This is the exact version DevOps teams use — simple, readable, powerful.

create_ec2_instance() {
local ami_id="$1"
local instance_type="$2"
local key_name="$3"
local subnet_id="$4"
local sg_id="$5"
local name="$6"

instance_id=$(aws ec2 run-instances \
    --image-id "$ami_id" \
    --instance-type "$instance_type" \
    --key-name "$key_name" \
    --subnet-id "$subnet_id" \
    --security-group-ids "$sg_id" \
    --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=$name}]" \
    --query 'Instances[0].InstanceId' \
    --output text)

if [[ -z "$instance_id" ]]; then
    echo "ERROR: Instance not created!"
    exit 1
fi

echo "Instance created: $instance_id"

}

⏳ EC2 Waiters — Wait Until Instance Becomes Running

This is pro-level — prevents CI/CD failures due to boot delay.

wait_for_instance() {
local instance_id="$1"

echo "Waiting for instance $instance_id to enter 'running' state..."

while true; do
    state=$(aws ec2 describe-instances \
            --instance-ids "$instance_id" \
            --query 'Reservations[0].Instances[0].State.Name' --output text)

    if [[ "$state" == "running" ]]; then
        echo "Instance is now running."
        break
    fi

    sleep 10
done

}

🐳 Docker + Django Deployment Script (Your Notes → Polished Version) Here is the polished, production-grade version of the Django deployment script you captured in your notes.

!/bin/bash

set -euo pipefail

repo="https://github.com/LondheShubham153/django-notes-app.git"

log() {
echo "$(date) - $1"
}

code_clone() {
log "Checking code directory..."
if [[ ! -d django-notes-app ]]; then
log "Cloning repository..."
git clone "$repo" || return 1
else
log "Repo already exists. Skipping clone."
fi
}

install_requirements() {
log "Installing dependencies..."
sudo apt-get update
sudo apt-get install -y docker.io docker-compose nginx || return 1
}

configure_docker() {
log "Configuring Docker..."
sudo chown "$USER" /var/run/docker.sock || return 1
}

deploy() {
log "Deploying Django app..."

docker build -t notes-app . || return 1
docker-compose up -d || return 1

log "Application deployed successfully."

}

main() {
log "Deployment started."
code_clone || cd django-notes-app
install_requirements
configure_docker
deploy
log "Deployment finished."
}

main "$@"

This is a superb Week-2 DevOps script — you should be proud.

🧨 Error Handling in DevOps Scripts (Industry Level)

Use the pattern:

if ! command; then
echo "Error occurred"
exit 1
fi

Example:

if ! mkdir demo; then
echo "Directory exists! Exiting..."
exit 1
fi

This prevents:
partial deployment
corrupted services
half-created resources
CI/CD failures

🛠 Real DevOps Cloud Automation Examples

These are practical, real-world scripts DevOps engineers write constantly.

8.1 Backup your EC2 metadata
aws ec2 describe-instances > instances.json

8.2 Restart EC2 instance using script
aws ec2 reboot-instances --instance-ids i-123456

8.3 Get EC2 public IP automatically
aws ec2 describe-instances \
--instance-ids i-123 \
--query 'Reservations[0].Instances[0].PublicIpAddress' \
--output text

8.4 Upload files to S3 automatically
aws s3 cp backup.tar.gz s3://mybucket/

8.5 Check if AWS credentials are expired
aws sts get-caller-identity || echo "AWS credentials expired!"

🧩 CI/CD Integration — Best Practices You’ll use your scripts in:

GitHub Actions
Jenkins
GitLab CI
Bitbucket Pipelines
AWS CodePipeline

Checklist:
✔ All scripts must return correct exit codes
✔ Avoid echoing secrets
✔ Never hardcode AWS keys
✔ Use IAM role-based credentials
✔ Scripts must work in non-interactive mode
✔ Include retries & timeouts

🎉 Part 6 Complete — AWS + DevOps Automation Achieved
You now know how to:
✔ Create, manage, and automate AWS EC2
✔ Build error-proof deployment scripts
✔ Integrate Docker + Nginx automation
✔ Apply real-world debugging
✔ Use production-grade error handling
✔ Follow DevOps scripting best practices

This Part brings your skill to junior → mid-level DevOps engineer level.

=================================================================

**Part 7 - Real-World DevOps Projects, Interview-Grade Tasks & Advanced Shell Scripting Patterns

**
By Ashish — Learn in Public DevOps Journey (Week 2)
🔗 LinkedIn: https://www.linkedin.com/in/ashish360/

📘 Table of Contents

Why Real-World Shell Scripting Matters for DevOps
Practical DevOps Project Scenarios (Beginner → Advanced)
File Automation & Log Processing Scripts
System Health, Monitoring & Alert Automation
Networking, Ports & Service Debug Scripts
Infrastructure Automation (AWS, Docker, Nginx)
CI/CD Pipeline Shell Tasks
Interview-Grade Shell Scripting Problems
Optimization, Best Practices & Production Standards
Final Summary + Week-2 Completion
🚀 Why Real-World Shell Scripting Matters in DevOps
Shell scripting is not about writing “scripts”;
it’s about automating infrastructure.

In real DevOps jobs, shell scripts are used to automate:
✔ deployments
✔ server provisioning
✔ log parsing
✔ monitoring
✔ backups
✔ docker builds
✔ pipeline steps
✔ cloud tasks
✔ alerts

This chapter focuses on everything interviewers expect AND everything real DevOps teams use daily.

🧩 Practical DevOps Project Scenarios

(Beginner → Intermediate → Advanced)

✅ Scenario 1 — Rotate & Archive Logs Daily (Cron + Shell)

!/bin/bash

log_dir="/var/log/nginx"
backup_dir="/backup/nginx"

timestamp=$(date +%F-%H-%M)
mkdir -p $backup_dir

tar -czf "$backup_dir/nginx-$timestamp.tar.gz" $log_dir/*

echo "Log backup completed: $timestamp"

Add cronjob:

0 1 * * * /home/ashish/rotate_logs.sh

✅ Scenario 2 — Clean Old System Logs Automatically

!/bin/bash

find /var/log -type f -mtime +7 -delete

Interviewers LOVE this one.

✅ Scenario 3 — Find Top CPU/Memory Processes & Alert

!/bin/bash

cpu_limit=80

ps aux --sort=-%cpu | awk -v limit=$cpu_limit '
NR>1 && $3 > limit {print "HIGH CPU:", $2, $3"%", $11}'

✅ Scenario 4 — Backup Database Daily

!/bin/bash

mysqldump -u root -pPASSWORD dbname > backup_$(date +%F).sql

With cron:
0 2 * * * /home/ashish/db_backup.sh

✅ Scenario 5 — Restart a Service if It’s Down
This script can literally SAVE a production outage:

!/bin/bash

service="nginx"

if ! systemctl is-active --quiet $service; then
echo "$(date): $service DOWN — restarting"
systemctl restart $service
fi

✅ Scenario 6 — Auto-Deploy Build Artifacts (CI/CD Ready)

!/bin/bash

rsync -avz ./build/ user@server:/var/www/app/
systemctl restart nginx

Perfect for GitHub Actions, Jenkins, GitLab CI.

🧪 File Automation & Log Processing Scripts 🔍 Extract all unique IPs from Nginx logs awk '{print $1}' access.log | sort | uniq

🛠 Count HTTP 500 errors
grep " 500 " access.log | wc -l

📂 Find largest files on server (production debugging)
find / -type f -exec du -Sh {} + | sort -rh | head -n 20

🧹 Delete files older than X days
find /tmp -type f -mtime +3 -delete

Very common DevOps interview question.

🖥 System Health, Monitoring & Alerts Memory alert script (interview favorite) #!/bin/bash threshold=80

used=$(free | awk '/Mem/ {print int($3/$2 * 100)}')

if [[ $used -gt $threshold ]]; then
echo "WARNING: RAM usage is ${used}%"
fi

Disk space alert

!/bin/bash

df -h | awk '$5+0 > 85 {print "HIGH DISK USAGE:", $0}'

CPU alert
top -bn1 | grep "Cpu(s)"

🌐 Networking, Ports & Service Debug Scripts Check if a port is open nc -zv localhost 8080

Find which process is using a port
sudo lsof -i :8080

Check DNS resolution in script
nslookup google.com

☁️ AWS Automation Scripts (Real DevOps Work) 🔸 Create EC2 🔸 Wait until ready 🔸 Deploy Docker app 🔸 Restart on failure

You already wrote EC2 creation in Part 6.
Here are two more production scripts:

🛢 Get EC2 instance public IP
aws ec2 describe-instances \
--instance-ids "$1" \
--query "Reservations[0].Instances[0].PublicIpAddress" \
--output text

🧹 Clean unused EBS snapshots (AWS cost savings)
aws ec2 describe-snapshots --owner self \
| jq -r '.Snapshots[] | select(.StartTime < "2024-01-01") | .SnapshotId' \
| xargs -I {} aws ec2 delete-snapshot --snapshot-id {}

🔁 CI/CD Pipeline Shell Scripting Patterns

Almost every CI/CD pipeline uses:

✔ mkdir
✔ cp, rsync
✔ sed for config editing
✔ grep/log parsing
✔ exit codes
✔ if conditions
✔ docker build & run

Example: extract version from package.json

version=$(grep version package.json | cut -d '"' -f4)
echo "Deploying version: $version"

🎯 Interview-Grade Shell Scripting Questions (Your Week 2 Version)

Here are the exact problems companies ask:

Print Fibonacci series up to N
Reverse a file content
Find the largest of 3 numbers
Monitor a log file for a keyword and alert
Parse CSV and print a column
Archive and rotate logs
Validate if a service is running
Extract IP address from logs
Write your own “ls” command using shell
Automate daily backups

If you want, I can generate solutions to all 10 (optional).

📝 Optimization, Best Practices & Production Standards ✔ Use set -euo pipefail ✔ Use functions ✔ Use clear naming ✔ Modular scripts ✔ Avoid hardcoding secrets ✔ Validate inputs ✔ Use logging function ✔ Test scripts with shellcheck

Example:

log() {
echo "$(date) — $1"
}

🎉 Part 7 Complete — You’ve Reached DevOps-Ready Shell Scripting Level

By completing Part 7, you’ve achieved:

✔ Hands-on DevOps automation skills
✔ Cloud scripting knowledge
✔ Error-safe, production-grade syntax
✔ Ability to write monitoring & alerting scripts
✔ Confidence for interviews
✔ CI/CD scripting proficiency
✔ AWS automation capability
✔ Understanding real infrastructure problems

This completes Week 2 of your Learn-in-Public DevOps Journey.

Linux for DevOps - The Complete Hands-On Beginner-to-Advanced Guide (Week 1 Series)

Ashish — Sun, 16 Nov 2025 20:17:13 +0000

Linux for DevOps — Part 1
Linkedin Profile: https://www.linkedin.com/in/ashish360/
Hashnode Link: https://devopswithashish.hashnode.dev/linux-for-devops-the-complete-hands-on-beginner-to-advanced-guide-week-1-series

Introduction + Filesystem & Directory Commands
By Ashish — Learn-in-Public DevOps Journey (Week 1)
📘 Table of Contents
Why Linux Matters in DevOps
Linux System Architecture
Linux Distributions
Setting Up Linux (Windows/Mac/Linux)
Package Managers Explained
Filesystem & Directory Commands (Full Deep Dive)

🚀Why Linux Matters in DevOps The more I explored Linux this week, the clearer it became why every DevOps engineer, cloud engineer, SRE, and backend developer depends on Linux daily. Linux is the foundation of DevOps, because it is: 🟦 Cost-Effective Free and open-source No licensing fees Low maintenance and predictable performance 🟦 High Performance Lightweight and fast Works on everything from Raspberry Pi to enterprise servers Efficient memory and process management 🟦 Secure & Reliable Strong permission system Very limited malware exposure Systems can run for years without reboot Every cloud provider (AWS, GCP, Azure), container runtime (Docker), orchestration system (Kubernetes), infrastructure tool (Terraform), and automation platform (Ansible) runs on Linux. If you want to be a DevOps engineer, you must master Linux deeply.
🔧Linux System Architecture (Simple Diagram) +----------------------------------------------------+ | User Applications (Docker, Vim, Git, Apache, etc.) | +----------------------------------------------------+ | Shell (Bash, Zsh, Fish, etc.) | +----------------------------------------------------+ | System Libraries (glibc, libc6, OpenSSL, etc.) | +----------------------------------------------------+ | System Utilities (ls, grep, ps, systemctl, etc.) | +----------------------------------------------------+ | Linux Kernel (Processes, Memory, FS, Network) | +----------------------------------------------------+ | Hardware (CPU, RAM, Disk, NICs, Peripherals) | +----------------------------------------------------+

✔ Hardware Layer
Physical components: CPU, RAM, storage, network cards.
✔ Kernel
The brain of Linux.
Handles:
process scheduling
memory allocation
file systems
networks
drivers
system calls
✔ Shell
Interface between user & kernel.
Examples: Bash, Zsh, Fish, Ksh.
✔ User Applications
Tools like Docker, Git, Terraform, Jenkins, Nginx, etc.

🐧Linux Distributions (Distros) Different distros package the Linux kernel with tools and package managers. ⭐ Best distros for DevOps: Ubuntu (most popular & beginner-friendly) Debian (stable, reliable) Fedora (cutting-edge) AlmaLinux / Rocky Linux (RHEL-based, enterprise) Arch Linux (advanced users) Alpine Linux (super lightweight, perfect for Docker) Kernel source: https://git.kernel.org https://github.com/torvalds/linux
🖥️Setting Up Linux (Windows & Mac) ✔ Best Method: Use Docker Fast, safe, sandboxed environment for DevOps practice. I won’t repeat commands here — they’re already in your notes above. (This article will stay focused on filesystem commands.)
📦Package Managers (Quick Summary) Distro Package Manager Example Ubuntu/Debian apt sudo apt install nginx RHEL/CentOS dnf / yum sudo dnf install nginx Arch Linux pacman sudo pacman -S nginx OpenSUSE zypper sudo zypper install nginx

Commands:
sudo apt update
sudo apt upgrade -y
sudo apt install nginx
sudo apt remove nginx
sudo apt autoremove
⭐6. FILESYSTEM & DIRECTORY COMMANDS (FULL DETAILED VERSION)
This is where Part 1 becomes extremely valuable.
Every command below includes:
Definition
What it does
Why DevOps uses it
Practical example
Variations
Notes (dangerous flags highlighted)
6.1 ls — List Files
Definition:
Lists files and directories.
What it does:
Shows names, sizes, permissions, timestamps, and ownership.
DevOps Use-Cases:
Checking log files
Checking deployment directories
Debugging incorrect paths
Inspecting mounted volumes in Docker/K8s
Example:
ls -ltr
l = long format
t = sort by latest
r = reverse order
Variations:
ls -a # show hidden files
ls -lh # human readable sizes
ls -R # recursive

6.2 pwd — Print Working Directory
Definition:
Displays the absolute path of the current directory.
DevOps Use-Cases:
Confirm before deleting
Confirm before copying files
Debug script working directories
Example:
pwd

/var/www/app

6.3 cd — Change Directory
Definition:
Navigates between directories.
Examples:
cd /etc/nginx
cd ~
cd ..
cd -

Notes:
cd - returns to the previous directory.
6.4 touch — Create Empty File
Definition:
Creates a new file or updates timestamp.
DevOps Use-Cases:
Create placeholders
Trigger builds watching timestamps
Create test logs
Example:
touch app.log

6.5 mkdir — Create Directory
Example:
mkdir logs
Create nested directories:
mkdir -p /opt/app/logs

6.6 rmdir — Remove Empty Directory
Example:
rmdir oldfolder

⚠️ Only works if directory is empty.
6.7 rm — Remove Files or Directories
Definition:
Deletes files/folders.
DevOps Use-Cases:
Delete artifacts/logs
Clean build folders
Examples:
rm file.txt
rm -r folder/
rm -rf /tmp/data
⚠️ Danger: Never run rm -rf /.
6.8 cp — Copy Files
Examples:
cp a.txt b.txt
cp -r /var/www /backup

6.9 mv — Move or Rename
Example:
mv config.old config.yaml
6.10 cat — Show File Content
Example:
cat /etc/hostname

6.11 tac — Show File in Reverse
Example:
tac server.log

6.12 head — First Lines
head -n 20 file.txt

6.13 tail — Last Lines
tail -n 20 access.log

Live logs:
tail -f /var/log/syslog

6.14 wc — Word Count
wc -l app.log

6.15 echo — Print / Write Content
echo "Hello World"
echo "PORT=8080" > app.env

6.16 cut — Extract Columns
cut -d',' -f2 users.csv
6.17 diff — Compare Files
diff old.conf new.conf

6.18 ln — Create Links
Soft link:
ln -s /var/www/html index
Hard link:
ln file1 file2

6.19 zcat — View Compressed Files
zcat logs.gz

6.20 which — Locate Executable Path
which python3

6.21 file — Identify File Type
file script.sh

Linux for DevOps — Part 2
Viewers, Editors & File Inspection Commands
By Ashish — Learn-in-Public DevOps Journey (Week 1)
This is one of the most important sections because DevOps engineers read logs, inspect configs, debug deployments, and modify system files daily.
📘 Table of Contents
File Viewing Tools
cat, tac, more, less, head, tail, tail -f, nl, od, strings, zcat
Redirection & Output Tools
Tee, echo
Editors
nano, vi / vim
Using Editors in DevOps Tasks
Summary & Next Section

⭐ 1. File Viewing Tools
These commands help you inspect file contents, logs, configs, and scripts, the daily life of a DevOps engineer.
1.1 cat — Display File Content
Definition:
Concatenates and prints file content to the terminal.
What it does:
Shows the entire content, useful for small/medium-sized files.
DevOps Use-Cases:
Inspect environment files
View configs (nginx.conf, .env)
Check last updated logs
Example:
cat /etc/os-release
Variation:
cat file1 file2 > merged.txt

1.2 tac — Display File Content in Reverse Order
Why DevOps uses it:
Logs append new entries at the bottom.
tac helps examine the latest entries first.
Example:
tac application.log

1.3 more — Page Through File
Definition:
Displays file one page at a time.
Example:
more /var/log/syslog

Note:
Only supports forward movement.
1.4 less — Advanced File Viewer (Better than more)
Definition:
Interactive file viewer with scrolling.
DevOps Use-Cases:
Inspect huge log files
Search inside configs
Scroll back-and-forth easily
Example:
less /var/log/auth.log
Navigation:
Up/Down arrows → scroll
G → go to end
1G → go to top
/keyword → search
1.5 head — Show First Lines
Example:
head -n 20 nginx.conf

1.6 tail — Show Last Lines
Example:
tail -n 20 access.log

1.7 tail -f — Live Log Monitoring
Definition:
Follow file changes in real-time.
DevOps Use-Cases:
💡 This is one of the MOST used DevOps commands.
Monitor services
Watch logs during deployment
Debug CI/CD issues
Observe API requests in real time
Example:
tail -f /var/log/syslog

1.8 nl — Add Line Numbers to Output
Definition: Prints file with numbered lines.
Example: nl index.html
Useful when:
Debugging long config files
Reviewing scripts with errors
1.9 od — View File in Octal/Hex Format
Definition: Displays binary files in readable form.
Example: od -c myfile.bin
Use-cases for DevOps:
Inspect corrupted config
Debug unusual characters
1.10 strings — Extract Text from Binary
Definition: Prints readable text from binary files.
Example: strings /usr/bin/ssh

1.11 zcat — View GZipped Files
Definition:
Read .gz compressed files without decompressing.
Example:
zcat nginx-access.log.gz
⭐ 2. Redirection & Output Tools
2.1 echo — Print or Write Text
Examples:
echo "Hello World"
echo "ENV=prod" > app.env
echo "PORT=8080" >> app.env

Uses for DevOps:
Append environment variables
Create config files dynamically
Print messages inside scripts
2.2 tee — Write Output to File + Display
Definition: Sends output to terminal and saves it to file.
Example: echo "server restarted" | tee /var/log/restart.log

Use-case: Logging automation scripts.

⭐ 3. Editors
3.1 nano — Simple Text Editor
Definition: Beginner-friendly terminal editor.
Example: nano config.yaml

Common Shortcuts:
Ctrl + O → Save
Ctrl + X → Exit
Ctrl + K → Cut line
Ctrl + U → Paste
3.2 vi / vim — Advanced Text Editor
Definition: Powerful modal editor used heavily in DevOps/SRE work.
Why DevOps uses it:
Edit configs in servers
Edit YAML, scripts, Dockerfiles
Works in all environments (SSH, containers, recovery mode)

⭐ VIM MODES
Mode
Purpose
Normal
Navigation & commands
Insert
Editing text
Command
Save, quit, search

Switching:
i → insert
Esc → normal
: → command mode
⭐ Most Used VIM Commands
Save & Quit
:w # save
:wq # save & quit
:q! # quit without saving
Navigation
h j k l # left, down, up, right
0 # start of line
$ # end of line
gg # start of file
G # end of file
:n # go to line n

Editing
x # delete char
dd # delete line
yy # copy (yank)
p # paste
u # undo
Ctrl+r # redo

Search
/pattern
n
N

⭐ 4. Using Editors in DevOps
Use-cases:
Edit nginx/apache configs
Update environment variable files (.env)
Modify system configs in /etc/
Fix shell scripts
Edit Dockerfiles, Jenkinsfiles
Make changes inside Linux containers
Every DevOps engineer uses vim daily — especially over SSH.

Linux for DevOps — Part 3
Text Processing: grep, awk, sed
By Ashish — Learn-in-Public DevOps Journey (Week 1)
📘 Table of Contents
✔ Introduction — Why Text Processing Matters in DevOps
✔ grep — Pattern Searching (Basic → Advanced)
✔ sed — Stream Editing (Line Edits, Replacements, Filters)
✔ awk — Text Extraction, Filters, Reports
✔ Practical DevOps Log Examples
✔ Real-World Automation One-Liners
✔ Summary & Next Part
⭐ 1. Why Text Processing Is Critical for DevOps
As a DevOps engineer, your entire workflow revolves around:
Logs
Configs
YAML/JSON files
System metrics
CI/CD output
Kubernetes events
Network traces
Monitoring alerts

And the three tools that dominate every Linux-based DevOps environment are:
👉 grep — find patterns
👉 sed — modify text
👉 awk — extract, manipulate & report text
Every DevOps/sre uses these commands daily.
⭐ 2. grep — Global Regular Expression Print
Definition: Searches for patterns inside files or command outputs.
What it does: Finds matching text, filters lines, highlights patterns.
Why DevOps uses it:
Debug errors in logs
Extract warning or failure messages
Filter Kubernetes logs
Analyze API access logs
Search configs during outages
Validate environment variables
🔹 2.1 Basic grep Usage
Example — Search for “error” inside a log:
grep "error" app.log
Case-insensitive search:
grep -i "error" app.log
🔹 2.2 Search recursively (folders)
grep -r "timeout" /var/log/
Use-case:
Search entire server logs for an error.
🔹 2.3 Count how many times a pattern appears
grep -c "404" access.log
🔹 2.4 Show line numbers
grep -n "invalid" /var/log/auth.log
🔹 2.5 Invert match (show NON-matching lines)
grep -v "200" access.log
Useful for finding errors in successful requests.
🔹 2.6 Highlight matches
grep --color=auto "failed" /var/log/secure
🔹 2.7 Filter processes
ps aux | grep nginx
This command is used by DevOps engineers hundreds of times.
🔹 2.8 Regex Search
End with .log:
ls | grep ".log$"
Match numbers:
grep -E "[0-9]+" file.txt
⭐ 3. sed — Stream Editor
Definition:
Edits text in a stream (file or pipeline) without opening an editor.
What it does:
Replace text
Delete lines
Extract lines
Insert content
Modify config files automatically
Why DevOps uses sed:
Automate config editing
Mass update parameters
Modify YAML/JSON in CI/CD
Fix logs in pipelines
Edit environment files
🔹 3.1 Replace Text (Global Replace)
Replace “error” with “failed”:
sed 's/error/failed/g' app.log
Case-insensitive:
sed 's/error/failed/gi' app.log
🔹 3.2 Replace text in a specific line range
sed '1,5 s/debug/info/g' log.txt
🔹 3.3 Print only matching lines
sed -n '/timeout/p' server.log
Equivalent to:
grep "timeout" server.log
🔹 3.4 Show line numbers where pattern matches
sed -n -e '/error/=' server.log
🔹 3.5 Combine multiple sed operations
sed -n -e '/error/=' -e '/error/p' logfile
🔹 3.6 Delete lines matching a pattern
sed '/DEBUG/d' app.log
🔹 3.7 Insert a line before a match
sed '/server {/i # Managed by DevOps' nginx.conf
🔹 3.8 Inline editing (overwrite original file)
sed -i 's/old/new/g' config.yaml
⚠ Use with care — this modifies the file permanently.
⭐ 4. awk — Text Processing & Reporting Language
Definition:
A powerful pattern-based text processing tool.
What it does:
Extracts columns
Filters rows
Performs calculations
Formats reports
Processes logs
Processes CSV/TSV
Why DevOps uses awk:
Analyze log fields
Extract IPs
Process CPU/memory data
parse API responses
Generate metrics
Extract from kubectl outputs

🔹 4.1 Print specific columns
awk '{print $1, $3}' file.txt
$1 = first column
$3 = third column
🔹 4.2 Filter by pattern and print fields
awk '/error/ {print $2, $5}' app.log
🔹 4.3 Count lines matching condition
awk '/404/ {count++} END {print count}' access.log
🔹 4.4 Use numerical conditions
Print IPs in a specific range:
awk '$2 >= "10.10.11.14" && $2 <= "10.10.11.51" {print $2}' file
🔹 4.5 Print lines with > X response time
awk '$5 > 500 {print $0}' api.log
🔹 4.6 awk as a CSV processor
awk -F',' '{print $1,$3}' users.csv
-F sets delimiter.
🔹 4.7 Sum values (analytics)
Example: sum total bytes from access logs:
awk '{sum += $10} END {print sum}' access.log
⭐ 5. Practical DevOps Examples
These are commands you’ll use daily as a DevOps engineer.
➡️ 5.1 Check how many 500 errors occurred today
grep " 500 " access.log | wc -l
➡️ 5.2 Extract all unique IPs
awk '{print $1}' access.log | sort | uniq
➡️ 5.3 Monitor logs for timeouts
grep -i "timeout" -r /var/log/
➡️ 5.4 Replace environment variable in CI
sed -i 's/ENV=dev/ENV=prod/g' .env
➡️ 5.5 Analyze API response times
awk '$9 > 400 {print $1, $9}' access.log
➡️ 5.6 Extract disk usage from df -h
df -h | awk 'NR>1 {print $1, $5}'
⭐ 6. DevOps One-Liner Automations
✔ Restart service if memory crosses threshold
free -m | awk '/Mem/ {if($3/$2*100 > 90) print "Restart needed!"}'
✔ Detect high CPU processes
ps aux | awk '$3 > 50 {print $0}'
✔ Validate YAML indentation in CI/CD
sed -n '/^[[:space:]]{2}[^ ]/p' file.yaml
✔ Extract container IDs using grep + awk
docker ps | grep nginx | awk '{print $1}'

📝 Linux for DevOps — Part 4
User, Group & Permission Management
By Ashish — Learn in Public DevOps Journey (Week 1)
📘 Table of Contents
✔ Users & User Accounts
useradd, adduser, userdel, passwd, su, Id, who / whoami
✔ Groups & Group Management
groupadd, groupdel, gpasswd, usermod (group membership)
✔ Permissions (rwx)
chmod, chown, chgrp, umask, special bits (SUID, SGID, Sticky Bit)
✔ Understanding Permission Structure
✔ Practical DevOps Scenarios
✔ Security Best Practices
✔ Summary & Next Part

⭐ 1. What Are Users & Groups in Linux?
Linux has a powerful permission model based on:
User (owner) — who created the file
Group — team members
Others — everyone else
Every resource (file, folder, process, service) has ownership + permissions.
This is why as a DevOps engineer you MUST understand:
Who can run a script?
Who can read logs?
Who can restart services?
Who can modify CI/CD configs?
Who can access production servers?
Now let’s go one by one.
⭐ 2. User Management Commands
2.1 useradd — Create a New User (Low-Level)
Definition:
Creates a new user account.
Why DevOps uses it:
Create service accounts
Create users for automation
Create test users inside containers
Add temporary developers/engineers
Example:
sudo useradd -m ashish
-m creates a home directory (/home/ashish)
Common flags:
useradd -s /bin/bash ashish # set default shell
useradd -d /data ashish # custom home directory
useradd -u 1050 ashish # specify UID

2.2 adduser — High-Level User Creation (Recommended)
Definition: Interactive wrapper for useradd.
Example: sudo adduser devopsuser
Prompts for:
password
full name
phone
room, etc.
Use-case:
Best for human users (developers, admins).
2.3 passwd — Set or Change User Password
Example:
sudo passwd ashish

Why DevOps uses it:
Reset developer access
Set passwords for test accounts
Update service account credentials
2.4 userdel — Delete User
Example:
sudo userdel ashish
Delete home directory too:
sudo userdel -r ashish

2.5 su — Switch User
Example:
su ashish
Switch to root:
su -

2.6 who — List Logged-In Users
who
Useful for multi-admin servers.
2.7 whoami — Current User
whoami

2.8 id — Show User Identity
Displays UID, GID, and group memberships.
Example:
id ashish

⭐ 3. Group Management
3.1 groupadd — Create Group
sudo groupadd devops
Used to group users by role.
3.2 gpasswd — Manage Group Membership
Add user to group:
sudo gpasswd -a ashish devops
Remove user:
sudo gpasswd -d ashish devops
Set multiple users:
sudo gpasswd -m user1,user2 devops

3.3 groupdel — Delete Group
sudo groupdel devops

⭐ 4. Permission Bits (rwx)
Linux permissions are shown like:
-rwxr-xr--

Breakdown:
Position
Meaning
1
file type (-, d, l)
2–4
owner permissions
5–7
group permissions
8–10
others permissions

rwx = read, write, execute

⭐ 5. chmod — Change Permissions
Definition:
Sets read/write/execute permissions.
5.1 Numeric (Octal) Notation
Value
Permission
4
read
2
write
1
execute

Examples:
chmod 755 script.sh
chmod 644 config.yaml
chmod 700 id_rsa

5.2 Symbolic Notation
chmod u+x script.sh
chmod g-w config.yaml
chmod o=r file

⭐ 6. chown — Change Owner
Example:
sudo chown ashish:devops file.txt
Only change user:
sudo chown ashish file.txt
Only change group:
sudo chown :devops file.txt

⭐ 7. chgrp — Change Group
sudo chgrp devops file.txt

⭐ 8. umask — Default Permissions
Definition: Controls default permissions for new files.
Example: umask 022

⭐ 9. Special Permissions (Advanced & Important)
These are used heavily in servers.
9.1 SUID (Set User ID)
Runs file with owner’s permissions.
chmod u+s /usr/bin/passwd

Used so normal users can change passwords.
9.2 SGID (Set Group ID)
Runs file with group’s permissions OR inherited group for directories.
chmod g+s /shared

Team shared folders use this.
9.3 Sticky Bit
Only file owner can delete files in a directory.
Used in /tmp.
chmod +t /tmp

⭐ 10. DevOps Use-Cases & Real Scenarios
➡️ 10.1 Shared Deployment Folder
sudo mkdir /var/www/app
sudo chown -R deploy:devops /var/www/app
sudo chmod -R 775 /var/www/app

➡️ 10.2 CI/CD Needs Permission to Restart Services
sudo usermod -aG systemd-cgls jenkins

➡️ 10.3 Give script execution permission
chmod +x deploy.sh

➡️ 10.4 Docker volume permission fix
sudo chown -R $USER:$USER /var/lib/docker/volumes

➡️ 10.5 Prevent developers from deleting each other's files
chmod +t /project/shared

📝 Linux for DevOps — Part 5
Process, Job & Service Management
By Ashish — Learn-in-Public DevOps Journey (Week 1)
📘 Table of Contents
✔ Understanding Linux Processes
✔ ps — View Processes
✔ top / htop — Real-Time Monitoring
✔ pgrep / pidof — Find Processes
✔ kill / pkill / killall — Terminate Processes
✔ nice & renice — CPU Priority Control
✔ jobs / bg / fg — Background & Foreground Jobs
✔ nohup — Run Commands After Logout
✔ systemctl — Manage Services (Super Important)
✔ Practical DevOps Scenarios
✔ Troubleshooting One-Liners

⭐ 1. What is a Process?
A process is a running program.
Every process has:
PID (Process ID)
PPID (Parent Process ID)
UID (User who started it)
State: running, sleeping, stopped, zombie
CPU & memory usage
Start time / elapsed time

Linux uses a hierarchical process tree:
systemd → parent for everything
├─ sshd
├─ nginx
├─ docker
└─ kubelet
Understanding processes is critical during:
Outages
High CPU/memory alerts
Crashes
Deployment issues
Network failures
Resource bottlenecks
⭐ 2. ps — View Running Processes
Definition:
Displays process information.
Why DevOps uses it:
Investigate memory leaks
Check running apps
Debug crashes
Search for failed processes
Audit whether a service is running
2.1 Basic process list
ps

2.2 Full process list
ps -ef
Shows:
PID
Parent PID
Start time
Command
2.3 Detailed metrics (CPU/MEM)
ps aux
Output includes:
%CPU
%MEM
TTY
COMMAND
2.4 Filter by process name
ps aux | grep nginx
DevOps uses this daily.
⭐ 3. top — Live Process Monitoring
Definition: Interactive real-time process viewer.
Use-Cases:
High CPU investigation
Memory inspection
Identify hung/stuck processes
Live debugging during traffic spikes
Run top:
top
Useful keys:
k → kill a process
r → renice
P → sort by CPU
M → sort by memory
q → quit
⭐ 4. htop — Enhanced top
Better UI, colors, CPU graphs.
htop

Why DevOps loves it:
Mouse support
Easy process killing
Clear CPU/memory visualization

(May require installation)
sudo apt install htop
⭐ 5. pgrep — Find PIDs by Name
Definition: Search for a process by name.
Example: pgrep nginx

Return:
List of PIDs running nginx.
⭐ 6. pidof — Return PID of Program
pidof sshd

Useful for automation scripts.
⭐ 7. kill — Terminate Process by PID
Definition: Sends signal to a process.
Graceful stop:
kill PID
Force kill (dangerous):
kill -9 PID
Used when:
Program hangs
Container stuck
Service unresponsive
⭐ 8. pkill — Kill by Process Name
Example:
pkill nginx
Kills all nginx processes.
⭐ 9. killall — Kill All Processes of a Specific Command
killall python

⭐ 10. nice — Start Process with Priority
Linux priorities range:
-20 = highest priority
0 = default
+19 = lowest priority

Start with low priority:
nice -n 10 python script.py

Useful for background tasks that shouldn’t slow the server.
⭐ 11. renice — Change Priority of Running Process
Example:
renice -n -5 -p 1234
Requires root for negative values
Used when a production process needs more CPU
⭐ 12. jobs — List Background Jobs
jobs

⭐ 13. bg — Resume Job in Background
bg %1
⭐ 14. fg — Bring Job to Foreground
fg %1

⭐ 15. nohup — Run Command After Logout
Definition:Runs command immune to terminal hangups.
Output saved to nohup.out.
Example:
nohup python server.py &
Case:
Running a process through SSH and you don't want it to stop when the connection closes.
⭐ 16. systemctl — Control System Services
(Most important command for DevOps)
Definition: Manages systemd services.

16.1 Start service
sudo systemctl start nginx

16.2 Stop service
sudo systemctl stop nginx

16.3 Restart service
sudo systemctl restart nginx

16.4 Check status
sudo systemctl status nginx

16.5 Enable service on boot
sudo systemctl enable docker

16.6 Disable service
sudo systemctl disable docker

⭐ 17. Troubleshooting One-Liners (Real DevOps Commands)
➡️ Find top CPU processes:
ps aux --sort=-%cpu | head

➡️ Find memory hogs:
ps aux --sort=-%mem | head

➡️ Kill all processes of a user:
pkill -u username

➡️ Check which process is using a port:
sudo lsof -i :8080

➡️ Restart service if it's down (manual check):
! systemctl is-active --quiet nginx && systemctl restart nginx

➡️ Find zombie processes:
ps aux | grep Z

➡️ Check uptime + load:
uptime

📝 Linux for DevOps — Part 6
System Monitoring & Performance Tools
By Ashish — Learn-in-Public DevOps Journey (Week 1)
📘 Table of Contents
✔ Disk Monitoring
df, du, lsblk, fdisk, mount, umount

✔ Memory Monitoring
free, vmstat,top (already covered), /proc/meminfo
✔ CPU & System Performance
uptime, vmstat, mpstat, iostat
✔ Kernel & Logs
dmesg, journalctl
✔ System Diagnostics
watch, nproc
✔ Real DevOps Scenarios
✔ Troubleshooting One-Liners
⭐ 1. Disk Monitoring Tools
1.1 df — Disk Free Space
Definition:
Shows filesystem disk usage.
Example:
df -h
Output includes:
Filesystem
Size
Used space
Available space
Mounted on
Flags:
-h # human readable
-T # show filesystem types
DevOps Use-Cases:
Debug “No space left on device” errors
Check container volume usage
Monitor production servers
1.2 du — Directory Space Usage
Definition:
Shows how much space a directory is using.
Example:
du -sh /var/log
Flags:
-s # summary
-h # human readable
DevOps Use-Cases:
Identify log-heavy folders
Clean up container image build folders
Debug sudden disk usage spikes
1.3 lsblk — List Block Devices
Definition:
Shows attached disks, partitions & mount points.
Example:
lsblk

Output shows:
Disk names
Size
Mount points
LVM volumes
DevOps Use-Cases:
Identify EBS volumes
Identify newly attached cloud disks
Read storage layout on nodes
1.4 fdisk -l — Disk Partition Table
Definition:
Displays partition details of disks.
Example:
sudo fdisk -l
DevOps Use-Cases:
Check disk type (MBR/GPT)
Validate attached volume from AWS/Azure/GCP
Prepare disk for formatting
1.5 mount — Mount Filesystem
Example:
sudo mount /dev/sdb1 /mnt

DevOps Use-Cases:
Mount ephemeral disks
Mount EBS or Azure disks
Mount Docker volumes
Mount NFS shared drives
1.6 umount — Unmount Filesystem
Example:
sudo umount /mnt

Notes:
Cannot unmount if directory is in use.
⭐ 2. Memory Monitoring Tools
2.1 free — System Memory Usage
Example:
free -h
Shows:
Total memory
Used / free
Buffer/cache
Swap
2.2 /proc/meminfo — Detailed Memory Stats
cat /proc/meminfo
Useful for Kubernetes node debugging.
2.3 vmstat — Virtual Memory Statistics
Example:
vmstat 1 5
Columns include:
Procs
Memory
Swap
IO
System
CPU
DevOps Use-Cases:
Investigate memory leaks
Analyze CPU wait time
Troubleshoot slow disk performance
⭐ 3. CPU & System Performance Tools
3.1 uptime — Load Average
Example:
uptime
Output:
21:10:21 up 10 days, 2:32, 2 users, load average: 0.45, 0.30, 0.25
Load average values represent:
last 1 minute
last 5 minutes
last 15 minutes
3.2 mpstat — CPU Usage Per Core
mpstat -P ALL 1
(Install via: sudo apt install sysstat)
Useful for:
Debugging CPU throttling
Multi-core performance issues
3.3 iostat — Disk IO Statistics
iostat -xz 1
Shows:
Disk read/write
IO wait
Utilization
⭐ 4. Kernel Logs & System Logs
4.1 dmesg — Kernel Messages
Example:
dmesg | tail
Use-Cases:
Disk attachment logs
USB failure logs
Kernel crashes
OOM (out of memory) events
4.2 journalctl — Systemd Logs
Example:
journalctl -u nginx
Flags:
-f # follow logs
-x # detailed info
DevOps Use-Cases:
Service boots
Service crashes
Authentication failures
Systemd service debugging

⭐ 5. Diagnostics & Utility Tools
5.1 watch — Repeat Command Continuously
Example:
watch -n 3 df -h
Updates every 3 seconds.
5.2 nproc — Number of CPU Cores
nproc
Used for:
Optimizing nginx workers
Docker resource tuning
CI/CD parallel jobs
⭐ 6. Real DevOps Scenarios (Critical)
➡️ 6.1 Disk is full — identify culprit
df -h
du -sh /* | sort -h
du -sh /var/* | sort -h

➡️ 6.2 High CPU — find top processes
ps aux --sort=-%cpu | head

➡️ 6.3 Memory leak — monitor live
watch -n 1 free -h

➡️ 6.4 Identify high IO usage
iostat -xz 1

➡️ 6.5 Kernel-level disk errors
dmesg | grep -i error

➡️ 6.6 Service troubleshooting
systemctl status nginx
journalctl -u nginx -f

⭐ 7. Troubleshooting One-Liners
Check uptime + load:
uptime

Find failed systemd services:
systemctl --failed

Identify biggest folders:
du -ah / | sort -h | tail

Check disk type:
lsblk -f

Find swap usage:
free -m | grep Swap

This is a critical article for your Hashnode audience, especially SRE & DevOps learners.

📝 Linux for DevOps — Part 7
Networking Commands (Complete Practical Guide + Real DevOps Debugging)
By Ashish — Learn-in-Public DevOps Journey (Week 1)
📘 Table of Contents
✔ Basics
Hostname, hostnamectl, ip, ifconfig, iwconfig,mac address
✔ Connectivity Tests
ping, traceroute, tracepath, mtr, curl, wget, telnet, nc
✔ Ports, Sockets, and Firewalls
ss, netstat, nmap, iptables, route, arp
✔ DNS Tools
dig, nslookup, host, whois
✔ Network Interface & IP Management
ip a, ip r, ip link, ifplugstatus
✔ Real DevOps Scenarios
✔ Troubleshooting One-Liners (Production-Grade)
⭐ 1. Host & Interface Identification
1.1 hostname — Show Hostname
Example:
hostname
Set hostname:
sudo hostname server-01

1.2 hostnamectl — Detailed System Identity
hostnamectl

Shows:
OS version
Kernel
Architecture
Machine name
⭐ 2. Interface & IP Address Commands
2.1 ip a — Show All Network Interfaces (Modern tool)
Example:
ip a

Shows:
All interfaces
IP addresses
MAC addresses
Operational state
Used 100x more than ifconfig in modern systems.
2.2 ifconfig — Legacy Interface Tool
ifconfig

Still used in:
Older servers
Some Docker containers
Network debugging tools
2.3 iwconfig — Wireless Config
iwconfig
Useful for laptops but not needed on cloud servers.
⭐ 3. Connectivity Testing Tools (Super Critical)
3.1 ping — Test Server Reachability
Example:
ping google.com
Use-Cases:
DNS working?
Server reachable?
Packet loss?
Latency check?
3.2 traceroute — Trace Hops to Destination
Example:
traceroute google.com

Used for:
Network path debugging
CDN routing issues
Cloud region traffic debugging
3.3 tracepath — Trace Without Root Privileges
tracepath google.com
Safer alternative.
3.4 mtr — Real-Time Traceroute + Ping (Combined)
mtr google.com

This is used heavily for debugging:
packet loss
network jitter
unstable cloud paths
⭐ 4. DNS Tools
4.1 dig — DNS Queries (Most Powerful)
Example:
dig google.com

Check full DNS chain:
dig +trace google.com

Check specific record:
dig A google.com
dig MX gmail.com
dig TXT domain.com

4.2 nslookup — Query DNS Records
nslookup google.com

4.3 whois — Domain Ownership Info
whois google.com

Useful for:
domain expirations
registrar details
⭐ 5. Ports, Sockets & Connections
5.1 ss — Socket Statistics (Modern & Fast)
Better than netstat.
Example:
ss -tulnp
Flags:
t → TCP
u → UDP
l → listening
n → numeric
p → process
Use-Cases:
Check if service is listening
Troubleshoot “port already in use”
Identify rogue processes

5.2 netstat — Legacy Socket Tool
netstat -tulnp

Still useful on older Linux.
5.3 nmap — Port Scanner
nmap
Use-cases:
Check open ports
Verify firewall rules
Security scanning
⭐ 6. HTTP/HTTPS & API Tools
6.1 curl — API & Endpoint Testing
Examples:
curl google.com
curl -I https://example.com
curl -X POST -d "a=1" https://api.com
Why DevOps uses it:
Test API health
Debug load balancers
Check SSL certificates
Validate domain routing
6.2 wget — File Download Tool
wget https://example.com/file.zip

⭐ 7. Debugging TCP/UDP & Connectivity
7.1 telnet — Check Port Connectivity (Old but Used)
telnet server 443
If it connects → port open.
7.2 nc (netcat) — Swiss Army Knife of Networking
Check if port open:
nc -zv server 22
Create simple server:
nc -l 8080
Transfer files:
nc -l 1234 > file.txt

⭐ 8. Routing & ARP Tables
8.1 route — Show Routing Table
route -n

8.2 ip r — Modern Routing Command
ip route
8.3 arp — MAC → IP Mapping
arp -a
Used to debug:
Local network
Duplicate IP issues
ARP poisoning checks
⭐ 9. Network Status & Interface Health
9.1 ifplugstatus — Cable Plug Status
ifplugstatus

Used for:
Bare metal servers
On-prem deployments
PXE boot debugging
⭐ 10. REAL-WORLD DEVOPS SCENARIOS
➡️ 10.1 Check if backend API reachable
curl -I http://backend:8080/health

➡️ 10.2 Debug Kubernetes Pod Networking
Inside pod:
curl http://service-name:8080
nslookup service-name

➡️ 10.3 Identify which process blocked a port
ss -tulnp | grep 8080

➡️ 10.4 Validate DNS routing
dig +trace domain.com

➡️ 10.5 Verify firewall allowing traffic
nc -zv server 443

➡️ 10.6 Check latency issues
mtr google.com

⭐ 11. TROUBLESHOOTING ONE-LINERS
Check open ports:
ss -tulnp

Check DNS poison:
dig domain.com @8.8.8.8

Check if port blocked by firewall:
iptables -L -n | grep 22

List all IPs of system:
ip -o -4 addr show | awk '{print $4}'

Test HTTPS certificate expiry:
echo | openssl s_client -servername domain.com -connect domain.com:443 | openssl x509 -noout -dates

📝 Linux for DevOps — Part 8
Disk, Filesystem & LVM Management (Complete Guide)
By Ashish — Learn-in-Public DevOps Journey (Week 1)
📘 Table of Contents
✔ Understanding Linux Storage Architecture
✔ Disk Information Commands
lsblk, fdisk, blkid, file, df, du
✔ Partitioning & Filesystems
fdisk, parted,mkfs, mount, umount
✔ LVM (Logical Volume Manager)
pvcreate, vgcreate, lvcreate, lvs / pvs / vgs, extend LV & Filesystem
✔ Swap Management
mkswap, swapon / swapoff
✔ Real DevOps Scenarios
✔ Troubleshooting One-Liners
⭐ 1. Understanding Linux Storage Architecture
Linux storage layers:
+----------------------------+
| Application (Nginx, DBs) |
+----------------------------+
| Filesystem (ext4, xfs) |
+----------------------------+
| Logical Volume (LV) |
+----------------------------+
| Volume Group (VG) |
+----------------------------+
| Physical Volume (PV) |
+----------------------------+
| Disk / Cloud Volume |
+----------------------------+

Cloud mapping:
Provider
Storage
AWS
EBS volumes
Azure
Managed Disks
GCP
Persistent Disks
K8s
PV/PVC

⭐ 2. Disk Information Commands
2.1 lsblk — List Block Devices
Definition:
Shows all disks, partitions, and LVM structures.
Example:
lsblk

Shows:
Disk name
Size
Type (disk/part/lvm)
Mount point
Why DevOps uses it:
After attaching EBS/Azure disks
Identify disk names (/dev/sda, /dev/nvme1n1)
Validate LVM structure
2.2 fdisk -l — List Partition Tables
sudo fdisk -l
Useful for:
Check if disk is formatted
Identify GPT or MBR
Inspect partition alignment
2.3 blkid — Show UUIDs & Filesystem Types
blkid

Used for:
/etc/fstab configuration
Mounting disks by UUID
Identifying filesystem: ext4/xfs/btrfs
2.4 file — Check File Type
file /dev/sdb
Can detect special block files.
2.5 df -h — Filesystem Disk Usage
df -h
Used for:
Out-of-disk emergencies
CI/CD cache filling
K8s node disk pressure
2.6 du -sh — Directory Space Usage
du -sh /var/log

Used to find large folders.
⭐ 3. Partitioning & Formatting Disks
3.1 Creating Partition using fdisk
Step-by-step:
sudo fdisk /dev/sdb

Inside fdisk:
Key
Meaning
n
create partition
p
print table
d
delete partition
w
write changes
q
quit

After partition:
lsblk

3.2 Formatting the Partition (mkfs)
Format as ext4:
sudo mkfs.ext4 /dev/sdb1

Format as XFS:
sudo mkfs.xfs /dev/sdb1

3.3 Mounting Filesystems
Temporary mount:
sudo mount /dev/sdb1 /mnt

Check:
df -h | grep /mnt

3.4 Unmounting
sudo umount /mnt

3.5 Persistent Mount (Permanent)
Add to /etc/fstab:
UUID= /data ext4 defaults 0 2

Get UUID:
blkid

⭐ 4. LVM — Logical Volume Manager (Production Standard)
LVM allows:
Resize disks without downtime
Combine multiple disks
Create flexible storage pools
⭐ 4.1 Step 1 — Create Physical Volume (PV)
sudo pvcreate /dev/sdb
Check:
pvs

⭐ 4.2 Step 2 — Create Volume Group (VG)
sudo vgcreate vg_data /dev/sdb
Check:
vgs

⭐ 4.3 Step 3 — Create Logical Volume (LV)
sudo lvcreate -L 10G -n lv_storage vg_data

Check:
lvs
⭐ 4.4 Step 4 — Create Filesystem on LV
sudo mkfs.ext4 /dev/vg_data/lv_storage

⭐ 4.5 Step 5 — Mount LV
sudo mount /dev/vg_data/lv_storage /mnt

⭐ 4.6 Extend Logical Volume (VERY IMPORTANT)
Extend LV:
sudo lvextend -L +5G /dev/vg_data/lv_storage
Resize filesystem (ext4):
sudo resize2fs /dev/vg_data/lv_storage
For XFS:
sudo xfs_growfs /mnt

⭐ 5. Swap Management
5.1 Create Swap
sudo mkswap /dev/sdb2

5.2 Enable Swap
sudo swapon /dev/sdb2

5.3 Disable Swap
sudo swapoff /dev/sdb2
Used during Kubernetes node tuning.
⭐ 6. Real DevOps Scenarios
➡️ 6.1 Attach & Configure AWS EBS Volume
lsblk
fdisk /dev/nvme1n1
mkfs.ext4 /dev/nvme1n1p1
mount /dev/nvme1n1p1 /data
➡️ 6.2 Expand disk on running server
AWS/GCP:
Increase disk size in console
Then on Linux:
sudo growpart /dev/sda 1
sudo resize2fs /dev/sda1

➡️ 6.3 LVM Expand live volume
pvcreate /dev/sdc
vgextend vg_data /dev/sdc
lvextend -l +100%FREE /dev/vg_data/lv_storage
resize2fs /dev/vg_data/lv_storage

➡️ 6.4 Find which folder filled the disk
du -sh /* | sort -h

➡️ 6.5 Unmount busy filesystem
lsof | grep /mnt

⭐ 7. Troubleshooting One-Liners
Check disk health:
dmesg | grep sdb

Show only mounted filesystems:
df -h | grep -v tmpfs

Check disk type:
lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINT

Check sector size:
sudo blockdev --getbsz /dev/sdb

Check IO wait:
iostat -xz 1

📝 Linux for DevOps — Part 9
Editors, Shell Scripting & Automation
By Ashish — Learn-in-Public DevOps Journey
📘 Table of Contents
✔ Editors
nano, vim (full cheatsheet)
✔ Shell Scripting Basics
What is a shell script?
Basic syntax
Comments
Variables
Input/output
Redirections
✔ Control Flow
if/else, case, loops (for / while / until)
✔ Functions
✔ Exit codes
✔ Error handling (set -e, set -x)
✔ Cron jobs
✔ Real DevOps Automation Examples

⭐ 1. Editors for DevOps
1.1 nano — Simple Editor
Definition: Beginner-friendly editor for modifying small config files.
Example:
nano app.conf

Common shortcuts:
Ctrl + O → save
Ctrl + X → exit
Ctrl + K → cut line
Ctrl + U → paste

DevOps Use:
Quick edits inside containers
Modifying environment files
Editing config files on SSH
1.2 vim — Most Important DevOps Editor
Vim is lightweight, always available, and perfect for editing files through SSH.
Modes:
Mode
Purpose
Normal
navigation + commands
Insert
editing text
Command
saving, quitting, search

Switch modes:
i → insert
Esc → normal
: → command
⭐ VIM COMMAND CHEATSHEET
Save & Quit:
:w
:wq
:q!
Navigation:
h j k l # left, down, up, right
0 # start of line
$ # end of line
gg # beginning of file
G # end of file
:n # go to line n

Editing:
x # delete char
dd # delete line
yy # copy
p # paste
u # undo
Ctrl+r # redo

Search:
/pattern
n
N

Replace:
:%s/old/new/g
Vim mastery = DevOps mastery.
⭐ 2. Shell Scripting Fundamentals
A shell script = a sequence of commands in a file executed by the Bash shell.
2.1 Create Script File
touch script.sh
chmod +x script.sh
nano script.sh

2.2 Basic Script Structure

!/bin/bash

echo "Hello from DevOps"

!/bin/bash → interpreter

Commands run line by line
Run it:
./script.sh

⭐ 3. Comments

This is a comment

⭐ 4. Variables
User-defined:
name="Ashish"
echo "Hello $name"
System variables:
echo $HOME
echo $USER
echo $PATH

⭐ 5. Input from User
read -p "Enter your name: " username
echo "Welcome $username"

⭐ 6. Output Redirection
echo "log entry" > file.txt # overwrite
echo "append entry" >> file.txt # append

⭐ 7. If-Else Conditions
if [ $age -ge 18 ]; then
echo "Adult"
else
echo "Minor"
fi

File checks:
if [ -f file.txt ]; then
echo "File exists"
fi

⭐ 8. Case Statement
read -p "Choose option: " opt

case $opt in
1) echo "Start service" ;;
2) echo "Stop service" ;;
*) echo "Invalid" ;;
esac

⭐ 9. Loops
9.1 For Loop
for i in {1..5}
do
echo "Number: $i"
done

9.2 While Loop
count=1
while [ $count -le 5 ]
do
echo "Loop $count"
count=$((count+1))
done

9.3 Until Loop
until [ $n -gt 5 ]
do
echo "n = $n"
n=$((n+1))
done

⭐ 10. Functions
deploy_app() {
echo "Deploying application..."
}

deploy_app

⭐ 11. Exit Codes
echo $? # shows exit code of last command

0 = success
Non-zero = error
⭐ 12. Script Debugging
Print each command:
set -x
Exit on error:
set -e
Combine:
set -xe
Used in CI/CD pipelines.
⭐ 13. Cron Jobs (Automation)
Edit cron tab:
crontab -e
Run script daily at midnight:
0 0 * * * /home/ashish/backup.sh

⭐ 14. REAL DEVOPS AUTOMATION SCRIPTS
➡️ 14.1 Cleanup old logs

!/bin/bash

find /var/log -type f -mtime +7 -delete

➡️ 14.2 Restart service on failure

!/bin/bash

if ! systemctl is-active --quiet nginx; then
systemctl restart nginx
fi

➡️ 14.3 Create user automatically

!/bin/bash

user=$1
sudo useradd -m $user
echo "$user created"

➡️ 14.4 Copy files to server (CI/CD)
rsync -avz build/ user@server:/var/www/app/

➡️ 14.5 Backup database

!/bin/bash

mysqldump -u root -p dbname > backup.sql

DevOps Quick Recipes & Troubleshooting Toolkit (Ultimate Guide)
By Ashish — Learn-in-Public DevOps Journey (Week 1)
📘 Table of Contents
✔ On-Call Survival Commands
✔ Log Debugging
✔ Disk Pressure & Out-of-Disk Errors
✔ Network Trouble Commands
✔ CPU/Memory Debugging
✔ Permission Issues
✔ Deployment & CI/CD Helpers
✔ SSH & Connectivity
✔ Docker / Container Debugging
✔ Must-Know One-Liners
⭐ 1. On-Call Survival Commands
These are the commands you will use when something breaks in production.
➡️ Check system load
uptime

➡️ Check which processes are eating CPU
ps aux --sort=-%cpu | head

➡️ Check which processes are eating RAM
ps aux --sort=-%mem | head

➡️ Check available memory
free -h

➡️ Live memory monitoring
watch -n 1 free -h

⭐ 2. Log Debugging (MOST IMPORTANT)
➡️ Tail logs live:
tail -f /var/log/syslog

➡️ Filter errors:
grep -i "error" /var/log/syslog

➡️ Filter warnings:
grep -i "warn" app.log

➡️ Watch Nginx access logs:
tail -f /var/log/nginx/access.log

➡️ Watch only failed nginx responses:
grep " 500 " access.log

⭐ 3. Disk Space & “No space left on device” Troubleshooting
➡️ Check disk usage:
df -h

➡️ Find biggest directories:
du -sh /* | sort -h | tail

➡️ Check /var/log size:
du -sh /var/log/*

➡️ Find largest files:
find / -type f -exec du -Sh {} + | sort -rh | head -n 20

➡️ Clear apt cache:
sudo apt autoremove
sudo apt clean

⭐ 4. Network Troubleshooting
➡️ Check connectivity:
ping google.com

➡️ Show local IPs:
ip a

➡️ Check DNS resolution:
dig google.com
nslookup google.com

➡️ Check which process is using port 80:
ss -tulnp | grep :80

➡️ Check routing table:
ip route

⭐ 5. CPU & Memory Troubleshooting
➡️ Check CPU usage live:
top

➡️ Check multi-core CPU usage:
mpstat -P ALL 1

➡️ Check disk IO pressure:
iostat -xz 1

⭐ 6. Permission Issues
➡️ Fix permission denied:
sudo chown -R $USER:$USER /path

➡️ Give execution permission:
chmod +x script.sh

➡️ Check file permissions:
ls -l file.txt

⭐ 7. Deployment & CI/CD Helpers
➡️ Sync build folder to server:
rsync -avz build/ user@server:/var/www/app/
➡️ Copy file to server:
scp -i key.pem file user@server:/path/
➡️ Restart service:
sudo systemctl restart nginx
➡️ Check service logs:
journalctl -u nginx -f
⭐ 8. SSH & Server Connectivity
➡️ Login into server:
ssh -i key.pem user@server
➡️ Test open port via nc:
nc -zv server 443
➡️ Keep process alive after logout:
nohup command &
⭐ 9. Docker/Container Troubleshooting
➡️ Check container logs:
docker logs container
➡️ Exec into container:
docker exec -it container bash
➡️ Check container IP:
docker inspect -f '{{ .NetworkSettings.IPAddress }}' container
➡️ Clean dangling images:
docker image prune -f
⭐ 10. Kubernetes-Specific Handy Commands (Bonus)
➡️ Check pod logs:
kubectl logs pod-name
➡️ Exec into pod:
kubectl exec -it pod -- bash
➡️ Check service endpoints:
kubectl get endpoints service
➡️ Debug DNS inside pod:
kubectl exec -it pod -- nslookup google.com
➡️ Check node resource usage:
kubectl top node
⭐ 11. Ultimate DevOps One-Liners
➡️ Restart crashed service:
! systemctl is-active --quiet app && systemctl restart app
➡️ Send alert if disk is > 90%:
df -h | awk '$5+0 > 90 {print $0}'
➡️ Extract top 10 IPs hitting server:
awk '{print $1}' access.log | sort | uniq -c | sort -nr | head
➡️ Check memory hogs:
ps aux --sort -rss | head
➡️ Find configs containing secret word:
grep -r "password" /etc

Thanks for Reading, Join Me on This DevOps Journey
Thank you for reading Linux for DevOps of my Learn-in-Public DevOps Series.
If you’re just starting your DevOps journey, or restarting it like I am, I hope these notes give you clarity, direction, and confidence.
I’m documenting everything openly so we can grow together, step by step.
If this helped you, follow along — I’ll publish the next part soon.
Let’s build a strong DevOps foundation, one skill at a time.
Linkedin Profile: https://www.linkedin.com/in/ashish360/