Forem: Selma Guedidi

How to use Kubectl more efficiently

Selma Guedidi — Mon, 04 May 2026 11:26:53 +0000

Selma Guedidi

Apr 29

kubectl Hacks That Changed How I Work With Kubernetes

#kubernetes #containers #devops #automation

Comments

12 min read

kubectl Hacks That Changed How I Work With Kubernetes

Selma Guedidi — Wed, 29 Apr 2026 19:27:03 +0000

Beyond the basics: patching, waiting, explaining, and squeezing the most out of the one tool that never leaves your terminal.

Most kubectl guides stop at get pods and apply -f. That's fine for getting started, but after years of running Kubernetes in production, the commands that make the real difference are the ones nobody writes about. The obscure flags. The output tricks. The commands that turn a 20-minute debugging slog into a 90-second fix.

This article is about the depth of kubectl: commands I reach for when things get weird, scripting patterns that made our CI pipelines solid, and the shorthand vocabulary that turns the tool from a keyboard workout into something that actually feels fast. No fluff, no padding. Just the stuff worth knowing.

01 — The Complete Shorthand Vocabulary

Before anything else: the complete list of resource type abbreviations. These are baked into the Kubernetes API itself, so they work universally across get, describe, delete, patch, edit, and every other verb. Memorise the ones you use daily and your hands will thank you.

Short	Full Resource Name	API Group
`po`	pods	core
`svc`	services	core
`deploy`	deployments	apps
`ds`	daemonsets	apps
`rs`	replicasets	apps
`sts`	statefulsets	apps
`ns`	namespaces	core
`cm`	configmaps	core
`ing`	ingresses	networking.k8s.io
`pv`	persistentvolumes	core
`pvc`	persistentvolumeclaims	core
`sa`	serviceaccounts	core
`ep`	endpoints	core
`no`	nodes	core
`ev`	events	core
`hpa`	horizontalpodautoscalers	autoscaling
`pdb`	poddisruptionbudgets	policy
`crd`	customresourcedefinitions	apiextensions.k8s.io
`crb`	clusterrolebindings	rbac.authorization.k8s.io
`rb`	rolebindings	rbac.authorization.k8s.io
`netpol`	networkpolicies	networking.k8s.io
`sc`	storageclasses	storage.k8s.io
`job`	jobs	batch
`cj`	cronjobs	batch
`quota`	resourcequotas	core
`sec`	secrets	core

💡 Pro tip: Run kubectl api-resources in your cluster to see every available shorthand, including those from installed CRDs. It shows the short name, API group, whether the resource is namespaced, and the kind.

02 — kubectl patch: The Surgeon's Tool

kubectl apply is great when you own the full manifest. But what about changing a single field on a resource managed by a Helm chart, an operator, or another team? You don't want to touch the whole file. You want a scalpel. That's kubectl patch.

There are three patch types, and picking the wrong one will lose you data or produce confusing merges.

Type	Behaviour	When to use
`strategic-merge`	Kubernetes-aware merge. Containers are merged by name, not replaced.	Default. Use most of the time.
`merge`	RFC 7396 JSON Merge. Arrays are replaced entirely, not merged.	CRDs and non-core resources.
`json`	RFC 6902. Precise add / remove / replace / move operations.	When you need surgical precision.

# Add an annotation without touching the rest of the manifest
kubectl patch deploy api-gateway -p '{"metadata":{"annotations":{"deploy-time":"2026-04-27"}}}' -n production

# Change replica count on the fly (useful in incidents)
kubectl patch deploy api-gateway -p '{"spec":{"replicas":6}}' -n production

# Remove a field entirely with JSON patch
kubectl patch deploy api-gateway --type=json \
  -p='[{"op":"remove","path":"/spec/template/spec/containers/0/resources/limits/cpu"}]' -n production

# Replace a specific env var value by index
kubectl patch deploy api-gateway --type=json \
  -p='[{"op":"replace","path":"/spec/template/spec/containers/0/env/0/value","value":"https://new-db-host"}]'

# Cordon a node : what kubectl cordon does under the hood
kubectl patch node ip-10-0-1-47 -p '{"spec":{"unschedulable":true}}'

# Add a label to a node using json patch
kubectl patch node ip-10-0-1-47 --type=json \
  -p='[{"op":"add","path":"/metadata/labels/topology.kubernetes.io~1zone","value":"us-east-1a"}]'

⚠️ Watch out: The / character in JSON Patch paths refers to object keys. If your key contains a forward slash (like kubernetes.io/role), escape it as ~1. The tilde ~ itself escapes as ~0. Easy to forget, expensive when wrong.

03 — kubectl wait: The CI/CD Game Changer

If you write shell scripts or CI pipelines that deploy to Kubernetes, you've probably written something like: deploy → sleep 30 → check status → sleep more → give up. It's flaky and slow. kubectl wait replaces all of that with a proper event-driven block.

# Block until a deployment finishes rolling out (timeout after 3 minutes)
kubectl wait deploy/api-gateway --for=condition=Available --timeout=3m -n production

# Wait for all pods with a label to be Ready
kubectl wait pods -l app=api-gateway --for=condition=Ready --timeout=2m -n production

# Wait for a Job to complete
kubectl wait job/db-migration --for=condition=Complete --timeout=10m -n production

# Wait for a Job to fail (useful for catching broken migrations early)
kubectl wait job/db-migration --for=condition=Failed --timeout=10m -n production

# Wait for a node to be Ready after a restart
kubectl wait node/ip-10-0-1-47 --for=condition=Ready --timeout=5m

# Wait for a CRD to be established (important in Helm chart init containers)
kubectl wait crd/certificates.cert-manager.io --for=condition=Established --timeout=60s

# Wait for a pod to be deleted
kubectl wait pod/old-pod-abc123 --for=delete --timeout=60s -n production

The --for=delete form is especially valuable. Whenever you need to drain a node or delete a pod and then take some action only after it's truly gone, this is your signal. No polling loops, no racy sleep calls.

💡 CI pattern: Chain kubectl apply and kubectl wait in your pipeline. If the wait times out, the pipeline fails with a non-zero exit code, no extra health-check scripts needed. You get free deploy verification.

04 — kubectl explain: The Docs Are Already in Your Terminal

Every time I see someone switching to a browser to look up what a field does in a Kubernetes spec, I cringe. The full API reference is built right into kubectl explain. It supports dot-notation for nested fields and --recursive to print the entire schema tree.

# What is a PodSpec?
kubectl explain pod.spec

# What fields does a container accept?
kubectl explain pod.spec.containers

# Go deep on a specific field
kubectl explain pod.spec.containers.resources.requests

# Print the entire schema tree for a resource
kubectl explain deployment --recursive

# Works on CRDs too
kubectl explain certificate.spec.renewBefore

# Check what API version a resource uses
kubectl explain horizontalpodautoscaler --api-version=autoscaling/v2

The --recursive output looks dense at first but becomes invaluable when you're unfamiliar with a CRD's schema. Pipe it through grep to find the field you care about rather than reading a thousand-line GitHub README.

05 — kubectl cp: Moving Files In and Out of Containers

This one saves you in two specific scenarios: extracting a generated file from a container (a rendered config, a certificate, a data dump), and pushing a temporary script into a running pod when you can't rebuild the image. It behaves like scp with a slightly different path syntax.

# Copy a file OUT of a container
kubectl cp production/api-pod-7d4f8b:/app/logs/error.log ./error.log

# Copy a file INTO a container
kubectl cp ./debug-script.sh production/api-pod-7d4f8b:/tmp/debug-script.sh

# Copy a whole directory out
kubectl cp production/api-pod-7d4f8b:/app/data ./data-backup

# Specify a container in a multi-container pod
kubectl cp production/api-pod-7d4f8b:/var/log/app.log ./app.log -c sidecar-logger

# Format: [namespace/]pod-name:path

⚠️ Heads up: kubectl cp requires tar to be present inside the container. Distroless images don't have it. If you hit that wall, use kubectl debug to attach a debug sidecar first, then copy from there.

06 — Generating YAML Without Writing YAML

The --dry-run=client -o yaml pattern lets kubectl write your manifests for you. Here's a full set of imperative-to-declarative shortcuts that produce valid, committable YAML.

# Deployment manifest
kubectl create deploy api-gateway --image=ghcr.io/myorg/api:v2.3 --replicas=3 \
  --dry-run=client -o yaml > deployment.yaml

# Service (ClusterIP)
kubectl create svc clusterip api-gateway --tcp=80:8080 --dry-run=client -o yaml

# Service (LoadBalancer)
kubectl create svc loadbalancer api-lb --tcp=443:8443 --dry-run=client -o yaml

# ConfigMap from file and literals
kubectl create cm app-config \
  --from-file=config.yaml \
  --from-literal=LOG_LEVEL=info \
  --from-literal=REGION=eu-west-1 \
  --dry-run=client -o yaml

# Secret (generic)
kubectl create secret generic db-creds \
  --from-literal=username=admin \
  --from-literal=password=hunter2 \
  --dry-run=client -o yaml

# Secret (TLS)
kubectl create secret tls api-tls --cert=tls.crt --key=tls.key --dry-run=client -o yaml

# Job
kubectl create job db-migration --image=ghcr.io/myorg/migrator:v1 --dry-run=client -o yaml

# CronJob
kubectl create cronjob nightly-report --image=ghcr.io/myorg/reporter:v1 \
  --schedule="0 2 * * *" --dry-run=client -o yaml

# ServiceAccount
kubectl create sa api-worker --dry-run=client -o yaml

# Role
kubectl create role pod-reader --verb=get,list,watch --resource=pods --dry-run=client -o yaml

# RoleBinding
kubectl create rolebinding api-pod-reader \
  --role=pod-reader --serviceaccount=production:api-worker \
  --dry-run=client -o yaml

07 — kubectl set: In-Place Updates Without Editing YAML

When you need to change a single well-defined property on a running resource, kubectl set is cleaner than patching raw JSON.

# Update the image of a single container
kubectl set image deploy/api-gateway api=ghcr.io/myorg/api:v2.4 -n production

# Update multiple containers at once
kubectl set image deploy/api-gateway api=ghcr.io/myorg/api:v2.4 sidecar=ghcr.io/myorg/proxy:v1.1

# Set resource requests and limits
kubectl set resources deploy/api-gateway \
  -c api \
  --requests=cpu=250m,memory=512Mi \
  --limits=cpu=1000m,memory=1Gi -n production

# Assign a service account
kubectl set serviceaccount deploy/api-gateway api-worker -n production

# Add or update an environment variable
kubectl set env deploy/api-gateway LOG_LEVEL=debug -n production

# Set env from a ConfigMap
kubectl set env deploy/api-gateway --from=configmap/app-config -n production

# Set env from a Secret
kubectl set env deploy/api-gateway --from=secret/db-creds -n production

# Remove an env var (suffix with a minus)
kubectl set env deploy/api-gateway LOG_LEVEL- -n production

ℹ️ Note: kubectl set image records the change in the deployment's rollout history, which means you can kubectl rollout undo it just like a kubectl apply. It's not a dirty in-place hack, it's a first-class operation.

08 — Labels, Annotations, and Taints: The Metadata Commands

Labels and annotations are the connective tissue of Kubernetes. Services find their pods through labels. Operators read annotations for config. Knowing how to manipulate this metadata live, without opening a YAML file, is surprisingly powerful.

# Add a label to a pod
kubectl label pod api-pod-7d4f8b version=v2.4 -n production

# Overwrite an existing label
kubectl label pod api-pod-7d4f8b version=v2.5 -n production --overwrite

# Remove a label (suffix with a minus)
kubectl label pod api-pod-7d4f8b version- -n production

# Label all pods matching a selector
kubectl label pods -l app=api-gateway canary=true -n production

# Label a node for topology-aware scheduling
kubectl label node ip-10-0-1-47 disk=ssd zone=us-east-1a

# Add an annotation
kubectl annotate deploy/api-gateway \
  deployment.kubernetes.io/revision-message="hotfix: null pointer on /checkout" -n production

# Taint a node so only tolerating pods schedule on it
kubectl taint node ip-10-0-1-47 dedicated=gpu-workloads:NoSchedule

# Remove a taint (suffix the effect with a minus)
kubectl taint node ip-10-0-1-47 dedicated:NoSchedule-

# Cordon and drain before maintenance
kubectl cordon ip-10-0-1-47
kubectl drain ip-10-0-1-47 --ignore-daemonsets --delete-emptydir-data --force
kubectl uncordon ip-10-0-1-47  # after maintenance

09 — Output Tricks: Beyond the Default Table

The default tabular output is readable, but when you're scripting or building dashboards, you need exact fields. Here are the output modes worth knowing:

Flag	What it does
`-o yaml`	Full resource as YAML. Great for inspection and editing.
`-o json`	Full resource as JSON. Pipe to `jq` for advanced queries.
`-o name`	`resource/name` format. Ideal for piping into delete or apply.
`-o jsonpath`	Extract specific fields. Best for scripts and alerts.
`-o wide`	Extra columns: node, IP, nominated node, readiness gates.
`-o custom-columns`	Define your own table headers and field paths.

# Pull a single field cleanly for scripting
kubectl get deploy api-gateway -o jsonpath='{.spec.replicas}' -n production

# Get all image names running across a namespace
kubectl get pods -n production \
  -o jsonpath='{range .items[*]}{.metadata.name}{"  "}{range .spec.containers[*]}{.image}{"\n"}{end}{end}'

# Get the cluster API server URL
kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'

# Custom table: pod name, QoS class, and CPU limit
kubectl get pods -n production -o custom-columns=\
'NAME:.metadata.name,QOS:.status.qosClass,CPU-LIMIT:.spec.containers[0].resources.limits.cpu'

# Pipe to jq for complex queries
kubectl get pods -n production -o json | \
  jq '.items[] | select(.status.phase=="Running") | .metadata.name'

# Find all images across all namespaces (deduplicated)
kubectl get pods -A -o jsonpath='{range .items[*]}{.spec.containers[*].image}{"\n"}{end}' | \
  tr ' ' '\n' | sort -u

# Use -o name to feed into another command
kubectl get pods -l app=api-gateway -n production -o name | \
  xargs -I{} kubectl annotate {} debug-session=2026-04-27 -n production

10 — kubectl proxy: Accessing the API Directly

kubectl proxy starts a local HTTP server that proxies authenticated requests to the Kubernetes API server. It's different from port-forward: proxy gives you access to the Kubernetes REST API itself, not to a specific service inside the cluster.

# Start a proxy on localhost:8001 (default)
kubectl proxy

# Now you can curl the API without any auth headers
curl http://localhost:8001/api/v1/namespaces/production/pods

# Access cluster metrics via the metrics API
curl http://localhost:8001/apis/metrics.k8s.io/v1beta1/nodes

# Access a service through the API proxy
# Format: /api/v1/namespaces/{ns}/services/{scheme:name:port}/proxy/
curl http://localhost:8001/api/v1/namespaces/monitoring/services/http:grafana:3000/proxy/

# Proxy on a specific port
kubectl proxy --port=9090

# Accept connections from all interfaces (remote dev environments)
kubectl proxy --address=0.0.0.0 --accept-hosts='.*'

The service proxy URL pattern is especially useful for accessing cluster-internal dashboards (Grafana, Prometheus, Argo CD) without setting up ingress or port-forward sessions. Keep the proxy running in a background terminal and bookmark the URLs.

11 — The Plugin Ecosystem: What krew Unlocks

kubectl's plugin system lets you drop any executable named kubectl-something onto your PATH and run it as kubectl something. krew is the community package manager for these plugins. Here are the ones that actually earn their place:

Debugging

kubectl neat — Strip managed fields from get -o yaml output so it's actually readable
kubectl images — List container images in a tree view
kubectl resource-capacity — Node capacity vs requests vs limits at a glance

Networking

kubectl ns — Switch namespace quickly (kubens alternative)
kubectl sniff — Capture pod traffic with tcpdump/Wireshark
kubectl mtail — Multi-pod log tail with regex

Security

kubectl who-can — Who has permission to do X?
kubectl access-matrix — Full RBAC matrix for a namespace
kubectl popeye — Cluster configuration linter

Productivity

kubectl ctx — Switch cluster context (kubectx alternative)
kubectl tree — Show owner references as a tree
kubectl view-secret — Decode secrets without base64 manually

# Install krew (macOS/Linux)
(
  set -x; cd "$(mktemp -d)" &&
  OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
  ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
  KREW="krew-${OS}_${ARCH}" &&
  curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&
  tar zxvf "${KREW}.tar.gz" &&
  ./"${KREW}" install krew
)

# Install the ones worth having
kubectl krew install neat who-can access-matrix resource-capacity view-secret tree images popeye sniff

# neat: removes clutter from -o yaml output (managedFields, creationTimestamp, etc.)
kubectl get deploy api-gateway -o yaml | kubectl neat

# who-can: find out which subjects can perform an action
kubectl who-can delete pods -n production

# view-secret: decode and display a secret's values
kubectl view-secret db-creds -n production

# tree: visualise the ownership hierarchy
kubectl tree deploy api-gateway -n production

# resource-capacity: see requests/limits vs actual usage per node
kubectl resource-capacity --sort cpu.limit --pods

12 — Watching Resources: Smarter Than You Think

The -w flag on kubectl get opens a watch stream against the API server. Most people use it for pods, but it works on any resource. Combined with field selectors and output formatting, it becomes a real-time monitoring tool without needing a dashboard.

# Watch pods but only show ones that are NOT Running
kubectl get pods -w -n production | grep -v Running

# Watch events filtered to a specific pod
kubectl get events -w --field-selector involvedObject.name=api-pod-7d4f8b -n production

# Watch a HPA during a load test
kubectl get hpa api-gateway -w -n production

# Watch all deployments for rollout changes
kubectl get deployments -w -n production

# Watch nodes during a cluster upgrade
kubectl get nodes -w

# Watch with a custom column showing only name and ready status
kubectl get pods -w -n production \
  -o custom-columns='NAME:.metadata.name,READY:.status.conditions[?(@.type=="Ready")].status'

# Watch PVCs for bound state during StatefulSet provisioning
kubectl get pvc -w -n production

Conclusion

kubectl is one of those tools where the ceiling keeps rising the longer you use it. The gap between someone who knows get and apply and someone who reaches for kubectl wait in a CI script, or kubectl patch --type=json when they need precision surgery on a live resource, is real and measurable in minutes saved per incident.

The pattern worth internalising: kubectl is just an HTTP client with great defaults and a thoughtful DSL. Once that clicks, every command stops being a magic incantation and starts being a readable REST call. You start predicting flags you've never tried before. You stop reaching for the browser. And when something breaks at 2am, you move faster.

Start with the shorthands. Add kubectl explain to your muscle memory. Reach for kubectl wait the next time you write a deploy script. The rest will follow naturally.

References

Kubernetes v1.36 "Haru" overview

Selma Guedidi — Fri, 24 Apr 2026 18:12:09 +0000

Released April 22, 2026.

Kubernetes v1.36, just released on April 22, 2026, brings a wide set of improvements that feel less like experimentation and more like completion. With around 70 enhancements across stable, beta, and alpha stages, this release is not about flashy new ideas. It is about finishing what has been in progress for years and making core features reliable enough for everyday production use.

The release is named Haru, a sound in Japanese associated with spring and clarity. It is a fitting name. Many long-running efforts have finally reached maturity, especially in areas like security, resource management, and cluster operations.

The security story is the headline

Three separate security features reached GA (General Availability) in this release, and taken together they represent a meaningful shift in how Kubernetes handles isolation and privilege. None of them are new ideas, but all three are now stable and production-ready.

User Namespaces is the biggest one. The concept has been in progress since KEP #127 and it has been a long road. What it does is remap a container's internal root user to an unprivileged UID on the actual host. A process inside the container thinks it is root. The host disagrees. If that process breaks out of the container, it lands on the node with no privileges at all. This does not eliminate container escapes, but it takes the worst-case outcome from "attacker has root on your node" to "attacker has nothing." For multi-tenant clusters, that gap is everything.

Fine-grained kubelet API authorization is the other one that should have existed sooner. Previously, granting monitoring tools access to the kubelet's API meant handing over the nodes/proxy permission, which is much broader than any metrics scraper actually needs. This feature enables precise, least-privilege access control over individual kubelet API endpoints. Monitoring and observability tools can now get exactly the access they need and nothing more.

Constrained impersonation is in beta, not GA, but it deserves mention here because it closes a real security gap. Kubernetes impersonation has historically been all-or-nothing: if you have permission to impersonate an identity, you can do anything that identity can do. The constrained model splits this into two separate checks. You need permission to impersonate, and you separately need permission for the specific action you want to perform as that identity. Misconfigured RBAC no longer translates into accidental privilege escalation.

Admission webhooks just got a serious competitor

If you have run admission webhooks in production you know the operational tax. There is the webhook service to maintain, the TLS certificates to rotate, the latency added to every single API request, and the debugging experience when something goes wrong mid-deploy. It is not unmanageable but it is never free.

Mutating Admission Policies reached stable in v1.36. You write mutation logic in CEL directly in the API server. No external service, no network hop, no certificate. For the majority of real-world use cases like enforcing labels, defaulting fields, injecting sidecars this covers the ground. The feature is stable, versioned, and backed by the full API machinery guarantees.

This does not mean webhooks are going away. Complex mutations that require external state or multiple API calls still need a webhook. But if your webhooks are doing simple declarative work, the migration path is now clear and the destination is considerably less painful to operate.

PSI metrics give you a better view of what your nodes are actually doing

Utilization metrics lie to you, slightly. A node at 70% CPU utilization can be perfectly healthy or completely saturated depending on whether processes are actually getting scheduled. Pressure Stall Information tells you what utilization cannot: how much time workloads are blocked waiting for CPU, memory, or I/O.

The difference in practice is significant. With PSI you can distinguish between a node that is busy and a node that is stalling. Vertical autoscalers become more accurate. Resource requests become easier to tune. Noisy neighbor problems become visible before they cause actual incidents instead of after. This feature is now stable, and it requires cgroupv2 which should already be your baseline at this point.

OCI artifacts as volumes is quietly interesting for ML teams

This one did not get top billing in the release notes but it has real implications for how ML inference workloads get built.

You can now mount an OCI artifact directly into a pod as a volume. The kubelet pulls from any OCI-compliant registry and mounts the content. No PVC, no storage backend, no init container pulling files down at startup. If you already package your model weights or static assets as OCI images for versioning purposes, you can now deliver them to pods using the exact same registry infrastructure you use for container images.

Content-addressable storage, immutable versioning, and familiar tooling, all for free, because you were already using a registry. Teams that are not yet doing this should at least be aware it is now an option.

Gang scheduling is finally coming to core Kubernetes

This has been an open gap for a long time. Distributed training jobs, simulation workloads, anything where a group of pods either all need to run together or it is not worth running any of them, vanilla Kubernetes has never had a good answer. You either used external schedulers like Volcano or you engineered around the limitation.

The new PodGroup API, landing in alpha, treats a group of pods as a single scheduling unit. Either all pods in the group are bound to nodes, or none of them are. No more partial allocations where a job half-starts and then sits waiting for resources that may never arrive.

It is alpha, so do not build production workflows on it today. But the design is sound and the direction is clear. Native gang scheduling in core Kubernetes is coming.

Things worth knowing before you upgrade

Node log query is GA. You can now read kubelet and kube-proxy logs via kubectl without SSH access to the node. The NodeLogQuery feature gate is on by default. Small ergonomic improvement but during an incident where node access requires a separate approval flow, it matters.

The gitRepo volume plugin is permanently removed. It was deprecated in v1.11 and has been gone in spirit for years, but until v1.36 it could still technically run. Now it cannot and there is no flag to bring it back. The plugin ran git as root on the node which is as bad as it sounds. Replace it with an init container running git-sync. If you have workloads still using gitRepo, they will fail after this upgrade. Check before you upgrade.

Service.spec.externalIPs is deprecated. This field has been a known MITM attack vector since CVE-2020-8554. Full removal is planned for v1.43, so you have time, but start looking at LoadBalancer services or the Gateway API as replacements.

Mixed version proxy is in beta. HA clusters doing rolling control plane upgrades have always had a window where different apiservers are serving different API versions, leading to sporadic 404s. This feature routes each request to whichever apiserver can actually serve it. Much safer upgrade windows for anyone running multiple apiserver replicas.

Mutable resources for suspended jobs is beta and enabled by default. You can suspend a Job, update its CPU, memory, or GPU requests, and resume it without destroying and recreating pods. If you are building queue-based batch scheduling systems this is the feature that makes dynamic resource adjustment actually clean to implement.

HPA scale-to-zero for external metrics is in alpha. The autoscaler can now bring a deployment down to zero replicas based on Object or External metrics like queue depth, and scale it back up when work arrives. Still behind a feature gate, but the infrastructure cost savings for idle queue-driven workloads are obvious.

Full changelog

✅ Stable (GA) in v1.36

User Namespaces in pods
Mutating Admission Policies
External signing of ServiceAccount tokens
Fine-grained kubelet API authorization
Volume Group Snapshots
Mutable CSINode Allocatable (volume attach limits)
OCI Artifact volume source
Node log query
PSI metrics on cgroupv2
SELinux volume labeling speedup
DRA admin access for ResourceClaims and ResourceClaimTemplates
DRA prioritized alternatives in device requests
DRA extended PodResources for Dynamic Resource Allocation
Declarative validation with validation-gen
Removal of gogoprotobuf dependency for Kubernetes API types
Portworx in-tree to CSI driver migration
ProcMount option
CSI driver opt-in for service account tokens via secrets field

🔵 Promoted to Beta in v1.36

Constrained impersonation
Strict IP/CIDR validation
Mutable container resources for suspended Jobs
Mixed version proxy
DRA partitionable devices and consumable capacity
DRA device taints and tolerations
DRA ResourceClaim device status
Memory QoS with cgroupv2
.kuberc user preferences for kubectl
Staleness mitigation for controllers
ComponentStatusz /statusz endpoint
ComponentFlagz /flagz endpoint
Resource health status via allocatedResourcesStatus

🧪 New in Alpha in v1.36

Workload Aware Scheduling with PodGroup API
HPA scale-to-zero for Object and External metrics
Native sparse histogram support for Kubernetes metrics
Manifest-based admission control configuration
CRI list streaming for the kubelet
DRA native resources for CPU management
DRA downward API for resource attributes
DRA ResourceClaim support for workload controllers

⚠️ Deprecations and Removals

Removed: gitRepo volume plugin, permanently and without a fallback option
Deprecated: Service.spec.externalIPs, planned removal in v1.43

Conclusion

v1.36 is a maturation release more than a revolution. The headline features are things the community has been waiting on for a while (user namespaces, mutating admission policies, PSI metrics) and watching them land at GA feels less like news and more like relief. The alpha features, particularly gang scheduling and HPA scale-to-zero, sketch out clearly where the next few releases are going for AI and batch workloads.

The practical upgrade advice is simple: audit for gitRepo usage before anything else. That is the one thing in this release that will break silently if you are not looking for it. Everything else is additive or behind feature gates.

The Kubernetes project has been shipping at a remarkable cadence for over a decade. v1.36 is another steady step in the same direction.

References

Publishing Your Java Library to Maven Central

Selma Guedidi — Sun, 05 Oct 2025 18:37:55 +0000

Why Maven Central Matters

You've built an amazing Java library. It solves real problems, the code is clean, and you're ready to share it with the world. But there's one crucial step between your local repository and global adoption: publishing to Maven Central.

Maven Central isn't just another package repository: it's the canonical repository for Java and JVM libraries. With over 10 million artifacts and billions of downloads monthly, it's where the entire Java ecosystem comes to find dependencies. When your library is on Maven Central, developers can add it to their projects with a single dependency declaration. No custom repositories, no manual downloads, no friction.

In this guide, I'll walk you through the entire process of publishing to Maven Central, from initial setup to your first release. Whether you're publishing your first library or your tenth, this guide will help you navigate the journey.

Step 1: Create Your Sonatype Account
Maven Central is operated by Sonatype, and they've recently modernized the onboarding process. You'll need to create an account on their new platform.

Go to https://central.sonatype.com
Click "Sign up" (you can use GitHub, Google, or email)
Verify your email address if using email registration

Step 2: Claim Your Namespace

Namespace ownership is fundamental to publishing on Maven Central. Your namespace (groupId) serves as the unique identifier for all your published artifacts, and you must demonstrate legitimate ownership before you can publish. The Central Portal offers two primary verification methods, each suited to different use cases.

Method 1: DNS-Based Verification (For Domain Owners)

If you own or maintain a registered domain name, you can leverage the Domain Name System in reversed format for your namespace. This approach mirrors Java's package naming convention and provides the most flexibility for organizational publishing.

How It Works:

The namespace uses your domain name in reverse order, allowing you to create as many subsections as needed for organizational purposes.

Examples:

Domain: example.com → Namespace: com.example
Domain: subdomain.example.com → Namespace: com.example
Domain: my-domain.com → Namespace: com.my-domain

Once verified, you can utilize any groupId starting with your reversed domain and extend it with additional subsections:

com.example.domain
com.example.test
com.example.utilities.logging

Important Considerations:

Exact Reversal: The groupId must reverse the domain name exactly as it appears, preserving hyphens and special characters even if they would produce invalid Java package names. This is acceptable and expected, your Java package names need not match your groupId.
Ownership Requirement: You must have control over the domain to add DNS verification records.

Verification Process:

Navigate to "Namespaces" in the left sidebar
Click "Add Namespace"
Enter your reversed domain namespace (e.g., com.example)
Select "DNS Verification" as your verification method
Copy the provided TXT record from the portal
Add this TXT record to your domain's DNS configuration
Return to the portal and click "Verify Namespace"
Verification typically completes within minutes once DNS propagates

Method 2: Code Hosting Service Verification (Personal Namespace)

For individual developers and open-source projects hosted on popular code hosting platforms, Maven Central provides a streamlined verification process using your platform identity. This method is ideal for personal libraries and projects without a custom domain.

Supported Platforms:

Code Hosting Service	Username Pattern	Example Namespace
GitHub	`myusername`	`io.github.myusername`
GitLab	`myusername`	`io.gitlab.myusername`
Gitee	`myusername`	`io.gitee.myusername`
Bitbucket	`myusername`	`io.bitbucket.myusername`

How It Works:

The namespace format follows the pattern io.{platform}.{username}, where the username matches your account on the respective platform. This creates a globally unique namespace tied to your platform identity.

Verification Process:

Navigate to "Namespaces"
Click "Add Namespace"
Enter your platform-based namespace (e.g., io.github.myusername)
Select your code hosting service as the verification method
The portal will prompt you to create a temporary public repository with a specific verification name
Create the repository on your platform with the exact name provided
Return to the portal and click "Verify Namespace"
Verification is typically instant once the repository is detected

Step 3: Generate GPG Keys

Maven Central enforces a strict security requirement: all published artifacts must be cryptographically signed using GPG (GNU Privacy Guard). This digital signature ensures artifact integrity and authenticity, protecting the entire Java ecosystem from tampering and unauthorized modifications.

3.1 Installing GPG

Before generating keys, you need to ensure GPG is installed on your system.

For macOS

Using Homebrew (recommended):

brew install gnupg

Using MacPorts:

sudo port install gnupg2

Verify installation:

gpg --version

For Linux

Debian/Ubuntu:

sudo apt-get update
sudo apt-get install gnupg

Fedora/RHEL/CentOS:

sudo dnf install gnupg2

Arch Linux:

sudo pacman -S gnupg

Verify installation:

gpg --version

For Windows

Option 1: Using Gpg4win (Recommended)

Download Gpg4win from https://gpg4win.org/download.html
Run the installer and follow the installation wizard
Add GPG to your PATH if not done automatically

Option 2: Using Chocolatey

choco install gnupg

Option 3: Using Windows Subsystem for Linux (WSL)

sudo apt-get install gnupg

Verify installation (in Command Prompt or PowerShell):

gpg --version

3.2 Generating Your Key Pair

Once GPG is installed, generate a new key pair for signing your artifacts:

gpg --gen-key

Follow the interactive prompts to configure your key:

Key type: Choose RSA and RSA (default)
Key size: Use 2048 bits or higher (4096 recommended)
Expiration: Set to 2-3 years (recommended) or no expiration
Real name: Your name or organization name
Email address: A professional email you control
Comment: Optional (e.g., "Maven Central Signing Key")
Passphrase: Create a strong passphrase

Critical: Your passphrase will be required for every release. Store it securely in a password manager—you cannot recover it if lost, and you'll need it throughout your publishing workflow.

Listing Your Keys

After generation, verify your keys:

gpg --list-keys

Output will look like:

pub   rsa4096 2025-10-05 [SC] [expires: 2027-10-05]
      1234567890ABCDEF1234567890ABCDEF12345678
uid           [ultimate] John Doe (Maven Central Signing Key) <john.doe@example.com>
sub   rsa4096 2025-10-05 [E] [expires: 2027-10-05]

The long hexadecimal string is your key ID. You can also use the last 8 characters as a short key ID.

3.3 Publishing Your Public Key

For Maven Central to verify your signatures, your public key must be published to a public key server:

# Using the full key ID
gpg --keyserver keys.openpgp.org --send-keys 1234567890ABCDEF1234567890ABCDEF12345678

# Or using the short key ID (last 8 characters)
gpg --keyserver keys.openpgp.org --send-keys 12345678

Note: It may take a few minutes for your key to propagate across key server networks.

Verifying Key Publication

Confirm your key is publicly accessible:

gpg --keyserver keys.openpgp.org --recv-keys YOUR_KEY_ID

If successful, you'll see:

gpg: key YOUR_KEY_ID: "John Doe (Maven Central Signing Key) <john.doe@example.com>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

Next Steps

With your GPG keys generated, published, and backed up, you're ready to configure your build tool to automatically sign artifacts during the publishing process. Your key ID and passphrase will be required in the build configuration covered in Step 4.

Step 4: Configure Your Build Tool

Configuring Maven for publishing to Maven Central requires understanding two distinct deployment workflows: release versions and snapshot versions. Each serves a different purpose in the development lifecycle and has different requirements.

Understanding Release vs Snapshot Versions

Release Versions (e.g., 1.0.0, 2.3.1, 1.5.0-beta1):

Immutable and permanent once published
Require complete metadata, including developers, SCM, and license information
Undergo full validation before publication
Intended for production use and stable APIs
Cannot be overwritten or deleted after publication

Snapshot Versions (e.g., 1.0.0-SNAPSHOT, 2.3.1-SNAPSHOT):

Mutable and can be republished with the same version
Require minimal metadata validation
Used for development, testing, and continuous integration
Not indexed in Maven Central search
Must be explicitly enabled in your namespace configuration

Generating Authentication Tokens

Before configuring your build, you need to generate authentication tokens from the Central Portal:

Log in to https://central.sonatype.com
Click on your account name in the top right corner
Select "View User Tokens" from the dropdown
Click "Generate User Token"
The portal will display your username and password token
Save these immediately—the password token is only shown once

Important: These tokens are not your Sonatype login credentials. They are specifically generated for programmatic publishing and should be treated as API keys.

Enabling Snapshot Publishing (Optional)

If you plan to publish snapshot versions, you must enable this feature in your namespace:

Log in to https://central.sonatype.com
Navigate to "Namespaces"
Select your verified namespace
Click on its "dropdown menu"
Click on "Enable Snapshots"
Save the configuration

Without this setting enabled, snapshot deployments will fail with an authorization error.

Configuration for Release Versions

Release versions require comprehensive metadata and validation. This configuration ensures your artifacts meet all Maven Central requirements.

Maven POM Configuration

Add the following to your pom.xml:

<project>
    <groupId>io.github.yourusername</groupId>
    <artifactId>your-library</artifactId>
    <version>1.0.0</version>
    <packaging>jar</packaging>

    <name>Your Library Name</name>
    <description>A comprehensive description of your library's purpose and functionality</description>
    <url>https://github.com/yourusername/your-library</url>

    <licenses>
        <license>
            <name>Apache License, Version 2.0</name>
            <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
            <distribution>repo</distribution>
        </license>
    </licenses>

    <developers>
        <developer>
            <id>yourusername</id>
            <name>Your Full Name</name>
            <email>your.email@example.com</email>
            <organization>Your Organization</organization>
            <organizationUrl>https://yoursite.com</organizationUrl>
        </developer>
    </developers>

    <scm>
        <connection>scm:git:git://github.com/yourusername/your-library.git</connection>
        <developerConnection>scm:git:ssh://github.com:yourusername/your-library.git</developerConnection>
        <url>https://github.com/yourusername/your-library/tree/main</url>
        <tag>HEAD</tag>
    </scm>

    <build>
        <plugins>
            <!-- Central Publishing Plugin -->
            <plugin>
                <groupId>org.sonatype.central</groupId>
                <artifactId>central-publishing-maven-plugin</artifactId>
                <version>0.9.0</version>
                <extensions>true</extensions>
                <configuration>
                    <publishingServerId>central</publishingServerId>
                    <autoPublish>true</autoPublish>
                    <waitUntil>published</waitUntil>
                </configuration>
            </plugin>

            <!-- GPG Signing Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-gpg-plugin</artifactId>
                <version>3.1.0</version>
                <executions>
                    <execution>
                        <id>sign-artifacts</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>sign</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <!-- Retrieves passphrase from settings.xml -->
                    <passphraseServerId>gpg.passphrase</passphraseServerId>
                    <!-- Required for gpg2 to avoid GUI prompts -->
                    <gpgArguments>
                        <arg>--pinentry-mode</arg>
                        <arg>loopback</arg>
                    </gpgArguments>
                    <!-- Ensure correct GPG binary is used -->
                    <executable>gpg</executable>
                </configuration>
            </plugin>

            <!-- Source Attachment Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-source-plugin</artifactId>
                <version>3.3.1</version>
                <executions>
                    <execution>
                        <id>attach-sources</id>
                        <goals>
                            <goal>jar-no-fork</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <!-- Javadoc Generation Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-javadoc-plugin</artifactId>
                <version>3.6.3</version>
                <executions>
                    <execution>
                        <id>attach-javadocs</id>
                        <goals>
                            <goal>jar</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <!-- Optional: Configure Javadoc strictness -->
                    <doclint>none</doclint>
                    <quiet>true</quiet>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Plugin Explanations

Central Publishing Maven Plugin

<groupId>org.sonatype.central</groupId>
<artifactId>central-publishing-maven-plugin</artifactId>
<version>0.9.0</version>

This is the modern publishing plugin that replaces the legacy Nexus staging plugin. It handles the entire deployment workflow to Maven Central.

Key Configuration Options:

publishingServerId: References the server credentials in settings.xml (must be central)
autoPublish: When true, automatically publishes after successful validation. When false, requires manual approval in the Central Portal
waitUntil: Controls deployment behavior
- uploaded: Waits until artifacts are uploaded
- validated: Waits until validation completes
- published: Waits until artifacts are fully published to Maven Central (recommended)

Maven GPG Plugin

<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>3.1.0</version>

Signs all project artifacts (JAR, sources, javadoc, POM) with your GPG key during the verify phase.

Key Configuration Options:

passphraseServerId: References the GPG passphrase stored in settings.xml, avoiding hardcoded credentials
gpgArguments: Configures GPG behavior
- --pinentry-mode loopback: Essential for non-interactive environments (CI/CD) and gpg2, prevents GUI password prompts
executable: Specifies the GPG binary path. Use gpg2 if you have both versions installed

Maven Source Plugin

<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>3.3.1</version>

Generates a JAR file containing your source code. Maven Central mandates source JARs for all releases.

Key Configuration:

jar-no-fork: Creates the source JAR without forking a new Maven lifecycle, improving build performance

Maven Javadoc Plugin

<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>3.6.3</version>

Generates comprehensive API documentation and packages it as a JAR. Maven Central requires Javadoc JARs for all releases.

Optional Configuration:

doclint: Set to none to disable strict Javadoc validation (useful if you have legacy code with incomplete docs)
quiet: Reduces console output during generation

Maven Settings Configuration

Add the following to your ~/.m2/settings.xml:

<settings>
    <servers>
        <!-- Central Portal Authentication -->
        <server>
            <id>central</id>
            <username>YOUR_GENERATED_TOKEN_USERNAME</username>
            <password>YOUR_GENERATED_TOKEN_PASSWORD</password>
        </server>

        <!-- GPG Passphrase -->
        <server>
            <id>gpg.passphrase</id>
            <passphrase>YOUR_GPG_PASSPHRASE</passphrase>
        </server>
    </servers>
</settings>

Deploying a Release

Execute the deployment with a single command:

mvn clean deploy

This command will:

Clean previous build artifacts
Compile your code
Run tests
Generate source and Javadoc JARs
Sign all artifacts with GPG
Upload to Maven Central
Validate the deployment
Automatically publish (if autoPublish is true)

Verification: Monitor the deployment at https://central.sonatype.com/publishing/deployments

Configuration for Snapshot Versions

Snapshot versions are designed for rapid iteration during development. They have relaxed validation requirements and can be republished multiple times.

Maven POM Configuration for Snapshots

Snapshots require minimal metadata compared to releases:

<project>
    <groupId>io.github.yourusername</groupId>
    <artifactId>your-library</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <!-- Minimal required metadata for snapshots -->
    <name>Your Library Name</name>
    <description>Development snapshot of Your Library</description>
    <url>https://github.com/yourusername/your-library</url>

    <build>
        <plugins>
            <!-- Central Publishing Plugin (same as releases) -->
            <plugin>
                <groupId>org.sonatype.central</groupId>
                <artifactId>central-publishing-maven-plugin</artifactId>
                <version>0.9.0</version>
                <extensions>true</extensions>
                <configuration>
                    <publishingServerId>central</publishingServerId>
                    <autoPublish>true</autoPublish>
                </configuration>
            </plugin>

            <!-- GPG Signing (same as releases) -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-gpg-plugin</artifactId>
                <version>3.1.0</version>
                <executions>
                    <execution>
                        <id>sign-artifacts</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>sign</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <passphraseServerId>gpg.passphrase</passphraseServerId>
                    <gpgArguments>
                        <arg>--pinentry-mode</arg>
                        <arg>loopback</arg>
                    </gpgArguments>
                    <executable>gpg</executable>
                </configuration>
            </plugin>

            <!-- Source and Javadoc plugins are optional for snapshots -->
            <!-- Include them if you want consistent artifact structure -->
        </plugins>
    </build>
</project>

Key Differences for Snapshots

Relaxed Requirements:

No <developers> section required
No <scm> section required
No <licenses> section required (though recommended)
Source and Javadoc JARs are optional

Version Format:

Must end with -SNAPSHOT (e.g., 1.0.0-SNAPSHOT, 2.1.0-SNAPSHOT)
Can be republished multiple times without changing the version number

Publication Behavior:

Not indexed in Maven Central search
Available immediately after upload
Stored in a separate snapshot repository
Automatically cleaned up after a retention period

Deploying a Snapshot

Deploy snapshots using the same command:

mvn clean deploy

Maven automatically detects the -SNAPSHOT suffix and routes the deployment to the snapshot repository.

Consuming Published Snapshots

To use snapshots published by others (or your own), configure your project's pom.xml or settings.xml:

In Your Project's POM

<repositories>
    <repository>
        <id>central-portal-snapshots</id>
        <name>Central Portal Snapshots</name>
        <url>https://central.sonatype.com/repository/maven-snapshots/</url>
        <snapshots>
            <enabled>true</enabled>
            <updatePolicy>always</updatePolicy>
        </snapshots>
        <releases>
            <enabled>false</enabled>
        </releases>
    </repository>
</repositories>

In Your Settings.xml (Global Configuration)

<settings>
    <profiles>
        <profile>
            <id>central-snapshots</id>
            <repositories>
                <repository>
                    <id>central-portal-snapshots</id>
                    <name>Central Portal Snapshots</name>
                    <url>https://central.sonatype.com/repository/maven-snapshots/</url>
                    <snapshots>
                        <enabled>true</enabled>
                        <updatePolicy>always</updatePolicy>
                    </snapshots>
                </repository>
            </repositories>
        </profile>
    </profiles>

    <activeProfiles>
        <activeProfile>central-snapshots</activeProfile>
    </activeProfiles>
</settings>

Configuration Options:

updatePolicy: Controls how often Maven checks for snapshot updates
- always: Check on every build (recommended for active development)
- daily: Check once per day (default)
- interval:X: Check every X minutes
- never: Never check for updates

Additional Resources

For more detailed information and the latest updates, refer to the official Sonatype documentation:

Central Portal Documentation: https://central.sonatype.org/register/central-portal/
Publishing Guide: https://central.sonatype.org/publish/publish-guide/
Maven Configuration: https://central.sonatype.org/publish/publish-maven/

The official documentation provides comprehensive coverage of edge cases, advanced configurations, and troubleshooting scenarios that may not be covered in this guide.

Note: Sonatype recently migrated from the old JIRA-based system to the new Central Portal at central.sonatype.com. If you find older tutorials mentioning JIRA tickets and issues.sonatype.org, that process is now deprecated. This guide reflects the current, streamlined process.

Questions or issues? Drop a comment below, and I'll do my best to help!

Understanding GitHub Webhooks: Leveraging Reverse Proxy with Ngrok for Local Development

Selma Guedidi — Thu, 25 Jul 2024 00:10:33 +0000

Webhooks are a powerful and flexible way for applications to communicate with each other in real-time. They are a crucial feature for developers who want to automate workflows and integrate with other services. In this blog post, we will explore what GitHub webhooks are, why we use them, and how to leverage a reverse proxy like Ngrok to trigger events on a local development environment.

What are GitHub Webhooks?

Webhooks allow you to subscribe to specific events occurring within a software system and automatically receive detailed data delivered to your server in real-time whenever those events take place.

When you create a webhook, you specify a URL and subscribe to specific events that occur within your GitHub repository. These events can range from code pushes and pull requests to issues and comments. Whenever an event that your webhook is subscribed to occurs, GitHub sends an HTTP POST request with detailed data about the event to the URL you specified.

The webhook's payload URL is the endpoint that will receive these HTTP requests. Your server must be set up to listen for webhook deliveries at this URL. When it receives a request, it can parse the event data and take appropriate actions. This allows for a wide variety of automated workflows, such as triggering CI/CD pipelines, sending notifications, or updating external systems with the latest repository information.

Detailed Workflow of GitHub Webhooks

Event Subscription: When configuring a webhook, you choose which events your webhook should listen for. These could include events like push, pull_request, issues, release, and more.
Event Trigger: When one of the subscribed events occurs in the repository, GitHub prepares an HTTP POST request with a payload containing details about the event. For example, a push event payload includes information about the commits, the branch involved, and the repository.
HTTP Request to Payload URL: GitHub sends the HTTP POST request to the specified payload URL. This request includes the payload data as JSON, providing all the necessary details about the event.
Server Processing: Your server, which is set up to handle requests at the payload URL, receives the webhook request. It then processes the payload data to determine what action to take. For instance, it might trigger a Jenkins build, post a message to a Slack channel, or update a database.
Response to GitHub: After processing the event, your server responds to GitHub with an HTTP status code. A 2xx status code indicates that the request was successfully received and processed.

Using Ngrok as a Reverse Proxy

In a local development environment, your server might not be accessible from the internet, making it difficult to test webhooks that require a publicly accessible endpoint. This is where Ngrok comes in handy. Ngrok is a tool that creates a secure tunnel to your local server, providing a public URL that GitHub can send webhooks to.

How does Ngrok Works?

Ngrok establishes a secure connection between your local machine and a publicly accessible URL, effectively exposing your local server to the internet. Here’s a detailed breakdown of the process:

Create an account in ngrok.com:

Follow given instructions to install ngrok and add authtoken:

Example: Setting Up Ngrok to Trigger a Simple CI/CD Pipeline in Jenkins

Let's walk through an example of setting up Ngrok to expose a local Jenkins server and configure a GitHub webhook to trigger a simple CI/CD pipeline when code is pushed to a repository.

Step 1: Ensure Jenkins is Running

For this example, let's assume that your Jenkins server is already installed and running on port 8080. If Jenkins is running correctly, you should be able to access it in your browser at http://localhost:8080.

Step 2: Install Github plugin

To allow Jenkins to be triggered by GitHub webhooks, you need to install the GitHub plugin:

Go to "Manage Jenkins” -> “Plugins”
Install the GitHub Plugin and restart Jenkins if required.

Step 3: Create a Jenkins Job

Click on “New Item,” enter a name for your job, select “Freestyle project,” and click “OK.”
Add git repository and specify branch(s)
Select the “GitHub hook trigger for GITScm polling” option.
Configure your job. For simplicity, you might just echo a message in the Build section.

Step 4: Start Ngrok

Open a terminal and start Ngrok to create a secure tunnel to your local Jenkins server:

ngrok http 8080

Ngrok will provide a public URL, such as:

Step 5: Configure GitHub Webhook

In your GitHub repository, configure a webhook to send events to your Jenkins server through the Ngrok URL:

Navigate to your GitHub repository and go to “Settings” -> “Webhooks”
Click “Add webhook”
In the “Payload URL” field, enter the Ngrok URL followed by /github-webhook/.
Click “Add webhook” to save.

Note: If your Jenkins server denies anonymous user access, you need to include authentication credentials in the webhook URL to avoid a 401 Unauthorized error. The URL format should be https://user:password@PUBLIC_URL/github-webhook/.

Step 6: Test the Setup
To test the webhook setup, push a change to your GitHub repository. This should trigger the webhook and send a request to your Jenkins server via the Ngrok tunnel. Jenkins will then trigger the configured job.

You can verify this by checking the Jenkins job console output. You should see the echo message "Build triggered by GitHub webhook," indicating that the webhook successfully triggered the CI/CD pipeline.

Conclusion

GitHub webhooks enable real-time automation by notifying external services about repository events. Testing these integrations locally can be challenging due to accessibility constraints. Ngrok simplifies this by creating a secure tunnel to your local server, allowing you to test webhook configurations as if they were exposed on the internet.

In our example, we demonstrated how to use Ngrok to trigger a Jenkins CI/CD pipeline from GitHub events. This setup allows for seamless testing of webhooks in a local environment, ensuring your integrations work as expected before deployment. Overall, leveraging GitHub webhooks with Ngrok enhances development efficiency and reliability.

A dive into Jenkins Configuration as Code (JCasC)

Selma Guedidi — Mon, 22 Jul 2024 19:05:06 +0000

In today's fast-paced DevOps world, automation is key to maintaining efficiency and consistency across development, testing, and deployment processes. Jenkins, a popular open-source automation server, offers a powerful feature called Jenkins Configuration as Code (JCasC). This allows you to manage your Jenkins configurations in a human-readable and declarative manner using YAML files, ensuring your setup is consistent, reproducible, and version-controlled.

In this blog post, we'll explore the benefits of JCasC and walk through a sample use case to get you started.

What is Jenkins Configuration as Code (JCasC)?

Jenkins Configuration as Code (JCasC) allows you to define and configure Jenkins instances using YAML files. This means you can describe all aspects of your Jenkins setup —from global configurations to job definitions— in a version-controlled repository.

Creating the Dockerfile

The Dockerfile provided below sets up Jenkins with the necessary plugins and configurations using JCasC:

FROM jenkins/jenkins:lts-jdk17

# Install Jenkins plugins
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN jenkins-plugin-cli --plugin-file /usr/share/jenkins/ref/plugins.txt

# Disable the setup wizard as we will set up Jenkins as code
ENV JAVA_OPTS="-Djenkins.install.runSetupWizard=false"

# Copy the Configuration as Code (CasC) YAML file into the image
COPY jenkins-casc.yaml /var/jenkins_home/casc_configs/jenkins.yaml

# Tell the Jenkins Configuration as Code plugin where to find the YAML file
ENV CASC_JENKINS_CONFIG="/var/jenkins_home/casc_configs/jenkins.yaml"

In this Dockerfile, we start with the jenkins/jenkins:lts-jdk17 image, ensuring we have a stable Jenkins version with JDK 17.

The jenkins-plugin-cli is a command-line tool that installs Jenkins plugins specified in plugins.txt, streamlining the setup of necessary plugins.

The JAVA_OPTS environment variable disables the Jenkins setup wizard since we are configuring Jenkins using JCasC.

The COPY commands include the required plugins.txt and jenkins-casc.yaml files into the Docker image.

Finally, the CASC_JENKINS_CONFIG environment variable points Jenkins to the location of the JCasC YAML file, allowing it to automatically apply the specified configurations at startup.

Specifying Plugins

To ensure Jenkins has the necessary plugins for its operations, create a plugins.txt file. This file lists the required plugins and their versions:

git:5.0.0
workflow-aggregator:600.vb_57cdd26fdd7
configuration-as-code:1810.v9b_c30a_249a_4c
matrix-auth:3.2.2
job-dsl:1.87

Each line in plugins.txt specifies a plugin. You can include the version after a colon (e.g., git:5.0.0) to ensure a specific version is installed. If you omit the version (e.g., git), the latest version of the plugin will be installed when the image is built. Specifying versions can help maintain consistency across different environments by ensuring the same plugin versions are used.

The matrix-auth plugin is useful for managing access control in Jenkins environments where multiple users or teams interact with Jenkins. We'll need it later when we set up users and assign them different roles based on their responsibilities.

On the other hand, job-dsl Plugin adds support for defining Jenkins jobs using Groovy DSL scripts.

Note: The configuration-as-code plugin is essential for JCasC to function. It enables Jenkins to read the YAML configuration file and apply the specified settings automatically.

Configuring Jenkins with JCasC

Create a jenkins-casc.yaml file to configure Jenkins. In this blog, we will define user authentication, tool installations, and a sample job. Here’s a comprehensive example:

jenkins:
  systemMessage: "Jenkins configured automatically by Jenkins Configuration as Code plugin"

  # Security Realm for user authentication
  securityRealm:
    local:
      allowsSignup: false
      users:
        - id: "admin"
          password: "admin"

        - id: "developer"
          password: "developer"

        - id: "viewer"
          password: "viewer"


  # Authorization Strategy
  authorizationStrategy:
    projectMatrix:
      entries:
        - user:
            name: admin
            permissions:
              - Overall/Administer
        - user:
            name: developer
            permissions:
              - Overall/Read
              - Job/Build
        - user:
            name: viewer
            permissions:
              - Overall/Read



# Tool Configuration
tool:
  git:
    installations:
      - name: "Default"
        home: "/usr/bin/git"
  maven:
    installations:
      - name: maven3
        properties:
          - installSource:
              installers:
                - maven:
                    id: "3.8.4"

# Sample Job Configuration
jobs:
  - script: >
      pipelineJob('example-pipeline-job') {
      definition {
        cps {
            script('''
                pipeline {
                    agent any
                    stages {
                        stage('Build') {
                            steps {
                                echo 'Building...'
                            }
                        }
                        stage('Test') {
                            steps {
                                echo 'Testing...'
                            }
                        }
                        stage('Deploy') {
                            steps {
                                echo 'Deploying...'
                            }
                        }
                    }
                }
            ''')
            }
          }
        }

User authentication is defined in the securityRealm section, where local user authentication is configured, and self-signup is disabled for security. Three users are predefined: admin, with full administrative access; developer, with read and build permissions; and viewer, with read-only access. This setup ensures appropriate access levels based on user roles.

Authorization is handled by the Project-based Matrix Authorization Strategy in the authorizationStrategy section. This strategy provides granular control over user permissions, giving the admin full control with (Overall/Administer), while both the developer and viewer have read access to the overall Jenkins instance (Overall/Read), with the developer also having the ability to build jobs (Job/Build).

The tool section Specifies the configuration for tools that Jenkins will use, in our case: git and Maven. The git tool is set up with a default installation path, and Maven is installed ensuring that these tools are readily available for use in jobs.

The jobs section defines a sample pipeline job using a script block.

Build and Run the Docker Container

Now, we'll build and run our Docker container.

# Build the Docker image
docker build -t jenkins-jcasc .

# Run the Docker container
docker run --name jenkins -p 8080:8080  jenkins-jcasc

Once the container is running, you can access Jenkins at http://localhost:8080 and log in using one of the user credentials created earlier.

We can notice the system message we defined earlier and the sample pipeline.

You can find the code used for this blog in the following repository: https://github.com/SelmaGuedidi/JCasC

Reference

For further information on Jenkins Configuration as Code (JCasC), you can consult these sources:

Configuration-as-code plugin documentation: https://plugins.jenkins.io/configuration-as-code/

demos: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos

Conclusion

Jenkins Configuration as Code simplifies the management of Jenkins configurations, making your CI/CD pipeline more robust and easier to maintain. By leveraging Docker and JCasC, you can quickly set up consistent Jenkins environments that are easy to reproduce and update.

Try out JCasC in your projects and experience the benefits of a streamlined, automated Jenkins setup. Happy automating!