Forem: Narendra Chauhan

Nginx + PHP + MySQL Optimisations and Parameter Calculations

Narendra Chauhan — Fri, 03 Apr 2026 06:38:00 +0000

“Premature optimization is the root of all evil but lack of optimisation is the root of outages.”

Introduction
Architecture Overview
Why Optimisation Is Required
Nginx Optimisations & Parameter Calculations
PHP-FPM Optimisations & Parameter Calculations
MySQL Optimisations & Parameter Calculations
System-Level Optimisations (Linux)
Practical Example: Small vs Medium vs Large Server
Interesting Facts & Statistics
FAQs
Key Takeaways
Conclusion

1. Introduction

Modern web applications rely heavily on the Nginx + PHP + MySQL (LEMP) stack. While default configurations work for testing, they are not suitable for production traffic.

Optimisation ensures:

Faster page load time
Better concurrency handling
Lower memory and CPU usage
Higher stability under load

This document explains what to optimise, why to optimise, and how to calculate parameters practically.

2. Architecture Overview

A typical request flow:

Client sends HTTP request
Nginx handles connection & static content
PHP-FPM processes dynamic PHP requests
MySQL serves data from database
Response sent back to client

Each layer must be tuned together, not individually.

3. Why Optimisation Is Required

Default settings:

Are conservative
Waste available RAM
Limit concurrency
Cause slow response under load

Common Problems Without Optimisation

502 / 504 Gateway errors
High CPU load
PHP-FPM “server reached max_children”
MySQL “Too many connections”

4. Nginx Optimisations & Parameter Calculations

Key Nginx Parameters
worker_processes auto;
worker_connections 4096;

What is worker_processes
Number of worker processes
Best practice: match CPU cores

nproc
Example:
4 CPU cores → worker_processes 4;
What is worker_connections
Maximum connections per worker

Total Max Connections
worker_processes × worker_connections

Example

4 × 4096 = 16,384 concurrent connections

Recommended Extra Optimisations

use epoll;
multi_accept on;

sendfile on;
tcp_nopush on;
tcp_nodelay on;

keepalive_timeout 65;
keepalive_requests 1000;

5. PHP-FPM Optimisations & Parameter Calculations

Key PHP-FPM Settings

pm = dynamic
pm.max_children = 20
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 10

How to Calculate pm.max_children

Find average PHP process memory: ps -ylC php-fpm --sort:rss
Formula: Available RAM for PHP / Avg PHP process size

Example

Available RAM: 2 GB
Avg PHP process: 100 MB

2048 / 100 ≈ 20
pm.max_children = 20

Other Important PHP Optimisations

request_terminate_timeout = 60
max_execution_time = 60
memory_limit = 256M

Enable OPcache

opcache.enable=1
opcache.memory_consumption=256
opcache.max_accelerated_files=20000

6. MySQL Optimisations & Parameter Calculations

Key MySQL Parameters

innodb_buffer_pool_size = 2G
innodb_buffer_pool_instances = 2
max_connections = 200

How to Calculate innodb_buffer_pool_size

Allocate 60–70% of total RAM (dedicated DB server)

Example

Server RAM: 4 GB

4 × 70% ≈ 2.8 GB
Use 2G or 3G

Connection Calculation

Additional Recommended Settings

query_cache_type = 0
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 2

7. System-Level Optimisations (Linux)

File Descriptors

ulimit -n 100000

Kernel Tuning

net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
vm.swappiness = 10

8. Practical Server Size Examples

Small Server (2 CPU / 2 GB RAM)

Nginx workers: 2
worker_connections: 2048
PHP max_children: 10
MySQL buffer pool: 1G

Medium Server (4 CPU / 8 GB RAM)

Nginx workers: 4
worker_connections: 4096
PHP max_children: 30–40
MySQL buffer pool: 4–5G

Large Server (8 CPU / 16 GB RAM)

Nginx workers: 8
worker_connections: 8192
PHP max_children: 60–80
MySQL buffer pool: 10–12G

Practical Demonstration (Images Explained)

Step 1. NGINX OPTIMISATION

Parameter Calculations
Step 1.1 Backup current nginx.conf

  sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak

Step 1.2 Check current config

  cat /etc/nginx/nginx.conf

Screenshot: Current state before changes

Step 1.3 Edit nginx.conf

sudo nano /etc/nginx/nginx.conf

Replace/update with this optimized config:

user www-data;
  worker_processes 2;                    # = nproc (2 cores)
  worker_rlimit_nofile 65535;
  pid /run/nginx.pid;
  include /etc/nginx/modules-enabled/*.conf;

  events {
      worker_connections 1024;           # 2 x 1024 = 2048 total connections
      use epoll;                         # Linux best event model
      multi_accept on;                   # Accept multiple connections at once
  }

  http {

      # Basic Settings
      sendfile on;
      tcp_nopush on;
      tcp_nodelay on;
      keepalive_timeout 30;              # Reduced from default 75s
      keepalive_requests 100;
      types_hash_max_size 2048;
      server_tokens off;                 # Hide nginx version

      client_max_body_size 20m;
      client_body_buffer_size 128k;
      client_header_buffer_size 1k;
      large_client_header_buffers 4 8k;
      client_body_timeout 12;
      client_header_timeout 12;
      send_timeout 10;

      include /etc/nginx/mime.types;
      default_type application/octet-stream;

      # Logging Settings
      access_log /var/log/nginx/access.log;
      error_log /var/log/nginx/error.log warn;   # Only warn+ to reduce I/O

      # Gzip Settings
      gzip on;
      gzip_vary on;
      gzip_proxied any;
      gzip_comp_level 3;                 # Level 3 = good ratio, low CPU
      gzip_min_length 1024;              # Don't compress tiny files
      gzip_buffers 16 8k;
      gzip_http_version 1.1;
      gzip_types
          text/plain
          text/css
          text/javascript
          application/javascript
          application/json
          application/xml
          image/svg+xml
          font/woff2;


      # Open File Cache
      open_file_cache max=1000 inactive=20s;
      open_file_cache_valid 30s;
      open_file_cache_min_uses 2;
      open_file_cache_errors on;

      # FastCGI Cache (optional - enable per site)
      fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=PHPCACHE:10m
                         max_size=100m inactive=60m use_temp_path=off;

      include /etc/nginx/conf.d/*.conf;
      include /etc/nginx/sites-enabled/*;
  }

Step 1.4 Create FastCGI cache directory

 sudo mkdir -p /var/cache/nginx
  sudo chown www-data:www-data /var/cache/nginx

Step 1.5 Test and reload Nginx

sudo nginx -t
  sudo systemctl reload nginx

Screenshot: nginx -t showing syntax is ok and test is successful

Step 1.6 Verify Nginx is running with new settings

sudo nginx -T | grep -E "worker_processes|worker_connections|gzip|keepalive_timeout"
systemctl status nginx

Screenshot: Running status + key parameters

Step 2. PHP-FPM OPTIMISATION

Parameter Calculations
Available RAM for PHP-FPM: ~150MB (conservative, leaving room for MySQL + Nginx)
Average PHP-FPM process size: ~30-40MB
Formula: pm.max_children = 150 / 35 ≈ 4

PHP-FPM Parameters Calculation
pm
Formula: Dynamic (best for variable traffic)
Value: dynamic

pm.max_children
Formula: 150MB ÷ 35MB/process
Value: 4

pm.start_servers
Formula: pm.max_children / 2
Value: 2

pm.min_spare_servers
Formula: pm.start_servers / 2
Value: 1

pm.max_spare_servers
Formula: pm.start_servers
Value: 2

pm.max_requests
Formula: Prevent memory leaks
Value: 500

Step 2.1 Check PHP-FPM version path

  php -v
  ls /etc/php/

Step 2.2 Backup pool config

sudo cp/etc/php/8.4/fpm/pool.d/www.conf/etc /php/8.4/fpm/pool.d/www.conf.bak
sudo cp /etc/php/8.4/fpm/php.ini /etc/php/8.4/fpm/php.ini.bak

Step 2.3 Check current PHP process memory usage

Run this to see actual PHP-FPM process sizes

ps aux | grep php-fpm | grep -v grep | awk '{sum += $6} END {print "Total RSS:", sum/1024, "MB"; print "Count:", NR; print "Avg per process:", sum/NR/1024, "MB"}'

Screenshot: Current process sizes.

Step 2.4 Edit PHP-FPM pool config

sudo nano /etc/php/8.4/fpm/pool.d/www.conf

Find and update these values (search with Ctrl+W in nano):

[www]
user = www-data
group = www-data

listen = /run/php/php8.4-fpm.sock
listen.owner = www-data
listen.group = www-data

pm = dynamic
pm.max_children = 4
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 2
pm.max_requests = 500     ;Restart workers after 500 requests (prevents memory leaks)
pm.process_idle_timeout = 10s

pm.status_path = /status  ; Enable FPM status page
ping.path = /ping

slowlog = /var/log/php8.4-fpm-slow.log
request_slowlog_timeout = 5s     ; Log requests taking > 5 seconds
security.limit_extensions = .php

Step 2.5 Tune PHP OPcache
sudo nano /etc/php/8.4/mods-available/opcache.ini

zend_extension=opcache

  ; Enable OPcache
  opcache.enable=1
  opcache.enable_cli=0

  ; Memory: 64MB for low-RAM server
  opcache.memory_consumption=64
  opcache.interned_strings_buffer=8
  opcache.max_accelerated_files=10000

  ; Production settings (set validate_timestamps=0 in prod)
  opcache.validate_timestamps=1
  opcache.revalidate_freq=60

  opcache.save_comments=1
  opcache.max_wasted_percentage=10
  opcache.use_cwd=1

  ; JIT (PHP 8.x feature)
  opcache.jit_buffer_size=32M
  opcache.jit=1255

Step 2.6 Tune PHP.ini key values
sudo nano /etc/php/8.4/fpm/php.ini

Find and update:
  ; Memory limit per PHP process
  memory_limit = 128M

  ; Upload/POST limits
  upload_max_filesize = 20M
  post_max_size = 25M
  max_execution_time = 60
  max_input_time = 60

  ; Error handling (production)
  display_errors = Off
  log_errors = On
  error_log = /var/log/php_errors.log

  ; Session handling
  session.gc_maxlifetime = 1440
  session.cookie_httponly = 1
  session.cookie_secure = 1

  ; Disable dangerous functions
  disable_functions = exec,passthru,shell_exec,system,proc_open,popen

Step 2.7 Restart PHP-FPM and verify

sudo php-fpm8.4 -t
  sudo systemctl restart php8.4-fpm
  sudo systemctl status php8.4-fpm

Screenshot: PHP-FPM status showing active (running)

Step 2.8 Verify OPcache is active

php -r "var_dump(opcache_get_status());" | head -30

- or check via CLI

php -i | grep -E "opcache|OPcache"

Screenshot: OPcache enabled status

Step 2.9 Check PHP-FPM processes after restart

  ps aux | grep php-fpm | grep -v grep

- Should show master + 2 worker processes (pm.start_servers=2)

Screenshot: FPM process list

Step 3. MYSQL OPTIMISATION

Parameter Calculations

RAM budget for MySQL: ~200MB (out of 914MB total)

innodb_buffer_pool_size
Formula: ~20% of RAM (shared server)
Value: 192M

innodb_buffer_pool_instances
Formula: buffer_pool ÷ 128M
Value: 1

max_connections
Formula: Low RAM = conservative
Value: 50

innodb_log_file_size
Formula: 25% of buffer pool
Value: 48M

tmp_table_size
Formula: Memory temporary tables
Value: 16M

max_heap_table_size
Formula: Same as tmp_table_size
Value: 16M

thread_cache_size
Formula: Reuse threads
Value: 8

table_open_cache
Formula: Open tables cache
Value: 400

Step 3.1 Check current MySQL config and status

sudo cp /etc/mysql/mysql.conf.d/mysqld.cnf /etc/mysql/mysql.conf.d/mysqld.cnf.bak

- Check current variables

mysql -u root -p -e "SHOW VARIABLES LIKE 'innodb_buffer_pool%';"
  mysql -u root -p -e "SHOW VARIABLES LIKE 'max_connections';"
  mysql -u root -p -e "SHOW STATUS LIKE 'Threads_connected';"

Screenshot: Current MySQL variables

Step 3.2 Check actual MySQL memory usage

mysql -u root -p -e "
  SELECT
    ROUND(@@innodb_buffer_pool_size/1024/1024, 0) AS 'Buffer Pool MB',
    ROUND(@@key_buffer_size/1024/1024, 0) AS 'Key Buffer MB',
    @@max_connections AS 'Max Connections',
    @@thread_stack/1024 AS 'Thread Stack KB';
  "

Screenshot: Current MySQL memory config

Step 3.3 Edit MySQL config

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Add/update under [mysqld]:

[mysqld]
  pid-file        = /var/run/mysqld/mysqld.pid
  socket          = /var/run/mysqld/mysqld.sock
  datadir         = /var/lib/mysql
  log-error       = /var/log/mysql/error.log

  #
  # ===== MEMORY SETTINGS =====
  # Server has 914MB RAM - allocate ~200MB for MySQL
  #
  innodb_buffer_pool_size         = 192M    # Main InnoDB cache (most important!)
  innodb_buffer_pool_instances    = 1       # 1 instance (< 1GB pool)
  innodb_log_file_size            = 48M     # ~25% of buffer pool
  innodb_log_buffer_size          = 8M
  innodb_flush_log_at_trx_commit  = 2       # Slight risk, big perf gain (use 1 for strict ACID)

  #
  # ===== CONNECTION SETTINGS =====
  #
  max_connections         = 50             # Low RAM = keep connections limited
  thread_cache_size       = 8             # Reuse threads, avoid creation overhead
  wait_timeout            = 120           # Kill idle connections after 2 min
  interactive_timeout     = 120

  #
  # ===== QUERY CACHE (removed in MySQL 8.0, skip) =====
  # MySQL 8.0 removed query_cache - use ProxySQL or app-level cache

  #
  # ===== TABLE CACHE =====
  #
  table_open_cache        = 400
  table_definition_cache  = 400

  #
  # ===== TEMP TABLES =====
  #
  tmp_table_size          = 16M
  max_heap_table_size     = 16M

  #
  # ===== InnoDB I/O =====
  #
  innodb_file_per_table           = 1
  innodb_flush_method             = O_DIRECT  # Avoid double buffering with OS cache
  innodb_read_io_threads          = 2         # = CPU cores
  innodb_write_io_threads         = 2         # = CPU cores

  #
  # ===== SLOW QUERY LOG =====
  #
  slow_query_log          = 1
  slow_query_log_file     = /var/log/mysql/slow.log
  long_query_time         = 2             # Log queries > 2 seconds
  log_queries_not_using_indexes = 1

  #
  # ===== BINARY LOG (disable if not using replication) =====
  #
  # skip-log-bin                           # Uncomment if no replication needed

Step 3.4 Validate and restart MySQL

sudo mysqld --validate-config
  sudo systemctl restart mysql
  sudo systemctl status mysql

Screenshot: MySQL status active

Step 3.5 Verify new MySQL variables

mysql -u root -p -e "
  SELECT 'innodb_buffer_pool_size' AS Variable,
         ROUND(@@innodb_buffer_pool_size/1024/1024,0) AS 'Value (MB)'
  UNION SELECT 'max_connections', @@max_connections
  UNION SELECT 'innodb_log_file_size MB', ROUND(@@innodb_log_file_size/1024/1024,0)
  UNION SELECT 'tmp_table_size MB', ROUND(@@tmp_table_size/1024/1024,0)
  UNION SELECT 'thread_cache_size', @@thread_cache_size;
  "

Screenshot: New values confirmed

Step 3.6 Check MySQL memory usage post-restart

  free -h
  ps aux | sort -k6 -rn | head -10

Screenshot: Overall memory after all services tuned

“Measure first, tune second, and monitor always.”

Step 4. FINAL VERIFICATION

Step 4.1 Check all services running

  systemctl status nginx php8.4-fpm mysql --no-pager

Step 4.2 Full memory picture

free -h
  ps aux --sort=-%mem | head -15

Step 4.3 Check nginx + PHP working together

Test nginx config is still valid
  sudo nginx -t

 Test PHP-FPM socket exists
  ls -la /run/php/php8.4-fpm.sock

 Check FPM status (if you added status page to your site config)
  curl http://localhost/status

Step 4.4 Check MySQL slow log is active

mysql -u root -p -e "SHOW VARIABLES LIKE 'slow_query%';"

Step 4.5 Final health summary

echo "=== NGINX ===" && nginx -v && systemctl is-active nginx
  echo "=== PHP-FPM ===" && php -v | head -1 && systemctl is-active php8.4-fpm
  echo "=== MYSQL ===" && mysql --version && systemctl is-active mysql
  echo "=== MEMORY ===" && free -h

Screenshot: Final health check

Your server is fully optimised. All phases verified:

Phase 0 Swap 2GB active
Phase 1 Nginx tuned
Phase 2 PHP-FPM + OPcache + JIT enabled
Phase 3 MySQL InnoDB / connections / slow query log active

9. Interesting Facts & Statistics

1 second delay can reduce conversions by 7%
OPcache can improve PHP performance by 2–3×
MySQL buffer pool cache hit ratio above 99% is ideal
Nginx handles 10× more concurrent connections than traditional servers

11. FAQs

Q1. Should I optimise Nginx, PHP, or MySQL first?
Start with PHP-FPM and MySQL, then tune Nginx.

Q2. Can wrong tuning crash the server?
Yes. Over-allocating RAM causes OOM kills.

Q3. Are these values fixed forever?
No. Recalculate after traffic growth.

Q4. Do I need load testing?
Yes. Use tools like ab, wrk, or k6.

12. Key Takeaways

Optimisation is calculation-based, not guesswork
PHP-FPM memory calculation is critical
MySQL buffer pool has the biggest performance impact
Nginx handles concurrency, not application logic
Monitoring is mandatory after tuning

13. Conclusion

Optimising Nginx + PHP + MySQL is not about copying configs from the internet—it is about understanding server resources, calculating limits, and balancing load across layers.
A well-optimised stack:

Handles higher traffic
Reduces downtime
Improves user experience
Saves infrastructure cost

About the Author:Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

Prometheus and Grafana Monitoring for a Node.js API

Narendra Chauhan — Thu, 05 Mar 2026 11:26:12 +0000

“Monitoring is not about collecting data, it's about gaining visibility into what truly matters.”

Introduction
Prerequisites
Architecture Overview
Step 1: Create a Sample Node.js API
Step 2: Dockerize the Node.js App
Step 3: Configure Prometheus
Step 4: Run Prometheus & Grafana with Docker
Step 5: Configure Grafana Dashboard
Step 6: Verify Metrics & Simulate Load
Interesting Facts & Statistics
FAQs
Key Takeaways
Conclusion

1. Introduction

Modern applications rely heavily on APIs, and Node.js APIs are widely used due to their high performance and scalability. However, without proper monitoring, even small issues - such as high latency or memory leaks - can quickly escalate into serious production outages.

In this Proof of Concept (POC), we demonstrate how to monitor a Node.js API using Prometheus for metrics collection and Grafana for visualization and alerting.

Goals of this POC

Monitor API performance in real time
Identify latency and error issues
Visualize metrics using dashboards
Prepare a production-ready observability foundation

2. Prerequisites

You should have the following:

Linux or macOS system
Docker & Docker Compose installed
Basic knowledge of Node.js
Required open ports: 3000, 9090, 3001

3. Architecture Overview

Architecture Flow

Node.js API exposes a /metrics endpoint
Prometheus scrapes metrics every 5 seconds
Grafana visualizes metrics from Prometheus
Alerts trigger when defined thresholds are exceeded

4. Step 1: Create a Sample Node.js API

Create project directory

mkdir nodejs-monitoring-poc
cd nodejs-monitoring-poc
mkdir app
cd app
npm init -y

Install dependencies

npm install express prom-client

Create index.js

const express = require('express');
const client = require('prom-client');
const app = express();
const PORT = 3000;

// Collect default system metrics
client.collectDefaultMetrics({ timeout: 5000 });
// Custom HTTP request counter
const httpRequestCounter = new client.Counter({
 name: 'http_requests_total',
 help: 'Total number of HTTP requests',
 labelNames: ['method', 'route', 'status']
});

// Middleware to track requests
app.use((req, res, next) => {
 res.on('finish', () => {
   httpRequestCounter.labels(req.method, req.path, res.statusCode).inc();
 });
 next();
});

// Sample API endpoint
app.get('/api/hello', (req, res) => {
 res.json({ message: 'Hello from Node.js API' });
});

// Metrics endpoint
app.get('/metrics', async (req, res) => {
 res.set('Content-Type', client.register.contentType);
 res.end(await client.register.metrics());
});

app.listen(PORT, () => {
 console.log(`Node.js app running on port ${PORT}`);
});

Run locally

5. Step 2: Dockerize the Node.js App

Create a Dockerfile inside the app/ directory:

FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["node", "index.js"]

6. Step 3: Configure Prometheus

Create Prometheus directory

cd ..
mkdir prometheus

Create prometheus.yml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'docker'
    static_configs:
      - targets: ['grafana:3000', 'prometheus:9090']
      - targets: ['poc-addweb-app.addwebprojects.com']

7. Step 4: Run Prometheus & Grafana with Docker Compose

Create docker-compose.yml

version: '3.8'

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    volumes:
      - ./.prome/prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin
    ports:
      - "3003:3000"
    depends_on:
      - prometheus

Start the stack
docker-compose up -d
Verify services

Node.js API → https://poc-addweb-app.addwebprojects.com
Prometheus → https://promotheus.65.1.109.206.nip.io
Grafana → https://poc-grafana.addwebprojects.com Default Grafana credentials

Username: admin
Password: admin

8. Step 5: Configure Grafana Dashboard

Add Prometheus Data Source

Grafana → Settings → Data Sources
Select Prometheus
URL: https://promotheus.65.1.109.206.nip.io
Click Save & Test Create Dashboard Panels (PromQL) Total Requests sum(node_app_total_requests)

Uptime Second
rate(node_app_uptime_seconds])

Memory Usage
Process_resident_memory_bytes
grafana_Process_resident_memory_bytes

“In production, everything fails eventually. Monitoring tells you when and why.”

9. Step 6: Verify Metrics & Simulate Load

Generate traffic
sudo apt install wrk -y
wrk -t8 -c500 -d120s https://poc-addweb-app.addwebprojects.com/
Verify

Prometheus → Status → Targets → UP
Grafana dashboard shows increasing metrics

Practical Demonstration (Images Explained)

Step 1: Verify Prometheus Targets Are Healthy

Navigate to the Prometheus Targets page at https://promotheus.65.1.109.206.nip.io/targets to confirm all scrape targets are reachable. The "State" column must show "UP" for every endpoint.

Figure 1: Prometheus Target Health page displaying all three endpoints under the "docker" job (poc-addweb-app, grafana:3000, prometheus:9090) with State = UP and scrape latency under 30 ms. This confirms that Prometheus is successfully collecting metrics from the Node.js application, its own internal metrics, and Grafana.

Step 2: Access the Grafana Dashboard
Open the Grafana web interface at https://poc-grafana.addwebprojects.com. The default login credentials are:
Username: admin
Password: admin

Figure 2: Grafana v12.3.3 login page served at poc-grafana.addwebprojects.com. After authentication, you are redirected to the Home dashboard where data sources and panels can be configured.
Open the "Node-Application" dashboard in Grafana. The "Up Status" panel uses the PromQL query "up" to display a binary availability indicator (1 = reachable, 0 = unreachable) for each monitored target.

Figure 3: Grafana "Up Status" panel showing all three services (Grafana at grafana:3000, Node.js app at poc-addweb-app.addwebprojects.com, and Prometheus at prometheus:9090) maintaining a constant value of 1 over the last 5 minutes, confirming 100% availability.

Step 4: Observe Baseline Request Volume (Before Load Test)
Before generating any synthetic traffic, examine the "node_app_total_requests" panel to establish a baseline. Under normal operating conditions, the request counter should show a slow, steady increase driven only by Prometheus scrape requests and occasional organic traffic.

Figure 4: Grafana "node_app_total_requests" panel displaying the baseline metric. The counter shows approximately 333,454 to 333,476 total requests over a 5-minute window, with a gentle linear slope. This confirms the application is healthy and processing only background scrape traffic at this point.

Step 5: Generate Synthetic Load with wrk
To simulate production-level traffic, use the wrk HTTP benchmarking tool to generate a sustained burst of concurrent requests against the Node.js API:

    wrk -t8 -c500 -d120s https://poc-addweb-app.addwebprojects.com/

This command spawns 8 threads maintaining 500 concurrent connections for 120 seconds, producing significant load on the application.

Figure 5: Terminal output of the wrk load test. Results show 203,609 total requests completed in 2 minutes at an average rate of ~1,696 requests/second. Average latency was 291.55 ms with a maximum of 956.07 ms. The 74.69 MB data transfer confirms sustained throughput throughout the test duration.

Step 6: Analyze Post-Load-Test Metrics in Grafana

After the load test completes, return to the Grafana dashboard to observe the impact. The metrics should show a dramatic spike correlating with the load test window, demonstrating that the monitoring pipeline correctly captured the traffic surge in real time.

Figure 6: Complete Grafana "Node-Application" dashboard captured during the load test. Key observations:

Up Status: All three services remained UP throughout the test (no downtime).
node_app_uptime_seconds: Uptime counter continued incrementing normally (14,200 seconds).
node_app_total_requests: Steep spike from 335,000 to 550,000 during the 2-minute load window.
grafana_alerting_request_duration_seconds_bucket: Alert evaluation latency remained stable.
prometheus_sd_inode_failures_total: No service discovery failures detected (flat at 0).
process_resident_memory_bytes: Memory usage remained stable for both Grafana (80 MB) and Prometheus (140 MB), indicating no memory leaks under load.

A closer look at the "node_app_total_requests" panel reveals the exact moment the load test began and the subsequent plateau once it completed:

Figure 7: Zoomed view of "node_app_total_requests" showing the dramatic increase from 335,000 to 540,000 during the wrk load test (approximately 17:42:30 to 17:43:30). After the test concluded, the counter flattened back to its baseline growth rate, confirming that the spike was entirely caused by the synthetic load. This panel validates that the custom prom-client Counter metric accurately tracks every HTTP request processed by the Node.js application.

Summary: The practical demonstration confirms that the Prometheus + Grafana monitoring stack is fully operational. All scrape targets are healthy, baseline metrics are collected correctly, and the system accurately reflects real-time traffic changes under load. The dashboards provide immediate visibility into application performance, making this setup suitable for production monitoring scenarios.

10. Interesting Facts & Statistics

Monitoring reduces MTTR by 50–60%
APIs without metrics often fail silently
Prometheus is used by thousands of production systems worldwide

11. FAQs

Q: Can I use this in production?
Yes, with persistent storage, authentication, and proper alerting.

Q: Does this support Kubernetes?
Absolutely. The same metrics approach applies to Kubernetes deployments.

Q: Can I add logs?
Yes, by integrating Grafana Loki.

12. Key Takeaways

Metrics are critical for API reliability
Prometheus is lightweight and scalable
Grafana simplifies observability and alerting
This POC closely mirrors real production setups

13. Conclusion

This practical POC demonstrates how to monitor a Node.js API end-to-end using Prometheus and Grafana.
By implementing monitoring early, teams gain better visibility, faster incident response, and improved operational stability.

About the Author: Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

Node.js Application - CI/CD with Bitbucket Pipelines on AWS EC2

Narendra Chauhan — Wed, 11 Feb 2026 06:40:07 +0000

"Automation is the key to scaling software delivery."

Introduction
Project Overview
Architecture & Deployment Flow
Setup Bitbucket Pipelines
Interesting Facts & Statistics
Frequently Asked Questions (FAQs)
Key Takeaways
Conclusion

Introduction

In modern software development, automation, reliability, and speed are critical for successful application delivery. Continuous Integration and Continuous Deployment (CI/CD) pipelines help developers deploy applications faster with minimal manual intervention.

This document explains a Proof of Concept (POC) where a Node.js login application is deployed automatically to an AWS EC2 instance using Bitbucket Pipelines, managed by PM2, and served securely via Nginx as a reverse proxy.

Project Overview

This project demonstrates an end-to-end CI/CD workflow:

A Node.js login application hosted in Bitbucket
Automated deployment triggered on code push to the prd branch
Secure server access via SSH
Application process management using PM2
Public access via Nginx reverse proxy

The deployed application is accessible through a public URL, validating the successful CI/CD pipeline execution.

Architecture & Deployment Flow

This CI/CD pipeline follows a practical, step-by-step deployment flow using Bitbucket Pipelines and AWS EC2.

High-level flow:

Developer pushes code to the prd branch
Bitbucket Pipelines is triggered automatically
Pipeline connects to AWS EC2 using SSH
Latest code is pulled from Bitbucket
Dependencies are installed
Application is restarted using PM2
Nginx routes HTTP traffic to the Node.js app

This ensures zero manual deployment effort and consistent releases.

EC2 server setup (one-time)

1.1 Install Node.js + Git + Nginx
sudo apt update
sudo apt install -y git nginx curl
Install Node.js (recommended using NodeSource, example Node 20):

curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt install -y nodejs
node -v
npm -v
1.2 Install PM2 globally
sudo npm i -g pm2
pm2 -v
1.3 Create project directory
Example:
sudo mkdir -p /var/www/myapp
sudo chown -R deploy:deploy /var/www/myapp
2) Clone web Application repo once (manual first time)
cd /var/www/myapp
git clone .

2.1 Install and start using PM2
Example: if your app entry is server.js and runs on port 3000:

npm ci
pm2 start server.js --name myapp
pm2 save

Enable PM2 auto-start on reboot:

pm2 startup
pm2 save

Nginx reverse proxy (one-time)

3.1 Create Nginx config

sudo nano /etc/nginx/sites-available/myapp

server {
  listen 80;
  server_name yourdomain.com www.yourdomain.com;
 location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}

Enable site:

sudo ln -s /etc/nginx/sites-available/myapp /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl restart nginx
(Optional SSL with Let’s Encrypt):
sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com

Setup Bitbucket Pipelines

This section explains the practical CI/CD implementation step by step, exactly as executed in the Bitbucket Pipeline.

SSH key for Bitbucket Pipelines → EC2 (required)

Create a deploy key pair (local machine)

On your local PC:
ssh-keygen -t ed25519 -C "bitbucket-pipeline" -f bb_pipeline_key

You will get:

bb_pipeline_key (private)
bb_pipeline_key.pub (public)

4.2 Add public key to EC2
On EC2 as deploy user:

mkdir -p ~/.ssh
chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys
Paste content of bb_pipeline_key.pub, then:**
chmod 600 ~/.ssh/authorized_keys

4.3 Add private key to Bitbucket repository variables

In Bitbucket:

Repo → Repository settings → Pipelines → Repository variables Add:
SSH_PRIVATE_KEY = (paste full content of bb_pipeline_key)
SSH_USER = User
SSH_HOST = your EC2 public IP / domain
APP_DIR = /var/www/myapp

Also add:

KNOWN_HOST (optional but recommended) or use ssh-keyscan in pipeline (we will do ssh-keyscan)

Bitbucket Pipeline YAML Configuration
1. vim bitbucket-pipelines.yml

image: node:24

pipelines:
 branches:
   prd:
     - step:
         name: Deploy to Production
         deployment: Production
         script:
           - apt-get update && apt-get install -y openssh-client
           - mkdir -p ~/.ssh
           - ssh-keyscan $SSH_HOST >> ~/.ssh/known_hosts


           - |
             ssh $SSH_USER@$SSH_HOST << EOF
               set -e
               set +x


               echo "Load NVM"
               export NVM_DIR="\$HOME/.nvm"
               [ -s "\$NVM_DIR/nvm.sh" ] && . "\$NVM_DIR/nvm.sh"


               echo "----Use Node 24-----"
               nvm use 24


               echo "-----Change directory-----"
               cd $APP_DIR


               echo "----Pull latest code----"
               git pull origin prd


               echo "--npm install----"
               npm install


               echo "----Restart PM2-----"
               pm2 restart all


               echo "-----Deployment completed Successfully------"
             EOF

Practical Demonstration (Images Explained)

"If it hurts, do it more often and automate it." - DevOps Principle

Step 1. BEFORE: Original Login Page

This shows the original application running at nodeapp.13.127.87.71.nip.io/login with the title "Login Page". This is the state before making any code changes.

Step 2. Making Changes & Pushing Code

This terminal screenshot shows the developer workflow

Step 3. Bitbucket Pipelines Dashboard

This shows the Pipelines page in Bitbucket with successful deployments:

The highlighted row #30 is the most recent deployment that was triggered automatically when code was pushed to the prd branch.

Step 4. AFTER: Updated Login Page

This shows the application after successful deployment. The title has changed from "Login Page" to "Addweb Login Page" - confirming the CI/CD pipeline worked correctly!

Pipeline Failure Scenario

Step 1. Intentionally Introduce an Error
To test pipeline failure behavior, we intentionally modified the package.json file by adding an invalid dependency:

Example change in package.json:

"this-package-does-not-exist-123": "1.0.0"

This package does not exist in the npm registry. the purpose was to simulate a real-world mistake such as:

Typo in package name
Incorrect dependency version
Invalid module added by mistake

Step 2. Commit and Push the Wrong Code
After modifying package.json, the changes were committed and pushed to the prd branch:

git commit -m "We mentioned the wrong package name in package.json."git push origin prd

Since our Bitbucket Pipeline is configured to run automatically on the prd branch, this push immediately triggered a new pipeline execution.

Step 3. Pipeline Triggered Automatically
As expected, Bitbucket Pipelines started running automatically as soon as the code was pushed.

In the Pipelines dashboard we can see:

New pipeline execution created
Status initially shown as “In Progress”

This confirms that the CI/CD automation is working correctly.

Step 4. Pipeline Execution Failed
During pipeline execution, the following command was executed on the server:

npm install

Because we added a non-existent package, the installation failed with this error:

npm error code E404
npm error 404 Not Found - GET https://registry.npmjs.org/this-package-does-not-exist-123
npm error 404 'this-package-does-not-exist-123@1.0.0' is not in this registry.

As a result:

The pipeline step stopped
Deployment process was aborted
Application was NOT restarted
Previous working version remained intact

If any step in the CI/CD pipeline fails, the deployment automatically stops. This protects production from broken or unstable code.

Interesting Facts & Statistics

CI/CD can reduce deployment time by up to 70% Source:- CI/CD up to 70%
PM2 improves Node.js app uptime close to 99.9% Source:- PM2 uptime 99.9%
Automated pipelines reduce deployment errors by 60-80% Source:- Automated by 60-80%

"CI/CD is not a tool, it’s a culture"

Frequently Asked Questions (FAQs)

Q1. Why use Bitbucket Pipelines?
Bitbucket Pipelines provides native CI/CD integration with repositories, reducing setup complexity.

Q2. Why PM2 instead of node directly?
PM2 ensures application stability, auto-restarts on failure, and better process control.

Q3. Why use Nginx as a reverse proxy?
Nginx improves security, handles traffic efficiently, and allows SSL termination.

Key Takeaways

CI/CD automates deployments and saves time
Bitbucket Pipelines integrates seamlessly with repositories
AWS EC2 provides flexible and scalable hosting
PM2 ensures high availability of Node.js apps
Nginx enhances security and request handling

Conclusion

This POC successfully demonstrates a real-world CI/CD pipeline for a Node.js application using Bitbucket Pipelines and AWS EC2. The setup ensures reliable, automated, and scalable deployments with minimal manual intervention.By implementing this approach, teams can improve deployment confidence, reduce errors, and accelerate delivery - making it a strong foundation for production-grade applications.

About the Author: Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

How to Automate Vulnerability Scans with Trivy

Narendra Chauhan — Mon, 24 Nov 2025 10:32:04 +0000

"Manual scanning finds yesterday’s risks, automated scanning protects tomorrow’s releases

Introduction
What Is Trivy?
Why Vulnerability Scanning Matters
Key Features of Trivy
Installing and Setting Up Trivy
How to Run a Manual Scan
Automating Vulnerability Scans in CI/CD Pipelines
Integrating Trivy with Docker and Kubernetes
Interesting Facts & Statistics
FAQs
Key Takeaways
Conclusion

Introduction

As DevOps and security merge into DevSecOps, automation becomes key to maintaining secure and fast software delivery. One tool that has become essential in this space is Trivy, an open-source vulnerability scanner that detects security issues in container images, file systems, IaC (Infrastructure as Code) templates, and more.

This guide walks you through how to automate vulnerability scans using Trivy, integrate it into CI/CD pipelines, and ensure your software stays secure — without slowing down your delivery cycle.

What Is Trivy?

Trivy (by Aqua Security) is a simple yet powerful open-source vulnerability scanner.

Container images
File systems and repositories
Infrastructure as Code (Terraform, Helm, etc.)
Known vulnerabilities (CVEs)
Misconfigurations
Secret leaks
Software Bill of Materials (SBOMs) Trivy’s lightweight design and easy integration make it a favorite among DevOps teams looking for quick security insights.

Why Vulnerability Scanning Matters

In 2025, over 85% of security breaches are linked to vulnerabilities in third-party components.
With continuous deployment cycles, vulnerabilities can slip into production unnoticed.
Automated scanning ensures:

Early detection of issues
Compliance with security standards
Reduced risk of production exploits
Faster incident response

Key Features of Trivy

Wide Coverage – Scans OS packages, libraries, IaC, and Kubernetes manifests.
Fast & Lightweight – Uses local caching to reduce scan times.
Comprehensive Security – Detects CVEs, secrets, and config flaws.
CI/CD Ready – Easily integrates with GitHub Actions, Jenkins, GitLab CI, and others.
SBOM Support – Generates Software Bill of Materials in multiple formats (JSON, SPDX, CycloneDX).

Installing and Setting Up Trivy

For Linux/macOS:

brew install aquasecurity/trivy/trivy
sudo apt install trivy -y

For Docker-based:

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock aquasec/trivy image nginx:latest

To verify installation:

trivy --version

How to Run a Manual Scan

Scan a Docker image:

trivy image nginx:latest

Scan a local directory:

trivy fs .

Scan a Git repository:

trivy repo https://{url}

You’ll get a detailed vulnerability report with severity levels: LOW, MEDIUM, HIGH, and CRITICAL.

"Trivy turns vulnerability scanning from a task into a habit and from a habit into a safety net

Automating Vulnerability Scans in CI/CD Pipelines

Example: GitHub Actions

name: Trivy Scan
on:
  push:
    branches: [ main ]

jobs:
  trivy-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'your-docker-image:latest'
          format: 'table'
          exit-code: '1'
          ignore-unfixed: true

If any critical vulnerability is detected, the pipeline will fail, preventing unsafe code from deploying.

Integrating Trivy with Docker and Kubernetes

Scan a Docker Image Before Push:

trivy image myapp:latest

Scan Running Pods in Kubernetes:

trivy k8s --report summary cluster

You can even automate this using CronJobs in Kubernetes to perform daily scans and push results to Slack or email.

Interesting Facts & Statistics

Around 70% of organizations rely on open-source components that contain known vulnerabilities, making automated scanning essential for security. Sources :- Open-source vulnerabilities
Trivy supports more than 30 vulnerability databases, including major sources like GitHub Security Advisories and the NVD (National Vulnerability Database). Sources:- GitHub Security Advisories
A typical Trivy scan is highly efficient and can complete in under 60 seconds, making it suitable even for fast-paced CI/CD environments. Sources:- Trivy scan is highly efficient

"A container without vulnerability scanning is a locked room with an open window."

Common Issues Explained

1. Slow vulnerability scans
This usually happens when Trivy is using an outdated local cache. Refreshing the cache resolves the issue.
Solution: Run trivy --refresh to update the data.

2. False positives during scans
This problem occurs if the vulnerability database is old or outdated.
Solution: Always update the database before running scans by executing a Trivy DB update command.

3. CI/CD pipeline failures caused by Trivy
If the pipeline keeps failing during scans, it often means the severity thresholds are too strict.
Solution: Adjust the exit-code configuration or relax the severity filters to match your risk tolerance.

4. Missing or unreported CVEs
This can happen if the base image uses an OS that Trivy doesn't fully support.
** Solution:** Check the container image’s base OS for compatibility or enable debug mode using --debug to identify the issue.

Best Practices for Effective Scanning

Use automation early in CI/CD — catch issues before builds.
Regularly update the vulnerability database (trivy --download-db-only).
Enable SBOM reports to enhance supply chain transparency.
Integrate with notification tools (Slack, Teams) for quick alerts.
Combine Trivy with policy enforcement tools like OPA or Kyverno.

FAQs

Q1: Is Trivy free to use?
Yes, Trivy is completely open-source under the Apache 2.0 license.

Q2: How often should I scan images?
Ideally, before every deployment — or at least daily in production environments.

Q3: Can Trivy scan for secrets?
Yes, it detects secrets and sensitive credentials in code and configurations.

Q4: Does Trivy work offline?
Yes, after downloading its vulnerability database locally.

Q5: How do I export Trivy reports?
Use the -f json -o report.json flag for JSON reports or --format template for custom ones.

Key Takeaways

Trivy is a lightweight, powerful, and free vulnerability scanner.
It integrates seamlessly with CI/CD pipelines and Kubernetes.
Automating scans helps maintain continuous security without manual effort.
Regular scanning and database updates minimize false positives.
A proactive vulnerability management strategy ensures secure and compliant releases.

Conclusion

Security isn’t a one-time task — it’s a continuous process. By automating vulnerability scans with Trivy, DevOps teams can shift security left, identifying and fixing issues before deployment. Trivy’s speed, accuracy, and ease of integration make it one of the best tools for DevSecOps automation in 2025 and beyond. Start small, automate often, and let your pipelines protect your production.

About the Author: Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

How to Debug Applications Running in Docker Containers

Narendra Chauhan — Wed, 29 Oct 2025 10:09:52 +0000

“Debugging is like being the detective in a crime movie where you are also the murderer.” Filipe Fortes

Introduction
Why Debugging in Docker is Important
Common Issues in Dockerized Applications
Debugging Techniques
- Using Docker Logs
- Interactive Container Shell
- Docker Exec for Live Debugging
- Network Troubleshooting
- Monitoring Tools
- Attaching Debuggers
Best Practices for Debugging Docker Applications
Interesting Facts & Statistics
Frequently Asked Questions (FAQs)
Key Takeaways
Conclusion

Introduction

Docker has revolutionized software deployment by packaging applications with their dependencies in isolated containers. However, debugging applications running inside these containers can be challenging because the traditional debugging tools may not always be directly applicable.
Debugging Docker applications involves understanding container logs, networking, runtime behavior, and sometimes interacting with the container in real-time to identify and fix issues efficiently.

Why Debugging in Docker is Important

Isolation Complexity: Containers abstract the environment, making it harder to see system-level problems.
Microservices Architecture: Modern apps often run multiple containers, so an issue in one container can affect others.
Production Debugging: Direct access to logs and live debugging in production containers helps reduce downtime.

Common Issues in Dockerized Applications

Container Won’t Start – Often caused by missing dependencies or configuration errors.
Application Crashes – Can be due to code errors, unhandled exceptions, or memory issues.
Networking Failures – Containers cannot communicate with each other or external services.
Resource Limitations – Containers may be restricted by CPU, memory, or storage limits.
Volume & File Permission Issues – Files mounted in containers may have access issues.

Debugging Techniques

4.1 Using Docker Logs

Command docker logs <container_name_or_id>
Use flags for continuous logs docker logs -f <container_name>
Helps in tracking application errors and warnings.

4.2 Interactive Container Shell

Access container shell for direct inspection: docker exec -it <container_name> /bin/bash
Inspect files, environment variables, or run debugging commands.

4.3 Docker Exec for Live Debugging

Run scripts or commands directly inside a container without restarting it docker exec -it <container_name> python manage.py shell
Useful for live debugging of services like Django, Node.js, or Java apps

4.4 Network Troubleshooting

Check connectivity between containers docker network ls docker network inspect <network_name>
Use ping or curl inside containers to verify service accessibility.

4.5 Monitoring Tools

Docker Stats: Monitor CPU, memory, network, and I/O docker stats
ctop: Terminal UI for container metrics ctop
Prometheus + Grafana:Advanced monitoring for container clusters.

4.6 Attaching Debuggers

For Node.js node --inspect=0.0.0.0:9229 app.js
For Python: Use pdb or remote-pdb inside the container.

Best Practices for Debugging Docker Applications

Use Logs Extensively – Ensure proper logging in the app
Minimize Container Complexity – Smaller images with fewer layers are easier to debug.
Reproduce Issues Locally – Replicate production issues in local containers before debugging live.
Automate Health Checks – Use Docker health checks to catch issues early.
Document Environment Differences – Know where dev, staging, and production differ.

Interesting Facts & Statistics

Over 55% of companies in 2024 use Docker for production workloads. Source:- Docker for Production workloads
Dockerized apps reduce deployment failures by up to 50% compared to traditional VMs.Source:- Dokerized vs VMs
Monitoring and proper logging can reduce debugging time by 30-40% in containerized environments.Source :- Monitoring logging and debugging

“Docker simplifies deployment but demands smarter debugging.” DevOps Engineer

FAQs

Q1. Can I debug a stopped container?
Yes, by committing it to a new image and starting it interactively:
docker commit debug-image
docker run -it debug-image /bin/bash

Q2. How do I debug multi-container apps?
Use docker-compose logs -f to aggregate logs and docker-compose exec to inspect individual services

Q3. Is it safe to debug in production?
Yes, but avoid making destructive changes. Always prefer logging, metrics, and read-only inspections when possible.

Key Takeaways

Logs and container shell access are your primary debugging tools.
Networking and resource issues are common causes of container failures
Always replicate production issues locally before live debugging
Monitoring tools and proper logging drastically reduce troubleshooting time

Conclusion

Debugging applications in Docker containers requires a combination of traditional debugging skills and container-specific techniques. By understanding container internals, using logs effectively, and leveraging interactive debugging tools, developers and DevOps engineers can quickly identify and resolve issues, ensuring stable and efficient deployments.

About the Author: Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

Troubleshooting Common DevOps Challenges

Narendra Chauhan — Fri, 19 Sep 2025 06:32:39 +0000

“Culture eats strategy for breakfast.” - Peter Drucker

Introduction
Common Challenges in DevOps
Troubleshooting Strategies for Each Challenge
Best Practices for Long-Term Success
Real-World Case Studies
Interesting Facts & Statistics
FAQs
Key Takeaways
Conclusion

1. Introduction

DevOps offers faster, more reliable software delivery by bridging development and operations. However, adopting DevOps comes with significant challenges. These challenges can be cultural, technical, or process-related. To gain maximum benefit from DevOps, organizations must learn to identify, troubleshoot, and overcome these obstacles effectively.

2. Common Challenges in DevOps

Cultural Resistance
Many organizations face hesitation from teams who are comfortable with traditional workflows. Fear of change, loss of control, and lack of trust can create resistance.
Tool Overload
The DevOps ecosystem is full of tools. Without proper selection and integration, tools can overwhelm teams and create inefficiencies instead of solving problems.
Communication Gaps
Siloed teams and unclear communication channels lead to misunderstandings, slower response times, and reduced productivity.
Security Concerns
When security is left as an afterthought, vulnerabilities appear late in the pipeline, causing major risks.
Scaling Issues
As applications grow, managing infrastructure, automation, and monitoring at scale becomes increasingly complex.
Legacy Systems
Older infrastructure and applications often don’t fit easily into modern DevOps pipelines, slowing down innovation.

3. Troubleshooting Strategies for Each Challenge

Addressing Cultural Resistance

Promote a culture of collaboration and shared responsibility.
Provide training and workshops to educate teams.
Encourage cross-functional teams to break down silos.

Managing Tool Overload

Focus on tools that integrate well with existing workflows.
Streamline tool usage by creating a centralized toolchain. strategy.
Regularly review and eliminate unnecessary tools.

Bridging Communication Gaps

Establish clear communication channels (Slack, Teams, Jira).
Encourage daily stand-ups and retrospectives.
Foster transparency with documentation and dashboards.

Handling Security Concerns

Shift security left by integrating DevSecOps practices.
Automate vulnerability scanning and compliance checks.
Provide security training for developers and operations teams.

Tackling Scaling Issues

Use cloud-native architectures with containers and Kubernetes.
Automate monitoring and scaling processes.
Design infrastructure with scalability in mind from the start.

Dealing with Legacy Systems

Gradually modernize legacy infrastructure through containerization or migration to the cloud.
Use APIs and middleware to integrate old systems with modern pipelines.
Adopt incremental modernization instead of a full rewrite.

“DevOps is not a goal, but a never-ending process of continual improvement.”- Jez Humble

4. Best Practices for Long-Term Success

Build a DevOps culture first, before focusing on tools.
Encourage continuous learning and feedback loops.
Prioritize automation wherever possible.
Implement metrics and monitoring to track progress and issues.
Keep security and compliance integrated at every stage.

5. Real-World Case Studies

Netflix: Overcame scaling challenges by moving to a cloud-native architecture and leveraging chaos engineering.
Etsy: Successfully addressed cultural resistance by fostering shared ownership of deployments.
Amazon: Embedded security in its CI/CD pipelines, making DevSecOps a standard practice.

6. Interesting Facts & Statistics

Organizations with strong DevOps practices deploy code 46 times more frequently than those without (Puppet State of DevOps Report). Source:- Deploy code 46 time
DevOps teams recover from failures 96 times faster compared to traditional teams. Source:- Failures 96 time faster
High-performing DevOps organizations spend 22% less time on unplanned work and rework. Source:- High-performing DevOps

“You can’t buy DevOps; you have to live it.”- Patrick Debois

7. FAQs

Q1: What’s the hardest DevOps challenge to solve?
Cultural resistance, because changing mindsets and habits takes time and effort.

Q2: How can security be handled effectively in DevOps?
By adopting DevSecOps and ensuring security is integrated early in the development lifecycle.

Q3: What’s the best way to deal with legacy systems in DevOps?
Gradually modernize them with cloud migration, containerization, or integration middleware.

8. Key Takeaways

DevOps challenges are inevitable but solvable with the right strategies.
Culture, communication, and collaboration matter as much as tools.
Automation, scalability, and security must be built into DevOps pipelines.
Incremental modernization helps handle legacy system constraints.

“Automation applied to an inefficient operation will magnify the inefficiency.”- Bill Gates

9. Conclusion

Troubleshooting DevOps challenges is not just about fixing issues — it’s about building resilience and adaptability. By addressing cultural, technical, and operational barriers, organizations can fully realize the promise of DevOps: faster, safer, and more innovative software delivery.

About the Author: Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

Why DevOps Is a Culture, Not a Role

Narendra Chauhan — Wed, 03 Sep 2025 11:52:26 +0000

"DevOps is not a job title it's a mindset that transforms how teams build and operate software together."

Introduction
The Problem: Misconceptions Around DevOps
DevOps as a Cultural Shift
Why DevOps Isn’t a Job Title
Interesting Stats
Real-World Impacts
FAQs
Key Takeaways
Conclusion

1. Introduction

DevOps is one of the most transformative movements in software development—but it’s also one of the most misunderstood. Many organizations treat DevOps as a role or department. They hire “DevOps Engineers” and expect instant improvements in delivery speed, reliability, and collaboration.
The truth? DevOps isn’t a role. It’s a culture.
At its core, DevOps is about breaking down silos between development and operations, fostering shared responsibility, and enabling fast, safe, and continuous delivery of software. The success of DevOps depends not on a person, but on how teams work together.

2. The Problem: Misconceptions Around DevOps

Despite widespread adoption, DevOps is often misinterpreted:

DevOps as a job title: Companies hire a “DevOps Engineer” and expect them to singlehandedly transform the pipeline.
Tool-first mentality: Teams focus on CI/CD tools but ignore culture, collaboration, and process.
Siloed responsibility: DevOps becomes a separate department rather than a shared philosophy.
Operational handoffs persist: Dev teams still throw code over the wall to ops, expecting them to run it.

This approach misses the point of DevOps entirely. Without cultural change, tooling and roles can’t deliver the full benefits.

3. DevOps as a Cultural Shift

Real DevOps success starts with mindset and collaboration, not hiring or tooling. Here's what defines a true DevOps culture:

3.1 Shared Responsibility
Both dev and ops teams are accountable for the software lifecycle—from code to customer. No more “not my job” mentalities.

3.2 Collaboration Over Handoffs
Teams work together from the start. Developers understand how their code runs in production. Ops teams understand what developers need to move fast.

3.3 Continuous Feedback Loops
Monitoring, alerting, and observability give both sides visibility into system health. Developers respond to issues. Ops provides insights during planning.

3.4 Trust and Autonomy
Developers are empowered to deploy their own code. Ops provides safe infrastructure and guardrails. Teams trust each other to own their responsibilities.

3.5 Learning from Failure
Instead of blame, teams run retrospectives. They identify process improvements and build resilience through shared learning.
A true DevOps culture means everyone owns quality, performance, and delivery speed.

4. Why DevOps Isn’t a Job Title

DevOps is not something a single person “does.” Here’s why thinking of DevOps as a role is misleading:

4.1 It Reinforces Silos
Hiring a “DevOps person” often means giving one team the burden of managing infrastructure and automation—while everyone else continues as before.

4.2 It Limits Collaboration
Teams may assume the “DevOps Engineer” handles deployment and monitoring, so developers disengage from ops work and vice versa.

4.3 It Misses the Point
DevOps is about transforming how teams collaborate, not delegating work to a specialist. Every team member must adopt the mindset.

4.4 DevOps Engineers Still Exist
That said, there’s value in roles that support DevOps culture:

Site Reliability Engineers (SREs)
Platform Engineers
Infrastructure Engineers
These roles enable and coach teams—but they don’t replace the cultural transformation.

"You can hire a DevOps Engineer, but without cultural change, you’re just adding another silo."

5. Interesting Stats

High-performing DevOps teams deploy 973x more frequently and recover 6,570x faster Source: High performing DevOps
83% of developers say DevOps improves job satisfaction Source: Developers
Companies with strong DevOps culture see 2x better customer satisfaction scores Source: customer satisfaction
DevOps practices reduce change failure rates by 3x Source: Change failure rates

6. Real-World Impacts

- Traditional Team
A developer finishes a feature, hands it off to ops, and moves on. If something breaks, the dev team says, “It worked on my machine.” The ops team scrambles to fix it without context. Tension grows. Blame circulates.

- DevOps Culture Team
Developers and ops work together from the start. Deployment scripts, monitoring, and rollback plans are part of the pull request. When an issue arises, devs and ops troubleshoot together. They learn, adapt, and improve.
Example: Netflix
Netflix embodies DevOps culture with:

Full service ownership by dev teams
Chaos engineering to test system resilience
Strong feedback loops with real-time observability This enables them to deploy thousands of times per day without compromising user experience.

"DevOps succeeds when everyone owns quality, performance, and delivery, not just one person or team."

8. FAQs

Q: Can small teams adopt DevOps culture?
A: Absolutely. In fact, small teams often adopt DevOps principles faster due to fewer silos and more direct communication.

Q: Is DevOps the same as Agile?
A: No, but they’re complementary. Agile focuses on iterative development; DevOps extends that mindset to delivery and operations.

Q: Do I need a DevOps Engineer to do DevOps?
A: Not necessarily. The focus should be on cultural and process changes. A dedicated role can help enable the shift but shouldn’t carry it alone.

Q: How do I start building DevOps culture?
A: Begin with shared goals, automate small tasks, introduce CI/CD, and improve communication between dev and ops teams.

9. Key Takeaways

DevOps is a culture, not a role or department
It emphasizes shared ownership, collaboration, and fast feedback
Hiring a “DevOps Engineer” isn’t enough—teams must adopt new behaviors
Strong DevOps culture leads to faster releases, happier teams, and more resilient systems
Start small, measure impact, and evolve continuously

10. Conclusion

DevOps is a transformative force—but only when understood and implemented as a cultural change. Hiring someone with “DevOps” in their title won’t magically solve your delivery problems. What will? Building a culture of collaboration, continuous improvement, and shared responsibility.
When DevOps becomes everyone’s job—not just a role—it becomes the foundation for high-performing teams and world-class software delivery.

About the Author: Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

Creating a CI/CD Pipeline with GitLab CI

Narendra Chauhan — Mon, 11 Aug 2025 06:48:09 +0000

"If it hurts, do it more often." Jez Humble, Co-author of Continuous Delivery

Introduction
Why CI/CD Matters
Creating a GitLab CI/CD Pipeline
Sample .gitlab-ci.yml File
Interesting Facts & Statistics
FAQs
Key Takeaways
Conclusion

1. Introduction

Continuous Integration and Continuous Deployment (CI/CD) has transformed the software development lifecycle, bringing automation, speed, and quality assurance to modern DevOps workflows.
GitLab CI is a built-in tool within GitLab that allows developers to integrate and deploy their code automatically, offering a complete DevOps platform under a single UI.

2. Why CI/CD Matters

Reduces Manual Errors: Automates builds, tests, and deployments.
Speeds Up Development: Quick feedback loops for code changes.
Improves Code Quality: Consistent testing and reviews before deployment.
Enables DevOps Culture: Promotes collaboration and delivery.

3. Creating a GitLab CI/CD Pipeline

** Prerequisites**

A GitLab project repository.
Runner registered (Shared or Custom).
.gitlab-ci.yml file in the root of your repo.

Steps
1. Create a GitLab Repository

→ Go to GitLab → New Project → Initialize with README

2. Set Up GitLab Runner
→ Install GitLab Runner on your server or use GitLab.com shared runners.
Register using:
→ gitlab-runner

3. Add a .gitlab-ci.yml File
→ This file defines your CI/CD pipeline stages, jobs, and scripts.

*4. Define Pipeline *
stages:

build
test
deploy

5. Create Jobs for Each Stage
build-job:

stage: build
 script:
   - echo "Compiling code..."

test-job:
  stage: test
  script:
    - echo "Running tests..."
deploy-job:
  stage: deploy
  script:
    - echo "Deploying app..."

6. Push Code to Trigger Pipeline

GitLab will detect the .gitlab-ci.yml file and run the pipeline automatically.
→ Sample .gitlab-ci.yml File

stages:
  - build
  - test
  - deploy

variables:
  APP_ENV: "production"

build:
  stage: build
  script:
    - npm install
    - npm run build

test:
  stage: test
  script:
    - npm test

deploy:
  stage: deploy
  only:
    - main
  script:
    - ./deploy.sh

5. Interesting Facts & Statistics

Teams using CI/CD deliver 200x more frequently than those who don’t (Source: DORA Report).Source: CI/CD DORA
Companies that adopt CI/CD reduce deployment failure rates by 75%. Source: Reduce deployment
Over 60% of organizations now use CI/CD in their SDLC (Gartner, 2023). Source: Organizations
GitLab CI/CD can reduce pipeline build times by 30-40% with caching and parallelization. Source: Optimizing build times

"CI/CD isn’t just automation, it's a shift in mindset toward responsibility and quality." Kelsey Hightower, Google Engineer

6. FAQs

Q1: Is GitLab CI/CD free to use?
Yes, GitLab offers free CI/CD minutes with shared runners. Self-managed runners are unlimited.
Q2: What languages are supported?
GitLab CI/CD is language-agnostic. You can build pipelines for Node.js, Python, Java, Go, PHP, etc.
Q3: Can I deploy to cloud services like AWS or Azure?
GitLab CI integrates with AWS, GCP, Azure, and supports deployment via CLI or APIs.
Q4: What happens if a job fails?
The pipeline will stop (unless configured otherwise). You can inspect logs and rerun failed jobs.

7. Key Takeaways

GitLab CI/CD is a powerful tool to automate your entire software delivery process.
Pipelines are defined with a .gitlab-ci.yml file in your project’s root.
Jobs are grouped into stages like build, test, and deploy.
Runners are essential agents that execute your jobs.
GitLab offers flexibility, integrations, and visibility across the DevOps lifecycle.

8.Conclusion

Creating a CI/CD pipeline with GitLab CI is not just a technical enhancement—it’s a fundamental step toward faster, safer, and more reliable software delivery. Whether you're a solo developer or managing a large engineering team, embracing CI/CD with GitLab boosts efficiency, reduces risk, and sets the stage for innovation.
Hashtags

About the Author: Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

SSL/TLS Certificates with Certbot and Nginx: The 2025 Guide

Narendra Chauhan — Fri, 25 Jul 2025 05:56:49 +0000

“HTTPS is no longer a feature, it’s the foundation of trust on the web.”— Troy Hunt, Security Researcher

Introduction
What Is SSL/TLS and Why It Matters
What Is Certbot?
How to Install an SSL Certificate with Certbot (Step-by-Step)
Common Configurations and Auto-Renewals
Key Stats & Interesting Facts
FAQs
Key Takeaways
Conclusion

1. Introduction

If you're running a website or web app in 2025 and it's not using HTTPS, you're doing it wrong. SSL/TLS certificates are no longer optional — they’re expected, even by browsers.

But the good news? It’s easier than ever to secure your websites using Certbot and Nginx — two powerful tools that make HTTPS setup simple and fast.

In this guide, you’ll learn how to issue, install, and auto-renew an SSL certificate using Certbot, all within a few minutes.

2. What Is SSL/TLS and Why It Matters

SSL (Secure Sockets Layer) and TLS (Transport Layer Security) are encryption protocols that keep data safe between your browser and server. When you visit a site with https://, you’re using TLS.

Why You Need It:

Security – Encrypts sensitive data like login details and payments
Trust – Boosts user confidence (padlock in the browser)
SEO – Google favors HTTPS sites in rankings
Compliance – Required for sites handling personal or payment data

"The internet runs on trust — and HTTPS is its currency."
— Scott Helme, Web Security Specialist

3. What Is Certbot?

Certbot is a free, open-source tool from the Electronic Frontier Foundation (EFF) that automates:

Getting SSL/TLS certificates from Let’s Encrypt
Configuring them with Nginx or Apache
Renewing them before expiry

In short, Certbot + Let’s Encrypt = free HTTPS with zero hassle.

4. How to Install an SSL Certificate with Certbot (Step-by-Step)

Let’s break it down for Ubuntu + Nginx setup:

Step 1: Install Certbot
sudo apt update
sudo apt install certbot python3-certbot-nginx

Step 2: Check Nginx is Running

sudo systemctl status nginx

If it's not active, start it: sudo systemctl start nginx

Step 3: Run Certbot with Nginx Plugin

sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com
Certbot will:

Verify your domain via HTTP challenge
Update your Nginx config
Reload Nginx with HTTPS settings

Step 4: Test HTTPS

Visit: https://yourdomain.com
You should see the padlock in your browser

5. Auto-Renewal (Hands-Free SSL Forever)

Let’s Encrypt certificates are valid for 90 days. But Certbot can auto-renew them.

Check the renewal process:
sudo certbot renew --dry-run

Certbot installs a cron job or systemd timer automatically, so you're covered.

6. Key Stats & Interesting Facts

300+ million websites use Let’s Encrypt (powered by Certbot) Source: Let’s Encrypt
SSL boosts SEO rankings — confirmed by Google Source: SEO rankings
HTTPS is now mandatory for all Chrome and Firefox features (e.g., geolocation, service workers) Source: HTTPS

“Let’s Encrypt has helped democratize encryption — now anyone can secure their site in minutes.” — Josh Aas, Executive Director of ISRG (Let’s Encrypt)

7. FAQs

Q1: Is Certbot free to use?
Yes, completely free. It works with Let’s Encrypt, which is a free certificate authority.

Q2: Do I need a domain name?
Yes. Let’s Encrypt verifies ownership of real domains via DNS or HTTP.

Q3: Can I secure subdomains?
Absolutely. Just include them in the Certbot command:
sudo certbot --nginx -d example.com -d api.example.com

Q4: What if I’m using Apache instead of Nginx?
Certbot has a plugin for Apache too:
sudo certbot --apache

Q5: Will it break my Nginx config?
Certbot is safe and creates backups. But it's always good to run:
sudo nginx -t

before and after to validate changes

8. Key Takeaways

SSL/TLS is essential for modern web security, SEO, and trust
Certbot makes HTTPS setup fast, free, and easy
Works smoothly with Nginx and Apache on most Linux servers
Auto-renewal ensures your certificate never expires
You can go from HTTP to HTTPS in under 5 minutes

9. Conclusion

Securing your website doesn’t have to be complicated. With Certbot and Nginx, you can enable HTTPS in just a few commands and forget about certificate renewal worries.
It’s 2025 — there’s no excuse to serve your app without encryption. Your users, your SEO, and your credibility depend on it.
So go ahead — grab a Let’s Encrypt certificate and give your website the security badge it deserves.

About the Author: Narendra is a DevOps Engineer at AddWebSolution, specializing in automating infrastructure to improve efficiency and reliability.

Forem: Narendra Chauhan

Nginx + PHP + MySQL Optimisations and Parameter Calculations

Table of Contents

1. Introduction

2. Architecture Overview

3. Why Optimisation Is Required

4. Nginx Optimisations & Parameter Calculations

5. PHP-FPM Optimisations & Parameter Calculations

6. MySQL Optimisations & Parameter Calculations

7. System-Level Optimisations (Linux)

8. Practical Server Size Examples

9. Interesting Facts & Statistics

11. FAQs

12. Key Takeaways

13. Conclusion

Prometheus and Grafana Monitoring for a Node.js API

Table of Contents

1. Introduction

2. Prerequisites

3. Architecture Overview

4. Step 1: Create a Sample Node.js API

5. Step 2: Dockerize the Node.js App

6. Step 3: Configure Prometheus

7. Step 4: Run Prometheus & Grafana with Docker Compose

8. Step 5: Configure Grafana Dashboard

9. Step 6: Verify Metrics & Simulate Load

Practical Demonstration (Images Explained)

10. Interesting Facts & Statistics

11. FAQs

12. Key Takeaways

13. Conclusion

Node.js Application - CI/CD with Bitbucket Pipelines on AWS EC2

Table of Contents

Introduction

Project Overview

Architecture & Deployment Flow

EC2 server setup (one-time)

Nginx reverse proxy (one-time)

Setup Bitbucket Pipelines

Pipeline Failure Scenario

Interesting Facts & Statistics

Frequently Asked Questions (FAQs)

Key Takeaways

Conclusion

How to Automate Vulnerability Scans with Trivy

Table of Contents

Introduction

What Is Trivy?

Why Vulnerability Scanning Matters

Key Features of Trivy

Installing and Setting Up Trivy

How to Run a Manual Scan

Automating Vulnerability Scans in CI/CD Pipelines

Integrating Trivy with Docker and Kubernetes

Interesting Facts & Statistics

Common Issues Explained

Best Practices for Effective Scanning

FAQs

Key Takeaways

Conclusion

How to Debug Applications Running in Docker Containers

Table of Contents

Introduction

Why Debugging in Docker is Important

Common Issues in Dockerized Applications

Debugging Techniques

Best Practices for Debugging Docker Applications

Interesting Facts & Statistics

FAQs

Key Takeaways

Conclusion

Troubleshooting Common DevOps Challenges

Table of Contents

1. Introduction

2. Common Challenges in DevOps

3. Troubleshooting Strategies for Each Challenge

4. Best Practices for Long-Term Success

5. Real-World Case Studies

6. Interesting Facts & Statistics

7. FAQs