Forem: Arif Hossain

Kubernetes-Nginx Ingress controller with Path based routing

Arif Hossain — Tue, 10 Dec 2024 11:18:59 +0000

Kubernetes Ingress Controller Setup Guide
A comprehensive guide for setting up and configuring NGINX Ingress Controller in Kubernetes, supporting both domain-based and path-based routing configurations.

Project Structure . ├── README.md ├── ingress/ │ └── controller/ │ └── nginx/ │ ├── manifests/ │ │ └── nginx-ingress.1.5.1.yaml │ └── config/ │ ├── domain-based-ingress.yaml │ └── path-based-ingress.yaml └── scripts/ └── install-helm.sh

Getting Started
Installing Helm
Helm is required for managing the NGINX Ingress Controller installation. Follow these steps to install Helm:

Download Helm

curl -o /tmp/helm.tar.gz -LO https://get.helm.sh/helm-v3.10.1-linux-amd64.tar.gz

Extract the archive

tar -C /tmp/ -zxvf /tmp/helm.tar.gz

Move helm binary to path

mv /tmp/linux-amd64/helm /usr/local/bin/helm

Make it executable

chmod +x /usr/local/bin/helm

Setting up Ingress Controller
Add the NGINX Ingress repository:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm search repo ingress-nginx --versions

Set Version Variables:

CHART_VERSION="4.4.0"
APP_VERSION="1.5.1"

Version Selection Guide:

The CHART_VERSION refers to the Helm chart version
The APP_VERSION refers to the NGINX Ingress Controller version
Always check the compatibility matrix
For production environments, use stable versions
Generate the Installation Manifest:

# Create directory structure
`mkdir -p ./ingress-controller/nginx/manifests/`

# Generate manifest using helm template
`helm template ingress-nginx ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--version ${CHART_VERSION} \
--namespace ingress-nginx \
> ./ingress-controller/nginx/manifests/nginx-ingress.${APP_VERSION}.yaml`

Modify Service Configuration:

Open the generated manifest file
Locate the Service configuration
Change the service type to NodePort
Configure ports:
HTTP: 30080
HTTPS: 30443
Deploy the Controller:

# Create namespace
`kubectl create namespace ingress-nginx`

# Apply the configuration
`kubectl apply -f ./ingress-controller/nginx/manifests/nginx-ingress.${APP_VERSION}.ya`ml

Configuration Guide
Deploy an app
deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: blog-app
  labels:
    app: blog-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: blog-app
  template:
    metadata:
      labels:
        app: blog-app
    spec:
      containers:
      - name: blog-app
        image: <YOUR-DOCKER-HUB>/blog-app:1.7
        ports:
        - containerPort: 80

service.yml

apiVersion: v1
kind: Service
metadata:
  name: blog-app-service
spec:
  selector:
    app: blog-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: ClusterIP

Path-based Routing
For routing based on URL paths:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: blog-service
spec:
  ingressClassName: nginx
  rules:
  - host: <YOUR-DOMAIN-NAME>
    http:
      paths:
      - path: /blog
        pathType: Prefix
        backend:
          service:
            name: blog-service
            port:
              number: 80

Rewrite Rules
For advanced path rewriting:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: blog-service
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
  - host: <YOUR-DOMAIN-NAME>
    http:
      paths:
      - path: /path-a(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: blog-service
            port:
              number: 80

Local Development
For local development and testing:

Port Forwarding Setup:

kubectl port-forward -n ingress-nginx service/ingress-nginx-controller 8080:80

For development and testing at poridhi lab, make sure you are using load balancer. Expose the ingress controller.

Testing Configuration:

Test the endpoint-

curl -H "Host: blog.<YOUR-DOMAIN-NAME>" http://localhost:8080/blog

For HTTPS-

curl -k -H "Host: blog.<YOUR-DOMAIN-NAME>" https://localhost:8443/blog
Troubleshooting
Common issues and their solutions:

Controller Pod Issues

Check pod status-

kubectl get pods -n ingress-nginx

View controller logs-

kubectl logs -n ingress-nginx deploy/ingress-nginx-controller

Describe pod for events-

kubectl describe pod -n ingress-nginx <pod-name>

Routing Problems

Verify ingress resource-

kubectl describe ingress <ingress-name>

Check endpoints-

kubectl get endpoints

Validate service-

kubectl describe service <service-name>

Common Error Solutions:

404 Not Found:
Verify path configuration and service endpoint

503 Service Unavailable:
Check if backend service is running

502 Bad Gateway:
Validate service port configuration

Self-Hosted GitHub Actions Runner in Kubernetes

Arif Hossain — Mon, 11 Nov 2024 18:21:07 +0000

Self Hosted Runner in K3s Cluster

Hands-on Guide: Self-Hosted GitHub Actions Runner in Kubernetes
GitHub Actions is a powerful tool for automating software workflows, and it can be used to build, test, and deploy code right from GitHub. It provides a way to automate repetitive tasks and can be integrated with many popular tools and platforms.

GitHub Actions can use two types of runners:

Hosted and
Self-hosted.

Hosted runners are provided by GitHub and run on virtual machines in the cloud.

Self-hosted runners are machines that you set up and manage yourself. They run on your infrastructure, and you can customize them to meet your needs.

Introduction
This guide provides step-by-step instructions for setting up a self-hosted GitHub Actions runner in Kubernetes using Docker-in-Docker (DinD). This setup allows you to run GitHub Actions workflows in your own infrastructure.

Prerequisites
Before starting, ensure you have:

A working Kubernetes cluster (k3s/kind/etc.)
kubectl installed and configured
Docker installed
A GitHub account and repository
A GitHub Personal Access Token (PAT)

Step 1: Setting Up the Project Structure
Create a new directory for the project:

mkdir github-runner-k8s
cd github-runner-k8s

Create necessary files:

touch Dockerfile entrypoint.sh kubernetes.yaml

Step 2: Creating the Dockerfile
The Dockerfile creates a container image that serves as our GitHub Actions runner environment. It creates a reproducible environment for the runner and installs necessary tools (Docker CLI, jq, etc.). It forms the base container image for the runner pod in Kubernetes.

docker-integration

Create the Dockerfile with the following content:

FROM debian:bookworm-slim
ARG RUNNER_VERSION="2.302.1"
ENV GITHUB_PERSONAL_TOKEN ""
ENV GITHUB_OWNER ""
ENV GITHUB_REPOSITORY ""

# Install Docker
RUN apt-get update && \
    apt-get install -y ca-certificates curl gnupg
RUN install -m 0755 -d /etc/apt/keyrings
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
RUN chmod a+r /etc/apt/keyrings/docker.gpg

# Add Docker repository
RUN echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  tee /etc/apt/sources.list.d/docker.list > /dev/null
RUN apt-get update

# Install required packages
RUN apt-get install -y docker-ce-cli sudo jq

# Setup github user
RUN useradd -m github && \
    usermod -aG sudo github && \
    echo "%sudo ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

# Create directories with correct permissions
RUN mkdir -p /actions-runner && \
    chown -R github:github /actions-runner && \
    mkdir -p /work && \
    chown -R github:github /work

USER github
WORKDIR /actions-runner

# Download and install runner
RUN curl -Ls https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz -o actions-runner.tar.gz && \
    tar xzf actions-runner.tar.gz && \
    rm actions-runner.tar.gz && \
    sudo ./bin/installdependencies.sh

COPY --chown=github:github entrypoint.sh /actions-runner/entrypoint.sh
RUN sudo chmod u+x /actions-runner/entrypoint.sh

ENTRYPOINT ["/actions-runner/entrypoint.sh"]

Step 3: Creating the Entrypoint Script
The entrypoint script handles the runner's lifecycle - registration, execution, and cleanup. It automatically registers the runner with GitHub. It acts as the bridge between container and GitHub.

entrypoint

Create entrypoint.sh with the following content:

#!/bin/sh
registration_url="https://api.github.com/repos/${GITHUB_OWNER}/${GITHUB_REPOSITORY}/actions/runners/registration-token"
echo "Requesting registration URL at '${registration_url}'"
payload=$(curl -sX POST -H "Authorization: token ${GITHUB_PERSONAL_TOKEN}" ${registration_url})
export RUNNER_TOKEN=$(echo $payload | jq .token --raw-output)

./config.sh \
    --name $(hostname) \
    --token ${RUNNER_TOKEN} \
    --labels my-runner \
    --url https://github.com/${GITHUB_OWNER}/${GITHUB_REPOSITORY} \
    --work "/work" \
    --unattended \
    --replace

remove() {
    ./config.sh remove --unattended --token "${RUNNER_TOKEN}"
}

trap 'remove; exit 130' INT
trap 'remove; exit 143' TERM

./run.sh "$*" &
wait $!

Make the script executable:

chmod +x entrypoint.sh

Step 4: Creating the Kubernetes Deployment
This step defines how the runner should be deployed and managed in Kubernetes.

k8s-integration

Create kubernetes.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: github-runner
  labels:
    app: github-runner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: github-runner
  template:
    metadata:
      labels:
        app: github-runner
    spec:
      containers:
      - name: github-runner
        imagePullPolicy: Never
        image: github-runner:latest
        env:
        - name: GITHUB_OWNER
          valueFrom:
            secretKeyRef:
              name: github-secret
              key: GITHUB_OWNER
        - name: GITHUB_REPOSITORY
          valueFrom:
            secretKeyRef:
              name: github-secret
              key: GITHUB_REPOSITORY
        - name: GITHUB_PERSONAL_TOKEN
          valueFrom:
            secretKeyRef:
              name: github-secret
              key: GITHUB_PERSONAL_TOKEN
        - name: DOCKER_HOST
          value: tcp://localhost:2375
        volumeMounts:
        - name: data
          mountPath: /work/
      - name: dind
        image: docker:24.0.6-dind
        env:
        - name: DOCKER_TLS_CERTDIR
          value: ""
        resources:
          requests:
            cpu: 20m
            memory: 512Mi
        securityContext:
          privileged: true
        volumeMounts:
          - name: docker-graph-storage
            mountPath: /var/lib/docker
          - name: data
            mountPath: /work/
      volumes:
      - name: docker-graph-storage
        emptyDir: {}
      - name: data
        emptyDir: {}

Step 5: Building and Loading the Image
Build the Docker image:

docker build . -t github-runner:latest

Load the image into your Kubernetes cluster:
For k3s:

# Save the image
docker save github-runner:latest -o github-runner.tar

# Import to k3s
sudo k3s ctr images import github-runner.tar

Step 6: Creating GitHub Token
Go to GitHub Settings → Developer settings → Personal access tokens → Tokens (classic)
Generate new token with following permissions:
repo (full control)
workflow
admin:org (if using organization repository)

Step 7: Creating Kubernetes Secrets
Create namespace:

kubectl create namespace host-runner

Create secrets (replace placeholder values):

kubectl -n host-runner create secret generic github-secret \
  --from-literal=GITHUB_OWNER=<your-github-username> \
  --from-literal=GITHUB_REPOSITORY=<your-repo-name> \
  --from-literal=GITHUB_PERSONAL_TOKEN=<your-github-token>

Step 8: Deploying to Kubernetes
Apply the Kubernetes deployment:

kubectl -n host-runner apply -f kubernetes.yaml

Verify the deployment:

# Check pod status
kubectl -n host-runner get pods

# Check runner logs
kubectl -n host-runner logs -f <pod-name> -c github-runner

Step 9: Verifying the Setup
Go to your GitHub repository
Navigate to Settings → Actions → Runners
You should see your self-hosted runner listed and "Idle"
Troubleshooting Guide
Common Issues and Solutions
Image Pull Error

# Check if image is properly loaded
sudo crictl images | grep github-runner

# If not visible, reload the image
sudo k3s ctr images import github-runner.tar
**Permission Issues**
# Check pod logs
kubectl -n host-runner logs -f <pod-name> -c github-runner

# Verify secrets
kubectl -n host-runner get secrets github-secret -o yaml

Runner Not Registering

# Check if token is valid
kubectl -n host-runner logs <pod-name> -c github-runner | grep "Requesting registration URL"

# Verify network connectivity
kubectl -n host-runner exec <pod-name> -c github-runner -- curl -s https://api.github.com

Cleanup Instructions
To remove the setup:

# Delete the deployment
kubectl -n host-runner delete -f kubernetes.yaml

# Delete the secrets
kubectl -n host-runner delete secret github-secret

# Delete the namespace
kubectl delete namespace host-runner

# Remove local images
docker rmi github-runner:latest

Testing the Runner
Create a simple workflow in your repository:

# .github/workflows/test.yml
name: Test Self-Hosted Runner
on: [push]
jobs:
  test:
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v2
      - name: Test Runner
        run: |
          echo "Hello from self-hosted runner!"
          docker --version

Commit and push this file to your repository
Check the Actions tab in your repository to see the workflow running
Maintenance Tips
Updating Runner Version:
Update RUNNER_VERSION in Dockerfile

Rebuild and redeploy
Scaling Runners:
Modify replicas in kubernetes.yaml
Apply changes with kubectl

Monitoring:

# Check runner status
kubectl -n host-runner get pods -w

# Check resource usage
kubectl -n host-runner top pod

Remember to:

Regularly update the runner version
Monitor resource usage
Rotate GitHub tokens periodically
Keep Docker images updated

Benefits of using Self-Hosted Runner:

Improved Performance: By hosting your runners, you can ensure that the build and deployment processes are faster and more reliable, as you have complete control over the hardware and networking resources.

Increased Security: GitHub self-hosted runners can be configured to run on your own servers, which provides an extra layer of security compared to using shared runners. This helps to protect sensitive information and data in your workflows.

Customizable Environments: With self-hosted runners, you can create custom environments that match your exact needs, including specific software versions and configurations.

Cost-Effective: If you have a large number of workflows or use cases that require a lot of resources, self-hosted runners can be more cost-effective than using GitHub’s shared runners or cloud-based solutions.

High Availability: With self-hosted runners, you can set up Horizontal Runner Autoscaler, which provides redundancy and high availability for your workflows.

Greater Control: Self-hosted runners give you complete control over the resources and environment used for your workflows, which can help you optimize performance and ensure that your builds and deployments run smoothly.

Overall, GitHub self-hosted runners offer greater flexibility, control, and customization options than using shared runners or cloud-based solutions. However, setting up and managing self-hosted runners requires additional effort and resources, so it’s essential to weigh the benefits against the costs and resources needed to maintain them.

Conclusion
In conclusion, the Actions Runner Controller with Self-Hosted Runner is a powerful tool that can help you improve your GitHub Actions workflows. It simplifies the management of runners by automating tasks such as the creation, scaling, and deletion of runners. It also provides an easy way to create and manage runners in a Kubernetes cluster. By using Actions Runner Controller with Self-Hosted Runner, you can take full advantage of the power of GitHub Actions while having full control over your infrastructure.

Actions Runner Controller (ARC) Setup Guide Using Kubernetes-cluster.

Arif Hossain — Sat, 09 Nov 2024 16:39:59 +0000

Actions Runner Controller (ARC)
This guide explains how to set up GitHub Actions Runner Controller (ARC) in a Kubernetes cluster.

Prerequisites

Kubernetes cluster
kubectl installed and configured
GitHub Personal Access Token (PAT) with repo and workflow permissions

Installing Helm (if not installed)

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

Quick Setup
Install ARC Controller

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Create namespace and install controller

helm install arc \
  --namespace arc-systems \
  --create-namespace \
  oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller

# Verify installation
kubectl get pods -n arc-systems

Create GitHub PAT
Go to GitHub → Settings → Developer Settings → Personal Access Tokens
Create new token with repo and workflow permissions
Save the token securely
Install Runner Set

# Set your GitHub details
export GITHUB_CONFIG_URL="https://github.com/YOUR_USERNAME/YOUR_REPO"
export GITHUB_PAT="your_pat_here"

# Install runner set
helm install arc-runner-set \
    --namespace arc-runners \
    --create-namespace \
    --set githubConfigUrl="${GITHUB_CONFIG_URL}" \
    --set githubConfigSecret.github_token="${GITHUB_PAT}" \
    oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-

scale-set
Using in GitHub Actions

In your workflow file (`.github/workflows/example.yml`)

name: My Workflow
on: [push]
jobs:
  build:
    runs-on: arc-runner-set
    steps:
      - uses: actions/checkout@v2
      - run: echo "Hello from self-hosted runner!"

Verification
Check if everything is running:

kubectl get pods -n arc-systems
kubectl get pods -n arc-runners

Your runner should appear online in your GitHub repository under Settings → Actions → Runners.

In Another workflow file (`.github/workflows/example.yml`)

name: CI Pipeline

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  # Build Job
  build:
    runs-on: arc-runner-set
    steps:
      - name: Check out code
        uses: actions/checkout@v2

      - name: Build Application
        run: echo "Building application..."

  # Test Job
  test:
    runs-on: arc-runner-set
    needs: build
    steps:
      - name: Check out code
        uses: actions/checkout@v2

      - name: Run Tests
        run: echo "Running tests..."

  # Lint Job
  lint:
    runs-on: arc-runner-set
    needs: build
    steps:
      - name: Check out code
        uses: actions/checkout@v2

      - name: Lint Code
        run: echo "Linting code..."

  # Deploy Job
  deploy:
    runs-on: arc-runner-set
    needs: [build, test, lint]
    steps:
      - name: Check out code
        uses: actions/checkout@v2

      - name: Deploy Application
        run: echo "Deploying application..."

Possible outputs:

Troubleshooting
If runners show as offline:

Check runner pods: kubectl get pods -n arc-runners
View logs: kubectl logs -n arc-runners <pod-name>
Verify PAT permissions and expiration
Ensure correct repository URL in configuration
Check Namespace and Pod Status
Ensure both namespaces, arc-systems and arc-runners, are created and pods are running without errors.
Run:

kubectl get pods -n arc-systems
kubectl get pods -n arc-runners

If any pods are in a CrashLoopBackOff or Error state, inspect their logs for errors:

kubectl logs <pod-name> -n arc-systems
kubectl logs <pod-name> -n arc-runners

Cleanup
To remove ARC:

helm uninstall arc-runner-set -n arc-runners
helm uninstall arc -n arc-systems

Basic GitHub Actions Checkout

Arif Hossain — Sat, 09 Nov 2024 13:39:08 +0000

Overview
This hands-on lab introduces the fundamentals of GitHub Actions by implementing a basic workflow that demonstrates repository checkout and command execution. You'll learn how to create a workflow file, understand its structure, and execute various commands.

Prerequisites

GitHub account
A GitHub repository where you have write access
Basic understanding of YAML syntax
Basic knowledge of command line operations
Learning Objectives
After completing this lab, you will be able to:
Create a basic GitHub Actions workflow file
Understand workflow trigger conditions
Execute single and multi-line commands
Use the checkout action
View and interpret workflow results

Lab Structure

your-repository/
├── .github/
│   └── workflows/
│       └── basic-checkout.yml
└── README.md

Implementation Guide
Step 1: Create Workflow Directory

mkdir -p .github/workflows

Step 2: Create Workflow File
Create .github/workflows/basic-checkout.yml with the following content:

name: Basic Checkout Lab

# Trigger on push to main branch
on:
  push:
    branches:
      - main

# Define jobs and their steps
jobs:
  basic-checkout:
    runs-on: ubuntu-latest
    steps:
      # Step 1: Checkout the repository
      - name: Checkout repository
        uses: actions/checkout@v4

      # Step 2: List repository contents
      - name: List files
        run: ls -la

      # Step 3: Display system information
      - name: Show system info
        run: |
          echo "Repository: $GITHUB_REPOSITORY"
          echo "Operating System: $(uname -a)"
          echo "Current Directory: $(pwd)"

      # Step 4: Check software versions
      - name: Check versions
        run: |
          echo "Node version: $(node --version)"
          echo "Python version: $(python --version)"
          echo "Git version: $(git --version)"

Detailed Explanation
Workflow Trigger
on:
push:
branches:
- main
This section configures the workflow to run whenever code is pushed to the main branch.

Job Configuration
jobs: basic-checkout: runs-on: ubuntu-latest
Defines a single job named basic-checkout
Specifies Ubuntu as the runner operating system
Steps Breakdown
`Checkout Step

name: Checkout repository uses: actions/checkout@v4Uses the official checkout action Clones your repository into the runner Essential for accessing repository filesFile Listing
name: List files run: ls -laLists all files and directories Includes hidden files Shows file permissions and ownership System Information- name: Show system info run: | echo "Repository: $GITHUB_REPOSITORY" echo "Operating System: $(uname -a)" echo "Current Directory: $(pwd)"Uses multi-line commands Demonstrates environment variable usage Shows system detailsVersion Checks
name: Check versions run: | echo "Node version: $(node --version)" echo "Python version: $(python --version)" echo "Git version: $(git --version)"` Checks installed software versions Uses command substitution Demonstrates multi-line commands Expected Outputs

File Listing Output
total 24
drwxr-xr-x 4 runner docker 4096 Feb 20 10:00 .
drwxr-xr-x 3 runner docker 4096 Feb 20 10:00 ..
drwxr-xr-x 8 runner docker 4096 Feb 20 10:00 .git
drwxr-xr-x 3 runner docker 4096 Feb 20 10:00 .github
-rw-r--r-- 1 runner docker  654 Feb 20 10:00 README.md

System Information Output

Repository: username/repository-name
Operating System: Linux runner-ubuntu-latest 5.15.0-1047-azure
Current Directory: /home/runner/work/repository-name/repository-name
Version Information Output
Node version: v20.11.0
Python version: Python 3.10.12
Git version: git version 2.42.0

Practice Exercises
Add New Commands Modify the workflow to include:

Current date and time
Available disk space
Memory usage Custom Environment Variables Add environment variables:

env: CUSTOM_MESSAGE: "Hello from GitHub Actions!"
Conditional Execution Add a conditional step:

- name: Conditional step if: github.event_name == 'push' run: echo "This was triggered by a push event"
Troubleshooting
Common Issues and Solutions
Workflow Not Triggering

Verify branch name matches trigger condition
Check workflow file syntax
Ensure Actions are enabled in repository settings
Checkout Action Fails

Verify Git configuration

git config --global --list

Check permissions

ls -la .git

Command Execution Errors

Check command availability on runner
Verify syntax for multi-line commands
Ensure proper environment variable usage
Debugging Tips
Enable debug logging by setting secret ACTIONS_RUNNER_DEBUG to true
Check workflow run logs in GitHub Actions tab
Use echo statements to debug variables

Additional Resources:

Deploy The Multi-Container App

Arif Hossain — Sat, 09 Nov 2024 13:08:28 +0000

Deploy the multi-container application using Docker Compose
In this session, we will learn how to define services, networks, and volumes in a Docker Compose file and verify the setup by running a simple application.

We'll need to set up the directory structure, create the necessary files (app.py, Dockerfile, compose.yaml), and then deploy and verify the application.

Here's a step-by-step guide:

Step 1: Set Up Directory Structure
Create a directory for your project:

mkdir multi-container
cd multi-container

Step 2: Create app.py
Inside the multi-container directory, create a file named app.py:

from flask import Flask
from redis import Redis

app = Flask(__name__)
redis = Redis(host='redis', port=6379)

@app.route('/')
def hello():
    count = redis.incr('hits')
    return f'Hello World! This page has been visited {count} times.\n'

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

The app.py script creates a simple web application using Flask. When accessed, it counts the number of visits to the root URL and displays a message with the visit count. This count is stored in a Redis database. The script runs a web server to serve the application on port 8080.

Step 3: Create Dockerfile
Create a Dockerfile in the same directory:

# Use the official Python image from the Docker Hub
FROM python:3.9-alpine

# Set the working directory in the container
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY . /app

# Install any needed packages specified in requirements.txt
RUN pip install flask redis

# Make port 8080 available to the world outside this container
EXPOSE 8080

# Define environment variable
ENV NAME World

# Run app.py when the container launches
CMD ["python", "app.py"]

The Dockerfile builds a Docker image for the web-fe service in the Docker Compose setup. It uses the official Python 3.9 Alpine image, sets the working directory to /app, copies the current directory's contents into the container, installs Flask and Redis, exposes port 8080, and runs the Flask application (app.py) when the container starts. This setup ensures the web-fe service is ready to serve the application and interact with the Redis service defined in the compose.yaml file.

Step 4: Create compose.yaml
Create a compose.yaml file:

version: '3.8'

services:
  web-fe:
    build: .
    command: python app.py
    ports:
      - target: 8080
        published: 5001
    networks:
      - counter-net
    volumes:
      - type: volume
        source: counter-vol
        target: /app

  redis:
    image: "redis:alpine"
    networks:
      - counter-net

networks:
  counter-net:

volumes:
  counter-vol:

The compose.yaml file defines a multi-container application using Docker Compose. It consists of two services:

web-fe:

Builds an image using the current directory's Dockerfile.
Runs a Python script (app.py) to serve a Flask application.
Maps port 5001 on the host to port 8080 in the container.
Connects to the counter-net network.
Mounts a volume named counter-vol to the /app directory in the container.
redis:

Pulls the redis:alpine image from Docker Hub.
Connects to the counter-net network.
The counter-net network facilitates communication between the web-fe and redis services. Additionally, the counter-vol volume provides persistent storage for the web-fe service.

It’s also worth knowing that Compose builds networks and volumes before deploying services. This makes sense, as networks and volumes are lower-level infrastructure objects that are consumed by services (containers).

Step 5: Deploy the Application
With all the files in place, you can now deploy the application using Docker Compose. Run the following command from the multi-container directory:

docker compose -f compose.yaml up -d

Step 6: Verify the Deployment
Verify that the services are running:

docker compose ps

You should see two services: web-fe and redis.

Expected output:

Check the logs for any errors:

docker compose logs

Expected output:

Use the following command:

curl localhost:5001

You should see the message indicating the number of visits to the page.

Expected output:

List the processes running inside of each service (container):

docker compose top

Use the following command to stop the app without deleting its resources:

docker compose stop

With the app in the stopped state, restart it with the docker compose restart command:

docker compose restart

Verify the volume and network:
List volumes to verify counter-vol:

docker volume ls

List networks to verify counter-net:

docker network ls

Cleanup
When you're done testing, you can stop and remove the containers, networks, and volumes:

docker compose down --volumes

###//Understand another Example//###
Let's investigate docker-compose.yml to understand.

version: '3.5'
services:
  api:
    build: .
    volumes:
      - "./app:/src/app"
    ports:
      - "1338:1338"
    depends_on:
      - db
      - cache
    networks:
        - test_nw
    environment:
      - DATABASE_HOST=mongodb://db:27017
      - REDIS_CACHE_HOST=redis://cache:6379
      - PORT=1338
  db:
    image: mongo:latest
    ports:
      - "27017:27017"
    networks:
        - test_nw
  cache:
    image: redis:latest
    ports:
      - "6379:6379"
    networks:
      - test_nw
networks:
  test_nw:
    driver: bridge

This is a simple docker-compse.yml example to deploy a Nodejs backend with MongoDb and Redis.

The Commands
Of course docker-compose has commands. Whaat! Okay I' will give you the most basic commands. I'm not super duper software engineer and I can live my life with these commands.
docker compose up is a command that will look for docker-compose.yml by default and will process the docker-compose.yml, create the environment and run the services.
-d means that terminal is yours, it runs the command detachable mode
-f #non-standard-compose.yml-name# means that you can pass a compose.yml file with different name. Usaually projects contains more than one docker-compose files. You can have compose file for production and development or you can seperate applications and tools in a different compose files.
docker compose down is a command that will look for running compose.yml file and shutdown containers then remove all of them including networks, volumes etc.
docker compose log is a command that will look for running compose.yml file and displays log which are generated by the containers.
Top Level Definitions
version : Defined by the Compose Specification for backward compatibility. It is only informative, and you'll receive a warning message that it is obsolete if used.

services : A service is an abstract definition of a computing resource within an application which can be scaled or replaced independently from other components. These services run in their containers and can communicate with each other.

network : A layer that allows containers to communicate with each other.

volume: Volumes are persistent data stores implemented by the container engine. Compose offers a neutral way for services to mount volumes, and configuration parameters to allocate them to infrastructure.

Service Definitions
build: Lets you define the Dockerfile path to build when the compose file is being processed.

volumes : Lets you define service-level persisted volumes to mount local files and folders.

ports: Lets you expose container ports. The left-hand of the definition is the localhost address and the right-hand will be the container port. It basically means binding port 1338 of the container to localhost:1338.

depends_on : option in Docker Compose is used to specify the dependencies between different services defined in your docker-compose.yml file.

networks: option in Docker Compose is used to specify which network will be used by this container.

environment: option in Docker Compose is used to specify environment variables which will be passed to container.

This setup allows you to run and verify the multi-container Flask application using Docker Compose, with persistent storage and network configuration as described.

Introduction to Docker Compose

Arif Hossain — Sat, 09 Nov 2024 12:20:01 +0000

Intro to Docker Compose
In this session we will learn the basics of Docker Compose and how to manage multi-container applications easily.

What are Multi-Container Applications?
Modern cloud-native applications are often composed of multiple smaller services that work together to form a complete application.

This is known as the microservices pattern. These could include:

Web front-end
Database
Authentication service
and so on.

Challenges with Microservices:
Managing and deploying multiple microservices can be complex and cumbersome, requiring careful orchestration of each service.

What is Docker Compose?
Docker Compose is a tool that allows you to define and manage multi-container Docker applications. It uses a declarative configuration file, typically in YAML format, to specify the services, networks, and volumes required for the application.

Benefits of Docker Compose:
Simplifies the orchestration of multi-container applications.
Allows for a single configuration file to define and manage all services.
Integrates with version control systems for better management.
Basic Docker Compose Commands
Check Installation:

docker compose version

Start an Application:

docker compose up

Stop an Application:

docker compose down

View Container Status:

docker compose ps

Simple Docker Compose Example
Let's create a basic Docker Compose setup with a web server using nginx:alpine image and a redis service using redis:alpine image.

Step 1: Create a Directory
Create a directory for your project.

mkdir myapp
cd myapp

Step 2: Create Docker Compose File
Create a file named docker-compose.yml and add the following content:

version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"
  redis:
    image: redis:alpine

This file defines two services: a web server (nginx) and a Redis server.

Step 3: Start the Application
Run the following command to start the services:

docker compose up

You should see output indicating the services are starting.

Expected output:

Step 4: Verify the Setup
Check localhost:

curl localhost

You should see the default Nginx welcome page.

Expected output:

Step 5: View Status
Check the status of the running containers:

docker compose ps
#or
docker ps

Expected output:

Step 6: Stop the Application
Stop the services with:

docker compose down

=====####====
Let's discuss Another Example:
Let's investigate docker-compose.yml to understand.

version: '3.5'
services:
  api:
    build: .
    volumes:
      - "./app:/src/app"
    ports:
      - "1338:1338"
    depends_on:
      - db
      - cache
    networks:
        - test_nw
    environment:
      - DATABASE_HOST=mongodb://db:27017
      - REDIS_CACHE_HOST=redis://cache:6379
      - PORT=1338
  db:
    image: mongo:latest
    ports:
      - "27017:27017"
    networks:
        - test_nw
  cache:
    image: redis:latest
    ports:
      - "6379:6379"
    networks:
      - test_nw
networks:
  test_nw:
    driver: bridge

This is a simple docker-compse.yml example to deploy a Nodejs backend with MongoDb and Redis.

The Commands
Of course docker-compose has commands. Whaat! Okay I' will give you the most basic commands. I'm not super duper software engineer and I can live my life with these commands.

docker compose up

is a command that will look for docker-compose.yml by default and will process the docker-compose.yml, create the environment and run the services.
-d means that terminal is yours, it runs the command detachable mode
-f #non-standard-compose.yml-name# means that you can pass a compose.yml file with different name. Usually projects contains more than one docker-compose files. You can have compose file for production and development or you can seperate applications and tools in a different compose files.

docker compose down

is a command that will look for running compose.yml file and shutdown containers then remove all of them including networks, volumes etc.
docker compose log

is a command that will look for running compose.yml file and displays log which are generated by the containers.
Top Level Definitions

version :
Defined by the Compose Specification for backward compatibility. It is only informative, and you'll receive a warning message that it is obsolete if used.
services :
A service is an abstract definition of a computing resource within an application which can be scaled or replaced independently from other components. These services run in their containers and can communicate with each other.
network :
A layer that allows containers to communicate with each other.
volume:
Volumes are persistent data stores implemented by the container engine. Compose offers a neutral way for services to mount volumes, and configuration parameters to allocate them to infrastructure.

Service Definitions

build:
Lets you define the Dockerfile path to build when the compose file is being processed.
volumes :
Lets you define service-level persisted volumes to mount local files and folders.
ports:
Lets you expose container ports. The left-hand of the definition is the localhost address and the right-hand will be the container port. It basically means binding port 1338 of the container to localhost:1338.
depends_on :
option in Docker Compose is used to specify the dependencies between different services defined in your docker-compose.yml file.
networks:
option in Docker Compose is used to specify which network will be used by this container.
environment:
option in Docker Compose is used to specify environment variables which will be passed to container.

Conclusion
You have learned the basics of Docker Compose, created a simple multi-container application, and practiced using basic commands. This knowledge will help you manage more complex applications in the future.

Multi-stage Builds with a Simple Node.js App

Arif Hossain — Sat, 09 Nov 2024 12:03:51 +0000

Multi-stage Builds with a Simple Node.js App
In the world of making programs run smoothly inside containers, we've found that using just one step to build them isn't always the best idea. When we stick to single steps, our containers can end up being really big and slow. But fear not! There's a better way: multi-stage builds. These help us make our containers smaller and faster without losing any important stuff.

Today, we're going on an adventure to discover how multi-stage builds work. We'll break down the complicated bits into easy-to-understand pieces.

Create a Simple Node.js App
Create a directory for our project and navigate into it:

mkdir node-app
cd node-app

Now, create a file named app.js and add the following code:

const http = require('http');

const server = http.createServer((req, res) => {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello from Docker! Welcome to our application.\n');
});

const PORT = process.env.PORT || 3000;
server.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}/`);
});

This simple Node.js application creates an HTTP server that listens on port 3000 and responds with "Hello, world!" to all requests.

Create a Dockerfile
Next, let's create a Dockerfile to containerize our Node.js application using two-stage builds:

# Stage 1: Build the Node.js application
FROM node:latest AS build

WORKDIR /app

COPY app.js .

# Stage 2: Create the production image
FROM node:slim

WORKDIR /app

COPY --from=build /app .

EXPOSE 3000

CMD ["node", "app.js"]

Stage 1 (build):
Purpose: Build the Node.js application.
Functionality:
Uses node:latest image as the base, providing necessary build tools.
Sets the working directory to /app where application code will be copied.
Copies index.js into the image, preparing for application build.
Stage 2:
Purpose: Create the production image.
Functionality:
Uses node:slim image as the base, providing a minimal runtime environment.
Sets the working directory to /app, ensuring consistency with the build stage.
Copies the built application from the build stage into this stage, excluding unnecessary build tools.
Exposes port 3000, allowing external communication with the application.
Defines the command to run the application (node index.js), starting the HTTP server.
Build and Run the Docker Image
Now, let's build and run the Docker image:

docker build -t node-app .
docker run -d -p 3000:3000 node-app

Check that the container is running:

docker ps

This will build the Docker image based on the Dockerfile and run a container based on that image. The application will be accessible at http://localhost:3000.

Testing the App
To test the app, we can use curl to send a request to the running container:

curl http://localhost:3000

This should return:

Conclusion
In this guide, we've demonstrated how to use two-stage builds in Docker to containerize a simple Node.js application. Each stage has a specific purpose, allowing us to create smaller and more efficient Docker images by separating the build process into distinct steps.

Containerize a Single-Container App

Arif Hossain — Sat, 09 Nov 2024 07:53:23 +0000

Containerize a Single-Container App
In this guide, we will walk through the process of containerizing a simple Node.js application. Containerization involves packaging an application and all its dependencies, libraries, and configurations into a single package known as a container. This ensures the app runs consistently across different environments, whether it's your local machine, a server, or the cloud.

Docker is the tool we'll use for containerization. It allows developers to automate the deployment of applications inside lightweight, portable containers. We’ll also explore how Docker layers work and their significance in the containerization process.

Creating the Application Code
First, let's create a basic Node.js application. We will begin by creating a directory for the app and navigating into it:

mkdir my-node-app
cd my-node-app

Next, create a file named app.js with the following content:

// app.js
const http = require('http');

const hostname = '0.0.0.0';
const port = 8080;

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello, World!\n');
});

server.listen(port, hostname, () => {
  console.log(`Server running at http://${hostname}:${port}/`);
});

This is a simple HTTP server that responds with "Hello, World!".

Creating the Dockerfile
A Dockerfile is a script that contains instructions for Docker to build your container image. Create a Dockerfile in the same directory with the following content:

# Use an official Node.js runtime as the base image
FROM node:14-alpine

# Set the working directory
WORKDIR /usr/src/app

# Copy the application code
COPY app.js .

# Install the necessary Node.js package
RUN npm install http@0.0.1-security

# Expose the port the app runs on
EXPOSE 8080

# Define the command to run the app
CMD ["node", "app.js"]

FROM: Specifies the base image (Node.js with Alpine Linux, a lightweight version of Linux).
WORKDIR: Sets the working directory inside the container.
COPY: Copies app.js from the local machine to the container.
RUN: Installs the necessary Node.js package.
EXPOSE: Informs Docker the container listens on port 8080.
CMD: Defines the command to run the app.
Containerizing the App / Building the Image
Now that the Dockerfile is ready, we can build the Docker image. In your terminal, run:

docker build -t my-node-app:1.0 .

This command tells Docker to build an image named my-node-app with the version 1.0 using the current directory.

To verify that the image was created, use the following command:

docker images

This will display a list of available images.

Pushing the Image (Optional)
If you'd like to share your image, you can push it to Docker Hub. Follow these steps:

docker login

Tag the image with your Docker ID:

docker tag my-node-app:1.0 <docker-id>/my-node-app:1.0

Push the image:

docker push <docker-id>/my-node-app:1.0

This will make the image available for others to pull and use.

Running the App
To run the containerized app, execute the following command:

docker run -d --name my-node-app-container -p 80:8080 my-node-app:1.0

-d: Runs the container in detached mode (in the background).
--name: Assigns a name to the running container.
-p 80:8080: Maps port 8080 inside the container to port 80 on your machine.
Check that the container is running by typing:

docker ps

This will list all active containers.

Testing the App
To test the app, you can use the curl command or open a browser and go to http://localhost. Using curl, run:

curl http://localhost:80

Docker Layers and Efficiency
Each instruction in the Dockerfile creates a layer. These layers are cached and reused during subsequent builds if the content doesn’t change, which improves build speed and reduces storage space. For example:

FROM creates a base layer.
COPY adds a new layer with the application files.
RUN installs dependencies in a separate layer.
You can inspect the layers of your image by running:

docker history my-node-app:1.0

This command displays each command from the Dockerfile and the corresponding layer it created.

Optimizing Docker Images: Multi-Stage Builds
To reduce image size, you can use multi-stage builds, especially for production environments. In multi-stage builds, unnecessary files (such as development dependencies) are not included in the final image.

Here’s an example of a multi-stage Dockerfile:

# Stage 1: Build
FROM node:14-alpine AS builder
WORKDIR /usr/src/app
COPY app.js .
RUN npm install http@0.0.1-security

# Stage 2: Production
FROM node:14-alpine
WORKDIR /usr/src/app
COPY --from=builder /usr/src/app .
EXPOSE 8080
CMD ["node", "app.js"]

In this setup, the first stage builds the app, and the second stage creates a minimal image without including the build tools, reducing the final image size.

Container Management and Best Practices
Restart Policies
You can ensure that containers automatically restart if they crash or your system reboots. Add the --restart flag when running your container:

docker stop my-node-app-container
docker rm -f  my-node-app-container
docker build -t my-node-app:2.0 .
docker images

docker run -d --name my-node-app-container --restart always -p 80:8080 my-node-app:2.0

Environment Variables
Environment variables allow you to pass dynamic configurations into your container at runtime. For example:

docker run -d -e NODE_ENV=production -p 80:8080 my-node-app:2.0

Security Considerations

Use Official Base Images:
Always start with trusted base images to reduce security risks.
Scan for Vulnerabilities: Regularly scan your images for known vulnerabilities using tools like Trivy or Docker Scan.
Limit Privileges:
Run containers with the least privileges needed by using Docker’s security options.

Conclusion
Containerizing applications with Docker ensures consistency, portability, and ease of deployment across various environments. By following best practices like using official base images, optimizing Docker layers, and employing security measures, you can create efficient and secure containerized applications.

This guide provides a clear step-by-step process for containerizing a Node.js app while explaining Docker's functionality, Dockerfile instructions, and optimization strategies. Happy containerizing!

Exploring Docker Image Layers and Size Management

Arif Hossain — Sat, 09 Nov 2024 07:14:25 +0000

Exploring Docker Image Layers and Size Management
Docker images are built from layers, where each layer represents a set of filesystem changes. The size of an image on disk is the sum of the sizes of its component layers. Docker allows you to commit changes to a running container, creating new image layers.

This lab will guide you through these concepts with hands-on practices, focusing on creating and modifying a Docker image with ubuntu as the base image. You will install and remove software within containers, observe the changes in image sizes, and understand the impact of Docker's Union File System (UFS) on image size.

Task: Building and Modifying Docker Images
Here, you will create a Docker image from the official Ubuntu image, install Git within a container, and commit the changes to create a new image. You will then modify this image by removing Git and observe how Docker manages image layers and size.

Steps:

Pull the ubuntu image from Docker Hub.

docker pull ubuntu

This command fetches the latest ubuntu image from Docker Hub and stores it in your local Docker repository.

Create a Container and Install Git:
Create a container from the ubuntu image and install Git.

docker run -d --name ubuntu-git-container ubuntu sleep infinity
docker exec -it ubuntu-git-container apt-get update
docker exec -it ubuntu-git-container apt-get install -y git

These commands run a container named ubuntu-git-container from the ubuntu image and install Git inside the container. The sleep infinity command keeps the container running. The apt-get update and apt-get install -y git commands update the package list and install Git, respectively.

Commit the Changes to Create a New Image:
Commit the container to create a new image with Git installed.

docker commit ubuntu-git-container ubuntu-git:1.0
docker tag ubuntu-git:1.0 ubuntu-git:latest

These commands commit the current state of the ubuntu-git-container container to a new image named ubuntu-git with a tag 1.0, and then tag this image as latest.

Check Image Sizes:
Check the sizes of all the images created.

docker images

Expected output:

Remove Git:
Create a new container from the ubuntu-git image and remove Git.

docker run --name ubuntu-git-remove --entrypoint /bin/bash ubuntu-git:latest -c "apt-get remove -y git"

This command runs a container named ubuntu-git-remove from the ubuntu-git:latest image with an entrypoint set to /bin/bash, and removes Git from the container.

Commit the Changes to Create a New Image with Git Removed:
Commit the container to create a new image with Git removed.

docker commit ubuntu-git-remove ubuntu-git:2.0
docker tag ubuntu-git:2.0 ubuntu-git:latest

These commands commit the current state of the ubuntu-git-remove container to a new image named ubuntu-git:removed and reassign the latest tag to this new image.

Check Image Sizes:
Check the sizes of all the images created.

docker images

Expected output:

Notice that even though you removed Git, the image actually same in size. Although you could examine the specific changes with docker diff, you should be quick to realize that the reason for the increase has to do with the union file system.

Remember, UFS will mark a file as deleted by actually adding a file to the top layer. The original file and any copies that existed in other layers will still be present in the image. When a file is deleted, a delete record is written to the top layer, which overshadows any versions of that file on lower layers.

It’s important to minimize image size for the sake of the people and systems that will be consuming your images. If you can avoid causing long download times and significant disk usage with smart image creation, then your consumers will benefit.

Modifying Docker Image Attributes

Arif Hossain — Sat, 09 Nov 2024 06:51:05 +0000

Modifying Docker Image Attributes
This lab aims to deepen your understanding of Docker image attributes and how they can be modified and inherited across different layers of an image.

By the end of this exercise, you will be familiar with setting environment variables, working directories, exposed ports, volume definitions, container entrypoints, and commands.

Description
When you use docker container commit, you create a new layer for an image. This layer includes not only a snapshot of the filesystem but also metadata about the execution context. The following parameters, if set for a container, will be carried forward to the new image:

Environment variables
Working directory
Exposed ports
Volume definitions
Container entrypoint
Command and arguments If these parameters are not explicitly set, they will be inherited from the original image.

This lab will provide you with hands-on experience in modifying these attributes and observing how they are inherited across image layers.

Create a Container with Environment Variables

Run a Docker container from the busybox:latest image, setting two environment variables.
Commit the running container to a new image.
Modify the Entrypoint and Command

Run a new container from the previously committed image, setting a new entrypoint and command.

Commit this container to update the image.
Verify Inheritance of Attributes

Run a container from the final image without specifying any command or entrypoint to verify that the environment variables and the entrypoint/command are inherited correctly.

Solution Steps
Create a Container with Environment Variables
Run the container:

docker run --name container1 -e ENV_EXAMPLE1=value1 -e ENV_EXAMPLE2=value2 busybox:latest

This command creates a new container named container1 from the busybox:latest image and sets two environment variables, ENV_EXAMPLE1 and ENV_EXAMPLE2.

Commit the container to a new image:

docker commit container1 new-image

This command commits the container1 container to a new image named new-image.

Varify the image creation using the following command:

docker images

Run a new container with a specific entrypoint and command:

docker run --name container2 --entrypoint "/bin/sh" new-image -c "echo \$ENV_EXAMPLE1 \$ENV_EXAMPLE2"

This command runs a new container named container2 from the new-image image, setting the entrypoint to /bin/sh and the command to -c "echo \$ENV_EXAMPLE1 \$ENV_EXAMPLE2". This setup will print the values of the environment variables.

Expected output:

Commit this container to update the image:

docker commit container2 new-image

This will commit the container container2 to the new-image image, updating the image with the new entrypoint and command. The updated image will have the entrypoint and the command with it.

Verify the new image's entrypoint and command settings:

docker inspect --format '{{ .Config.Entrypoint }}' new-image
docker inspect --format '{{ .Config.Cmd }}' new-image

Expected output:

Run a container from the final image to verify the inherited behavior:

docker run --rm new-image

This command runs a container from the final new-image image, verifying that the environment variables and the entrypoint/command are inherited correctly.

Expected output:

By completing this lab, we hope, you have a practical understanding of how to modify and verify Docker image attributes, and how these changes are inherited across image layers.

Reviewing Filesystem Changes

Arif Hossain — Sat, 09 Nov 2024 06:40:46 +0000

Reviewing filesystem changes
In this lab, we will learn how to review filesystem changes made inside a Docker container. Docker provides a command to list all changes to the filesystem, including added, changed, or deleted files and directories.

When you read a file from a union filesystem, that file will be read from the topmost layer where it exists. If a file was not created or changed on the top layer, the read will fall through the layers until it reaches a layer where that file does exist. Here is a simple example:

The layer functionality is hidden by the UFS. No special actions are required by the software running in a container to utilize these features. The UFS manages the complexity of handling files across multiple layers.

Most union filesystems use something called copy-on-write, which is easier to under stand if you think of it as copy-on-change. When a file in a read-only layer (not the top layer) is modified, the whole file is first copied from the read-only layer into the writable layer before the change is made.

In this illustration, files are added, changed, deleted, and added again over a range of three layers.

Task

Create a container and add a new file and review changes
Create another container and delete an existing file and review changes
Create yet another container and change an existing file and review changes
Clean up our workspace Steps Follow these steps to complete the task:

Create a container and add a new file:

Command:

docker container run --name tweak-a busybox:latest touch /HelloWorld

This command runs a new container named tweak-a from the busybox:latest image and creates an empty file named /HelloWorld inside the container.

Review the filesystem changes:

docker container diff tweak-a

This command lists all the changes made to the filesystem of the tweak-a container. You should see:

Expected Output:

A /HelloWorld
This indicates that the file /HelloWorld was added.

Create another container and delete an existing file:

Command:

docker container run --name tweak-d busybox:latest rm /bin/vi

This command runs a new container named tweak-d from the busybox:latest image and deletes the file /bin/vi.

Review the filesystem changes:

docker container diff tweak-d

This command lists all the changes made to the filesystem of the tweak-d container. You should see:

Expected Output:

This indicates that the directory /bin was changed and the file /bin/vi was deleted.

Create yet another container and change an existing file:

Command:

docker container run --name tweak-c busybox:latest touch /bin/vi

This command runs a new container named tweak-c from the busybox:latest image and updates the modification time of the file /bin/vi.

Review the filesystem changes:

docker container diff tweak-c
This command lists all the changes made to the filesystem of the tweak-c container. You should see:

This indicates that the directories /bin and /bin/busybox were changed.

Clean up our workspace by removing the containers:

docker container rm -vf tweak-a
docker container rm -vf tweak-d
docker container rm -vf tweak-c

These commands forcefully remove the tweak-a, tweak-d, and tweak-c containers, cleaning up our workspace.

Explanation of docker container diff Output
Lines that start with an A indicate files that were added.
Lines that start with a C indicate files that were changed.
Lines that start with a D indicate files that were deleted.
By following these steps, you'll be able to track and understand filesystem changes within Docker containers. This is particularly useful for debugging and for ensuring that your containerized applications behave as expected.

#/Commands Summary
Create a container and add a new file:
docker container run --name tweak-a busybox:latest touch /HelloWorld

#Review the filesystem changes:
docker container diff tweak-a

#Create another container and delete an existing file:
docker container run --name tweak-d busybox:latest rm /bin/vi

#Review the filesystem changes:
docker container diff tweak-d

#Create yet another container and change an existing file:
docker container run --name tweak-c busybox:latest touch /bin/vi

#Review the filesystem changes:
docker container diff tweak-c

#Clean up your workspace:
docker container rm -vf tweak-a
docker container rm -vf tweak-d
docker container rm -vf tweak-c

Create and Commit an Ubuntu Container with Git Installed

Arif Hossain — Sat, 09 Nov 2024 06:20:44 +0000

Create and Commit an Ubuntu Container with Git Installed
This session will guide us through creating an Ubuntu container, installing Git, and committing the changes to a new image. Additionally, we'll learn how to set an entrypoint to make using the image more efficient.

Task
We will perform the following steps:

Create an ubuntu container and open a bash session.
Install git inside the container and verify the git.
Create new image by using commit.
Set the entrypoint for the new image to make it easier to use.

Simple Explanation of the Process
In this lab, we will start by creating a container from the Ubuntu image and open a bash session within it. Inside this container, we will install git and verify that the installation was successful by checking the git version. After exiting the container, we will commit these changes to create a new Docker image that includes git. We will run a container from that image.

Finally, we will set an entrypoint for this new image to make it easier to use git directly without needing to specify the git command each time we start a container from this image.

Steps
Create an Ubuntu container and open a bash session:

docker run -it --name image-dev ubuntu:latest /bin/bash

This command creates a new container named image-dev from the ubuntu:latest image and opens an interactive bash session.

Install Git inside the container:

apt-get update
apt-get install -y git

This updates the package list and installs Git in the container.

Verify the Git installation by checking its version:

git --version
Expected output:

root@d15b204bcec5:/# git --version
git version 2.43.0

This command confirms that Git was installed correctly.

Exit the container:

exit

This command exits the interactive bash session and returns to the host terminal.

Review the filesystem changes and commit these changes to create a new image:

docker container commit -a "@arif" -m "Added git" image-dev ubuntu-git

This command commits the changes made in the image-dev container to a new image named ubuntu-git with an author tag and a commit message.

Remove the modified container:

docker container rm -vf image-dev

This command forcefully removes the image-dev container to clean up.

Verify the new image by checking the Git version in a new container:

docker container run --rm ubuntu-git git --version

This command runs a temporary container from the ubuntu-git image to verify that Git is installed correctly.

Setting the Entrypoint to Git
Create a new container with the entrypoint set to Git:

docker container run --name cmd-git --entrypoint git ubuntu-git

This command creates a new container named cmd-git with the entrypoint set to git, showing the standard Git help and exiting.

Commit the new image with the entrypoint:

docker container commit -m "Set CMD git" -a "@poridhi" cmd-git ubuntu-git
This command commits the changes to the ubuntu-git image, setting the entrypoint to Git.

Remove the modified container:

docker container rm -vf cmd-git

This command forcefully removes the cmd-git container to clean up.

Test the new image:

docker container run --name cmd-git ubuntu-git version

This command runs a new container from the ubuntu-git image, verifying that the entrypoint is set correctly and showing the Git version:

git version 2.43.0

This setup ensures that any container started from the ubuntu-git image will automatically use Git as the entrypoint, making it easier for users to work with Git directly.

Forem: Arif Hossain

Kubernetes-Nginx Ingress controller with Path based routing

Download Helm

Extract the archive

Move helm binary to path

Make it executable

Test the endpoint-

For HTTPS-

Check pod status-

View controller logs-

Describe pod for events-

Verify ingress resource-

Check endpoints-

Validate service-

Self-Hosted GitHub Actions Runner in Kubernetes

Actions Runner Controller (ARC) Setup Guide Using Kubernetes-cluster.

In your workflow file (.github/workflows/example.yml)

In Another workflow file (.github/workflows/example.yml)

Basic GitHub Actions Checkout

Verify Git configuration

Check permissions

Additional Resources:

Deploy The Multi-Container App

Introduction to Docker Compose

Multi-stage Builds with a Simple Node.js App

Containerize a Single-Container App

Exploring Docker Image Layers and Size Management

Modifying Docker Image Attributes

Reviewing Filesystem Changes

Create and Commit an Ubuntu Container with Git Installed

In your workflow file (`.github/workflows/example.yml`)

In Another workflow file (`.github/workflows/example.yml`)