Forem: Deborah Emeni

How to run a multi-container app with Docker compose

Deborah Emeni — Sun, 01 Feb 2026 09:16:49 +0000

What you'll learn

By the end of this section, you'll:

Understand what Docker Compose is and why it’s useful for managing multi-container applications
Learn how a docker-compose.yml file defines and runs an entire application
Run multiple services together as a single system instead of starting containers manually
Configure environment variables inside Docker Compose
Enable communication between services using Docker’s built-in networking
Persist database data using volumes
Deploy a Node.js API with MongoDB using one command
Start, stop, and manage your whole application using Docker Compose commands

Why managing containers manually doesn’t scale

Before Docker Compose, the usual way to run a multi-container setup is to start each container separately, then manually connect everything together.

If you’ve been following along in this series, you already know the building blocks:

Dockerfiles package your application into an image
Networking allows containers to communicate
Volumes preserve your data between restarts

Individually, these are simple.

But once your project has more than one container, the setup quickly turns into a long list of commands you need to remember and repeat every time you run the app.

Take a look at what the manual approach can feel like:

docker network create app-net
docker run mongo
docker run node-api
docker run -e MONGO_URL=...

This works, but there are some clear problems:

too many commands to manage
easy to forget flags or the correct order
harder for teammates to follow
not easily reproducible across machines

At some point, it stops feeling like “running an application” and starts feeling like manually configuring containers to work together.

There has to be a simpler way to treat these containers as one application.

What is Docker Compose?

Docker Compose is a tool that lets you define and run multiple containers as a single application.

Instead of starting containers individually and remembering a long list of commands, you describe everything in one place, then start the entire system together.

In practice, that means:

you define your services in one file
Docker sets up the network for you
Docker creates volumes for you
and you start everything with a single command

Here’s the mental model I want you to keep in mind:

a Dockerfile describes one container
a docker-compose.yml file describes your whole application

So rather than managing containers separately, you treat them as one system.

You’ll see terms like services, networks, and volumes as we go. Don’t worry about them yet. We’ll learn them naturally in the hands-on sections.

For now, just remember: Docker Compose helps you run multiple containers together like one application.

Understanding the docker-compose.yml structure

When most people see a YAML file for the first time, it can feel a little intimidating with all the many indentation and keys.

And it looks more complicated than it really is.

So instead of pasting a huge file and trying to explain everything at once, let’s build it gradually.

We’ll start small and add each part step by step.

Let’s start with the smallest possible Compose file:

services:
  app:
    image: node

That’s it.

Let’s break this down together:

services → the containers we want Docker to run
app → the name of our service (you can choose this)
image → the image Docker should use to create the container

So with just these few lines, we’ve already told Docker:

“Start one container called app using the Node image.”

From here, we’ll keep extending the same file.

As our application becomes more complex, we’ll add things like:

ports to expose the app to our browser
environment for configuration values
volumes for persistent data
depends_on to control startup order

The key idea is simple: we don’t write everything at once.

We add one piece, understand it, test it, then move on.

That way, the file never feels overwhelming, and you always know exactly what each line is doing.

Next, we’ll put all of this into practice with a hands-on project and apply these concepts in an actual setup.

Setting up the demo project

Before we start writing our docker-compose.yml file, we need a small project to run. The goal is not to build a feature-rich API. We just need something simple that can connect to a database so we can focus on learning Docker Compose.

If you prefer to skip the setup, you can clone the complete project here: https://github.com/d-emeni/node-api-compose-demo

git clone https://github.com/d-emeni/node-api-compose-demo.git
cd node-api-compose-demo

Prerequisites

To follow along, you should already have:

Docker installed and working on your machine (Follow this guide)
a basic understanding of Docker images and containers (read up here if you're new to containers)
familiarity with Dockerfiles (you have already seen this earlier in the series)
basic Node.js knowledge (enough to run a small API)

If you have gone through the earlier posts in this series, you are in a good place to continue.

What we’re building

We’ll build a small Node.js API that:

connects to MongoDB
saves data
returns it

That gives us a realistic setup for Docker Compose without adding unnecessary complexity.

Project structure

This is the structure we’ll work with:

node-api/
 index.js
 package.json
 package-lock.json
 Dockerfile
 .dockerignore
 .gitignore
 docker-compose.yml
 README.md

Don’t worry if some of these files look unfamiliar. We’ll walk through the important ones as we go.

Step 1: Creating our `docker-compose.yml` file

Now that we have a working Node.js API, the next step is to run the API and MongoDB as a single application using Docker Compose.

This is where Compose starts to feel useful.

Instead of:

starting MongoDB manually
starting the API manually
remembering flags like ports, environment variables, and container names

We’re going to describe the entire setup in one file, then run everything together.

Create the file

In the root of your project (the same level as your Dockerfile), create a file named:

docker-compose.yml

In the next section, we’ll start with the smallest Compose configuration and build it up step by step.

Step 2: Start with the smallest working Compose file

Let’s start small and build up from there.

Add this to your docker-compose.yml:

services:
  app:
    build: .
    ports:
      - "3000:3000"

Let’s break down what this means:

services is where we define the containers our application needs
app is the name of our first service (this is your Node.js API)
build: . tells Docker Compose to build an image using the Dockerfile in the current folder
ports: "3000:3000" maps port 3000 in the container to port 3000 on your machine

Run it

From the project root, run:

docker compose up --build

If everything is set up correctly, Docker will build the image and start your API container.

You should see Docker building the image, creating the container, and then starting the Node.js server logs.

Don’t worry if you see a MongoDB connection error at the end. That’s expected for now, because we haven’t added MongoDB yet.

Your output should look similar to this:

Step 3: Add MongoDB as a service

Right now, our Compose file only starts the API container. That’s why the app fails to connect to MongoDB.

The fix is simple:

add a MongoDB service
point our Node.js app to that service using an environment variable

Update your `docker-compose.yml`

Replace the contents of your docker-compose.yml with this:

services:
  app:
    build: .
    ports:
      - "3000:3000"
    environment:
      MONGO_URL: mongodb://mongo:27017
      DB_NAME: compose_demo
    depends_on:
      - mongo

  mongo:
    image: mongo:latest
    ports:
      - "27017:27017"

What changed?

mongo is a new service running the official MongoDB image
MONGO_URL is now mongodb://mongo:27017 (the service name becomes the hostname)
depends_on tells Docker Compose to start MongoDB before starting the API

Run it again

If your previous Compose run is still active, stop it with:

Ctrl + C

Then start everything again:

docker compose up --build

When you run the command, Docker Compose will go through two phases.

First, it builds and prepares everything for you.

You’ll see Docker pulling the MongoDB image (if you don’t already have it), building your Node.js image from the Dockerfile, and creating the containers.

It should look something like this:

After the images are built, Docker starts both containers and begins streaming their logs.

You should see MongoDB starting up first, followed by your API connecting to it.

If everything is working, look for a message like:

Connected to MongoDB at mongodb://mongo:27017

Your output should look similar to this:

Step 4: A quick recap of what just happened

Before we move on, let’s pause for a moment and connect the dots.

With one docker-compose.yml file, we:

built our Node.js image
started a MongoDB container
created a shared network automatically
connected both services together
and launched the entire application with a single command

Instead of running multiple docker run commands and configuring networking manually, you defined the system once and Docker Compose handled the setup for you.

That’s really the core idea behind Docker Compose.

You describe your application in one place, and Docker takes care of creating and running the containers.

Now that everything is up and running, let’s test the API and confirm our application behaves as expected.

Step 5: Testing the application

At this point, both containers are running:

the Node.js API
the MongoDB database

They’re connected through Docker Compose, and your app should already be talking to MongoDB in the background.

Now let’s quickly verify that everything works as expected.

Step 1: Check the health endpoint

Open a new terminal and run:

curl http://localhost:3000/health

You should see:

{ "status": "ok" }

This confirms that:

the API is running
the server is reachable
the container started correctly

Step 2: Save some data

Now let’s store something in the database.

curl -X POST http://localhost:3000/notes \
  -H "Content-Type: application/json" \
  -d '{"text":"hello from docker compose"}'

You should get a response similar to:

{
  "message": "Note created",
  "id": "...",
  "note": {
    "text": "hello from docker compose"
  }
}

This tells us the API successfully wrote data to MongoDB.

Step 3: Read the data back

curl http://localhost:3000/notes

You should see your saved note returned in the response.

If you can create and read notes successfully, your containers are:

running
connected
and communicating correctly

At this point, you officially have a multi-container application running with Docker Compose.

In the next section, we’ll make this setup more practical by adding persistent storage so your database data survives container restarts.

Step 6: Adding persistent storage with volumes

Right now, everything works.

You can create notes, read them back, and the API communicates with MongoDB correctly.

But there’s one problem.

If you stop and remove the containers, all your database data disappears.

That’s because containers are ephemeral by default. When a container is deleted, its filesystem is deleted too.

Let’s prove that quickly.

Stop everything

Press:

Ctrl + C

Then remove the containers:

docker compose down

Now start everything again:

docker compose up

If you try:

curl http://localhost:3000/notes

You’ll notice your notes are gone.

The database started fresh.

This is not what we want in our applications.

We need our data to survive container restarts.

What are volumes?

Docker volumes are persistent storage managed by Docker.

Think of them as:

storage outside the container
that containers can mount and reuse
even after they are stopped or deleted

So instead of storing MongoDB data inside the container, we store it in a volume.

If you need more practical knowledge about volumes, we covered it in our previous series: "Run a MySQL container with persistent storage using Docker volumes"

Update your docker-compose.yml

Add a volume to the MongoDB service:

services:
  app:
    build: .
    ports:
      - "3000:3000"
    environment:
      MONGO_URL: mongodb://mongo:27017
      DB_NAME: compose_demo
    depends_on:
      - mongo

  mongo:
    image: mongo:latest
    ports:
      - "27017:27017"
    volumes:
      - mongo_data:/data/db

volumes:
  mongo_data:

What this does

- mongo_data:/data/db

This means:

mongo_data → Docker volume
/data/db → where MongoDB stores its files inside the container

So MongoDB now saves data to the volume instead of the container filesystem.

Even if the container is removed, the volume remains.

Test it

Run:

docker compose up --build

Docker will rebuild the containers and create a new volume for MongoDB. Watch for a line that says the volume was created. Your output should look similar to this:

Create a note again to test persistence (so we have some data to test with):

curl -X POST http://localhost:3000/notes \
  -H "Content-Type: application/json" \
  -d '{"text":"hello again after restart"}'

Then stop everything:

docker compose down

Start it again:

docker compose up

Now check:

curl http://localhost:3000/notes

Your data should still be there.

That’s persistence working.

In the next section, we’ll clean things up further by using environment variables more cleanly with a .env file.

Step 7: Managing configuration with a `.env` file

So far, everything works.

But our docker-compose.yml is starting to look a little noisy.

Right now we have configuration values hardcoded directly inside the file:

database URL
database name
ports

This works for small demos, but in your actual projects it can quickly become chaotic.

As your app becomes more complex, you don’t want to edit the Compose file every time you change:

a port
an environment variable
or a setting for a different environment (dev, staging, production)

Instead, we separate configuration from infrastructure.

That’s where a .env file helps.

What is a `.env` file?

A .env file stores environment variables in one place.

Docker Compose automatically reads this file and replaces variables inside docker-compose.yml.

So instead of hardcoding values, we reference them.

This keeps your setup:

cleaner
easier to change
and more portable

Create a `.env` file

In your project root, create:

touch .env

Add:

PORT=3000
MONGO_URL=mongodb://mongo:27017
DB_NAME=compose_demo

Update `docker-compose.yml`

Now replace the hardcoded values with variables:

services:
  app:
    build: .
    ports:
      - "${PORT}:3000"
    environment:
      MONGO_URL: ${MONGO_URL}
      DB_NAME: ${DB_NAME}
    depends_on:
      - mongo

  mongo:
    image: mongo:latest
    volumes:
      - mongo_data:/data/db

volumes:
  mongo_data:

What changed?

Instead of:

MONGO_URL: mongodb://mongo:27017

We now use:

MONGO_URL: ${MONGO_URL}

Docker Compose reads the value from .env.

So if you ever want to change something, you only edit one file.

No touching the Compose config.

Run it again

docker compose up --build

This time, Docker doesn’t rebuild everything from scratch. It simply recreates the containers and reuses the existing volume, so startup is much faster. Your output should look similar to this:

Everything should behave exactly the same.

The difference is that your configuration is now cleaner and easier to manage, and you can change values without editing the docker-compose.yml file.

This becomes more valuable as your application becomes more complex or when you deploy to different environments.

In the next section, we’ll look at a few everyday Docker Compose commands that make managing multi-container apps much easier.

Everyday Docker Compose commands

Now that everything is running, let’s look at a few commands you’ll use regularly when working with Docker Compose.

These make it much easier to start, stop, rebuild, and debug your application during development.

Start all services

docker compose up

Builds (if needed) and starts every service defined in docker-compose.yml.

Start in the background (detached mode)

docker compose up -d

Runs containers in the background so your terminal stays free.

This is how you’ll usually run your app during development.

Rebuild after code changes

docker compose up --build

Forces Docker to rebuild images before starting containers.

Useful when you change your Dockerfile or dependencies.

View logs

docker compose logs

See logs from all services.

Follow logs live:

docker compose logs -f

Very helpful for debugging.

Check running containers

docker compose ps

Shows which services are running and their ports.

Stop everything

docker compose down

Stops and removes containers and the network.

Your data stays safe because it’s stored in volumes.

Reset everything (including database data)

docker compose down -v

Removes containers and volumes.

This wipes the database and gives you a completely fresh start.

Use this when testing or troubleshooting.

At this point, you should feel comfortable managing your entire multi-container app with just a few simple commands.

Wrap up

You started with a single container and gradually built up to a complete multi-container setup.

Along the way, you learned how to:

run multiple services with Docker Compose
connect containers using service names
persist data with volumes
clean up configuration using environment variables
manage everything with simple Compose commands

You now have a small but realistic Node.js + MongoDB application running exactly how many real-world projects run in development.

What’s next in the Docker learning series?

So far, everything has been running locally on your machine.

You now know how to package an app into containers, run multiple services with Docker Compose, connect them together, and manage them in development.

But local environments are only half the story.

In the next part of this series, we’ll move beyond your laptop and deploy containers to the cloud.

You’ll:

understand cloud container platforms like AWS ECS, Google Cloud Run, and Azure Containers
deploy a containerized app to the cloud
manage and run containers in cloud environments
hands-on: deploy an API container to Google Cloud Run (and try the same on AWS and Azure)

By the end, you’ll be able to deploy your containers beyond your local machine and into the cloud.

Run a MySQL container with persistent storage using Docker volumes

Deborah Emeni — Tue, 02 Dec 2025 07:17:37 +0000

What you'll learn

By the end of this section, you'll:

Understand what happens to data inside a container and why it does not stay once the container is removed.
Learn what Docker volumes are and how they help you keep data safe even when containers stop or get recreated.
See the difference between bind mounts and named volumes, and know which one to use in simple development situations.
Learn how a container reads and writes data to a location outside its own filesystem using volumes.
Complete a hands-on project where you run a MySQL container with persistent storage so your database stays intact across restarts.

We’ll begin by looking at how container storage works and why volumes are needed.

Why do containers lose data?

Before we start working with volumes, it helps to understand how container storage works.

Every container has its own isolated filesystem. It's like a temporary workspace that only that container can see.

When you start a container, Docker creates a thin writable layer on top of the image. Any changes the container makes, such as logs, uploaded files, and database entries, go into this writable layer.

The key point is that this layer is not designed to stay around forever. Once you stop or remove the container, Docker wipes the writable layer along with anything stored inside it. The image remains, but your data does not.

A quick example to visualize this

Let's say you run a MySQL container, create a database table, and insert some rows.

As long as the container is running, everything looks fine. But if you delete that container and start a new one using the same image, MySQL starts up as if it is a fresh installation.

The data you added earlier is gone because it lived only inside the old container’s temporary filesystem.

This is why we need a way to store data outside the container’s lifecycle. And that is exactly what Docker volumes provide.

What are Docker volumes?

Docker volumes give containers a place to store data that lives beyond the container itself.

So, instead of writing everything to the container’s short-lived filesystem, a volume provides a separate storage location that Docker manages for you.

A better way to understand it is this: the container has its own internal workspace, but a volume sits outside that workspace. When you attach a volume to a container, the container can read from it and write to it as if it were part of its own filesystem, even though the data is actually stored elsewhere.

Why are Docker volumes used?

Docker handles all the behind-the-scenes work. It creates the volume, keeps track of where it lives on your machine, and makes sure it is available whenever a container needs it. You do not have to manage the actual folder path or worry about accidentally deleting it.

Volumes are used because writing data directly inside a container is not reliable. If the container is removed, everything inside it disappears.

By using a volume, the data stays safe, and you can start fresh containers that still have access to the same information. This is very helpful for databases, logs, or anything you expect to keep long-term.

Bind mounts vs named volumes - what’s the difference?

When you attach external storage to a container, you’re choosing how the container should access data that lives outside its own filesystem. The two ways to do this are bind mounts and named volumes, and each one fits a different situation.

Bind mounts

A bind mount directly links a folder on your host machine into the container, so both sides see the same files immediately.

Named volumes

A named volume is storage created and managed by Docker, giving your container a stable place to store data without relying on your host’s folder structure.

See this table below that summarizes this comparison

Let's look at a simple way to compare both approaches so you can decide when to use each one:

Feature	Bind mounts	Named volumes
Where data lives	A folder on your host machine	Docker-managed storage
Best for	Local development, hot-reloading	Databases, production workloads
Portability	Low (path depends on the host)	High (Docker manages it)
Reliability	Tied to host system setup	More stable and predictable
Visibility	Easy to browse directly on your machine	Harder to inspect manually
Permissions	Can be tricky across OSes	Handled by Docker

How do containers store and retrieve data using volumes?

Before we run our MySQL example, I'll explain what happens when a container uses a volume.

Containers normally write data into their own internal filesystem, but when you attach a volume, you’re telling the container:

“Store this data somewhere safer, outside your temporary filesystem.”

Every container keeps its data in specific directories. For MySQL, that directory is:

/var/lib/mysql

This is where MySQL writes tables, logs, metadata, and all database files.

If this path stays inside the container, the data disappears when the container is removed. But if we mount a volume to this path, something different happens.

What exactly happens when a volume is mounted?

So, for example, when you attach a volume to /var/lib/mysql:

Docker gives the container a storage location that lives outside the container.
Anything MySQL writes goes into the volume instead of the container’s own filesystem.
If you delete the container and start a new one with the same volume, the new container can see all the old files immediately.

This is why volumes are used for databases. They let the container come and go, while the data stays in place.

What does the data flow look like, in simple terms?

The container starts.
Docker mounts the volume into the container at a specific path.
The application (like MySQL) reads/writes files as if the folder was part of its normal filesystem.
Docker handles the actual storage behind the scenes.

The key idea is this: the container thinks it’s writing to its own filesystem, but the data is safely stored in the volume instead.

Hands-on project: Run a MySQL container with persistent storage using Docker volumes

Now that you understand what volumes are and how containers use them, let’s put it all together by running a MySQL container whose data survives restarts and rebuilds.

We’ll go step by step so you not only run the container but also understand the importance of each step.

Before you begin

To follow along with the hands-on MySQL project, make sure you have the following set up:

Docker installed and running
You should have Docker Desktop (macOS/Windows) or Docker Engine (Linux) installed.
If you’ve completed the earlier parts of this series, your setup should already be ready. (If not, follow this guide to make sure Docker is installed and running.)
Basic familiarity with running containers
You should know how to use commands like docker run, docker stop, and docker exec.
We’ll build on these, but you don’t need advanced knowledge.
A terminal or command prompt open
All the commands in this section will be run from your terminal.
Port 3306 available on your machine
MySQL uses port 3306. If another MySQL server is already running locally, you may need to stop it.
At least a few hundred MB of free disk space
The MySQL image and its data files need some space to run smoothly.

Once you have these ready, you can safely move on to the first step.

Step 1: Pull the MySQL image

Open your terminal or command prompt. This can be:

Terminal on macOS or Linux
Command Prompt or PowerShell on Windows
Or the built-in terminal inside tools like VS Code

Make sure Docker Desktop is running in the background.
You can check by running:

docker --version

If Docker responds with a version number, you’re good to continue.

Now, let’s download the official MySQL image from Docker Hub. This image contains everything needed to run a MySQL server inside a container.

Run:

docker pull mysql:latest

The first time you pull it, Docker will download all the layers it needs.

If you already have it, Docker will simply check for updates and use the cached version.

When this completes, you now have the MySQL image available locally and ready to run in a container.

Step 2: Create a named Docker volume

Before we run MySQL, we need a place for it to store its data. Instead of letting it write into the container’s temporary filesystem, we’ll give it a named Docker volume. This ensures the data stays safe even if the container is deleted.

In your terminal, run:

docker volume create mysql_data

This creates a volume named mysql_data.

You won’t see a folder appear on your machine, because Docker manages the volume internally. The important thing is that the volume now exists, and we can attach it to the MySQL container in the next step.

Why create the volume ahead of time instead of letting Docker create it automatically?

Because doing it explicitly helps you see the whole flow: you create the storage location, and MySQL simply uses it.

This makes the idea of persistent storage much clearer when you start working with real databases and production workloads.

You can confirm that the volume was created by running:

docker volume ls

Look for mysql_data in the list, if it’s there, you’re ready to move on.

Step 3: Run a MySQL container using the volume

Now that the volume is ready, the next step is to run a MySQL container and attach that volume to MySQL’s data directory.

This is the point where the container and the volume start working together.

In your terminal, run:

docker run -d \
  --name mysql_container \
  -e MYSQL_ROOT_PASSWORD=my-secret-password \
  -v mysql_data:/var/lib/mysql \
  -p 3306:3306 \
  mysql:latest

This confirms the container has started successfully, seeing a long container ID means it’s running.

This single command does a lot, so here’s what each part means:

--name mysql_container
Assigns a readable name to the container so it’s easy to start, stop, or access later.
-e MYSQL_ROOT_PASSWORD=my-secret-password
Creates the root password for MySQL inside the container.
(This password will be needed later when connecting to the database.)
-v mysql_data:/var/lib/mysql
Mounts the volume created in the previous step into MySQL’s internal data directory.
This is what makes the data persist outside the container.
-p 3306:3306
Exposes MySQL on your machine so you can connect using tools or clients.

Once this container is running, every table, row, and record MySQL writes from now on will live inside the mysql_data volume instead of the container’s temporary filesystem.

That means even if we delete the container, the data remains.

If you’d like, you can run the next command to confirm that the container is up and running:

docker ps

If mysql_container appears in the list, you’re ready for Step 4.

Step 4: Connect to MySQL inside the container

Now that MySQL is running inside Docker, let’s connect to it using the MySQL CLI built into the container.

Run this command from your terminal:

docker exec -it mysql_container mysql -u root -p

This opens an interactive MySQL session inside the container.

Right after running the command, MySQL asks for the root password you set earlier. You should see a password prompt like this:

Enter the password and press Enter.

If authentication works, you’ll be dropped into the MySQL shell and shown the MySQL welcome banner. A successful login looks like this:

Now you’re inside MySQL and can run SQL commands normally. Let’s create some data so we can verify persistence later. Run the following inside the MySQL prompt:

CREATE DATABASE testdb;
USE testdb;
CREATE TABLE users (id INT PRIMARY KEY, name VARCHAR(50));
INSERT INTO users VALUES (1, 'Alice');
SELECT * FROM users;

The final SELECT should return the row you inserted. Here’s what the output looks like when the row is returned:

This confirms two things:

MySQL is running correctly inside your container.
The database files are being written into the mysql_data volume.

When you’re done, exit the MySQL shell:

exit;

The container will continue running in the background.

Step 5: Stop and remove the container

Now that we’ve added data to MySQL, let’s demonstrate what happens when the container itself is removed.

The goal here is to show that even if the container is removed, the data stored in the volume remains intact.

From your terminal, run:

docker stop mysql_container
docker rm mysql_container

Here’s what each command does:

docker stop shuts the container down safely.
docker rm deletes the container entirely.

When you run those two commands, you should see output similar to this:

Even though the container is now gone, your mysql_data volume still exists (remains untouched), and it still contains every database file MySQL wrote earlier.

This sets us up perfectly for the next step, where we’ll start a new container and confirm that the data is still there.

Step 6: Start a new container using the same volume

Now that the previous container has been removed, let’s start a brand-new MySQL container and attach the same volume (mysql_data).

This is where you’ll see Docker volumes doing exactly what they’re designed for.

Run the container:

docker run -d \
  --name mysql_container \
  -e MYSQL_ROOT_PASSWORD=my-secret-password \
  -v mysql_data:/var/lib/mysql \
  -p 3306:3306 \
  mysql:latest

Here's what it looks like when the container starts successfully:

This is the same command you ran earlier. The container is new, but the volume still contains all of MySQL’s data files from before.

Connect back into MySQL

Now that the container is running again, open a MySQL shell inside it:

docker exec -it mysql_container mysql -u root -p

Docker will prompt you for the root password:

Enter the same password you set earlier.

Verify that your data is still there

Once inside the MySQL shell, run the following SQL commands:

SHOW DATABASES;
USE testdb;
SELECT * FROM users;

You should now see the same database and table you created earlier, including the “Alice” row:

Your new container didn’t have to recreate anything. It simply reused the data stored in the volume. This is the entire point of Docker volumes: containers may be temporary, but your data is not.

Step 7: Inspect the volume (optional)

If you want to see where Docker is storing your MySQL data on your machine, you can inspect the volume directly:

docker volume inspect mysql_data

This command prints the full details of the mysql_data volume, including the path on your system where Docker keeps the files.

Pay attention to the "Mountpoint" field in the output. That path is where Docker stores all of MySQL’s internal data files, and it remains available even after containers are removed. This confirms that the volume persists independently of any container.

So what you just learned (and why it’s useful)

In this part of the series, I walked you through how Docker volumes keep your data safe even when containers are stopped or removed.

You went through the process step by step:

You created a named volume for MySQL’s data.
You ran a MySQL container and attached that volume to its data directory.
You created a database, table, and record inside MySQL.
You removed the container and started a new one using the same volume.
You confirmed that the data was still there.

By doing this, you now know how to:

Keep important data outside a container’s temporary filesystem.
Reuse the same data across multiple containers.
Inspect a volume to see where Docker stores it.
Prevent data loss when containers are rebuilt during development.

This understanding becomes very practical once you start working with databases, background workers, message queues, or any service that needs to persist data.

In the next part of the series, I’ll show you how to manage everything with Docker Compose, so you can start multiple containers with a single command.

Docker networking: How to connect containers in a full-stack project

Deborah Emeni — Tue, 04 Nov 2025 09:10:30 +0000

What you'll learn

By the end of this section, you'll:

Understand how Docker networking works and why it's important for multi-container apps.
Learn the difference between bridge, host, and overlay networks in Docker.
Know how to expose ports using -p so services can communicate across containers or expose endpoints to your host.
See how to connect multiple containers so they can communicate internally.
Complete a hands-on project where a React app communicates with a Node.js backend over a shared custom bridge network in Docker.

We'll start by understanding why Docker networking is important.

Why Docker networking is important in multi-container applications

When you're working on a project, it's rarely a single container doing all the work. You'll usually have a:

backend API
frontend application
database
sometimes a message broker

... with all running as separate containers.

The key is that these containers need to communicate with each other over a network.

For example, if your frontend can't reach the backend, or your API can't access the database, the entire application breaks. That's where Docker networking comes in.

It enables containers to communicate reliably and securely, without exposing everything to the internet.

Okay, let me give you a common scenario.

Let's say you're building a React application that fetches data from a Node.js backend. During development, you might call the backend using "http://localhost:4000", right?

That works when both apps run directly on your local machine. But once they run in separate containers, localhost no longer refers to the same environment. The React container’s localhost is not the same as the backend’s.

Now you need a way for those containers to discover and communicate with each other.

So, what's the solution?

Docker solves this by creating a virtual network and allowing containers to discover each other by name, like an internal DNS system.

So if you name your backend container "backend", your frontend can make requests to "http://backend:4000".

Without any need for IPs or manual linking, it just works as long as both containers are on the same Docker network.

In this project, you'll see how this works in practice. You’ll:

Create a shared custom network
Run both containers on it
Configure the frontend to communicate with the backend using its container name

This workflow is foundational for larger containerized systems and directly applies to more advanced tooling like Docker Compose or Kubernetes.

Since you now have a clear understanding of why containers need to communicate. In the next section, you'll learn how Docker makes that communication possible behind the scenes.

How Docker networking works

Now that you understand why containers need to communicate, it’s important to see how Docker enables that communication.

When Docker is installed, it creates a default network called bridge.

This bridge is a virtual network that Docker uses to connect containers on the same host. If you run containers without explicitly assigning them to a custom network, Docker attaches them to this default bridge network.

Every container connected to this bridge gets its own internal IP address. But more importantly, when containers are connected to the same network, Docker allows them to resolve each other by name.

This means instead of using localhost or an IP address, a container can communicate with another container simply by using its name.

For example, if a container is started with--name backend, any other container on the same network can reach it at: "http://backend"

This feature is what makes internal container communication seamless. You don’t need to hardcode IPs or expose every service to the outside world.

The 3 types of Docker networks

Now, let's break down the three main network types to know when working with Docker:

1. Bridge network (default for single-host setups)

This is the most commonly used network for local development. When multiple containers are attached to the same bridge network, they can communicate with each other using their container names.

In this project, a custom bridge network will be used. Creating a named network gives more control and ensures both the frontend and backend containers are communicating within a shared, isolated environment.

2. Host network

This mode removes network isolation between the container and the host. The container shares the host’s network stack. It is typically used when maximum network performance is needed or when the container must bind directly to host ports.

We won't use this in our project, but it's important to be aware of its use cases and limitations.

3. Overlay network

This one is for multi-host setups, like when you're running Docker Swarm or Kubernetes. It allows containers running on different physical machines to communicate over a secure virtual network.

It's not needed for single-machine setups, but essential when deploying distributed systems in production.

So, to recap:

For most development environments and projects running on a single host, the bridge network is the most practical choice. It provides container name resolution, clean isolation, and is simple to configure.

This is why the upcoming steps in the project will use a custom bridge network, which ensures that the frontend and backend containers can communicate securely and reliably.

Coming up next, I'll show you how to expose container ports with the -p flag so services running in containers can be accessed from the host machine or other tools. Let's walk through that now.

Exposing container ports with `-p`

Now that you’ve seen how containers can communicate with each other on a Docker network, it’s also important to understand how your host machine (such as your browser, Postman, or terminal) can communicate with those containers.

By default, Docker containers are isolated. Even if a service is running correctly inside a container, it cannot be accessed from outside unless a port is explicitly exposed.

This is where the -p flag comes in.

When starting a container, the -p option maps a port inside the container to a port on the host machine. This allows external tools, like your browser, to access services running inside the container.

For example:

-p 3000:3000

This tells Docker to take port 3000 from inside the container and map it to port 3000 on your host machine.

So if your React app is running inside the container on port 3000, you can open your browser and visit "http://localhost:3000"

If you forget to use the -p flag, the container might still be running and responding internally, but you will not be able to access it from your host environment.

Where you'll find this mapping mostly important is for:

Frontend applications that need to be opened in a browser
APIs that should be tested using Postman, curl, or other tools
Any service that should be exposed to the outside world during development or testing

In our project, we'll expose both the backend (on port 4000) and the frontend (on port 3000) using -p. This will allow us to view the React app in the browser and test the Node.js API externally.

Up next, I’ll walk you through how containers communicate internally without needing to expose their ports to the host at all. This is particularly useful when two services need to communicate entirely within Docker.

How containers communicate with each other internally

Earlier, you learned how to expose a container’s port to your host machine using the -p flag. That setup is useful when you want to access a service from your browser, testing tools like Postman, or terminal utilities like curl.

But what happens when two containers need to communicate internally, without routing through your host machine?

For example, say your React frontend container needs to fetch data from a Node.js backend container. Both are running inside Docker. In this case, you do not need to expose any ports with -p to enable communication between them.

This works because Docker automatically sets up internal networking for containers that are on the same network. Docker provides an internal DNS system that allows containers to resolve each other by name.

If your backend container is named backend, your React app can send a request like this:

http://backend:4000

There’s no need to use IP addresses or expose backend ports to the host. The container name acts as the hostname, and Docker handles the rest.

To enable this setup, both containers must be attached to the same Docker network. You can do this by creating a custom bridge network and passing the --network flag when running each container.

In this project, that’s exactly what we’ll do:

Create a custom bridge network
Connect both containers to it
Configure the React app to communicate with the backend using the container name (backend) and the correct port (4000)

This internal communication model is how multi-container systems typically work. Services communicate over a private network without needing to expose every service to the outside.

Alright, now let’s move on to the project where we’ll put all of this into practice. You’re going to use a React frontend + Node.js backend running in separate containers, communicating over Docker’s internal network.

Let’s go.

Project: Connect a React frontend container to a Node.js backend container

You’ve now seen the theory behind Docker networking. Let’s put that knowledge into practice with a hands-on project.

In this section, you’ll build/clone a two-service application using Docker: a React frontend that fetches data from a Node.js backend. Both services will run in separate containers, and you’ll configure them to communicate over a shared Docker network.

Here’s what you’ll do:

Containerize each application with a custom Dockerfile
- The React app will be built and served using Nginx
- The backend will run on Node.js 22
Create a custom Docker bridge network
Configure the frontend to communicate with the backend using the backend’s container name, not localhost
Verify that everything works in your browser and from inside the containers

If you don’t already have a project set up, you can clone these two minimal demo repositories to follow along:

React frontend: https://github.com/d-emeni/react-demo
Node.js backend: https://github.com/d-emeni/node-api-demo

Let’s get started.

Step 1: Understand the project structure

Let’s start by walking through what each part of the application does and how they interact once containerized.

The React app (frontend) runs on port 3000 in development. It fetches a list of users from a backend API.
The Node.js backend listens on port 4000, serving a JSON response at the /api/users endpoint.

In a typical local development setup, the frontend would send requests like this:

http://localhost:4000/api/users

That works because both the frontend and backend are running directly on your machine.

However, once you move both apps into containers, that changes. Each container has its own isolated environment, including its own version of "localhost". So if the frontend container tries to send a request to "localhost:4000", it's actually trying to call itself, not the backend.

To solve this, we’ll:

Connect both containers to a shared Docker network
Update the frontend to communicate with the backend using the container name, like "http://backend:4000"
And in production (when the frontend is served via Nginx), we’ll configure it to call the backend using a relative path like /api, which Nginx will proxy to the backend container

This approach allows the two services to communicate reliably within Docker, without hardcoding IP addresses or exposing unnecessary ports.

Step 2: Add Dockerfiles to both apps

To run your applications inside Docker containers, you need to define how each one should be built. That’s what a Dockerfile does, it’s a step-by-step recipe that tells Docker how to package your app into a runnable image.

Dockerfile for the React frontend

In the root of your React project (react-demo/), create a file called Dockerfile with the following content:

# Stage 1: Build the React app
FROM node:22-alpine AS builder
WORKDIR /app
COPY . .
RUN npm install && npm run build

# Stage 2: Serve the app with Nginx
FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Let’s break this down:

Stage 1 (builder): Uses Node.js 22 to install dependencies and run the Vite build, which outputs the static files into the dist/ folder.
Stage 2 (Nginx): Copies the build output into the default Nginx web root and starts the Nginx server to serve the static files.

This multi-stage Dockerfile keeps your final image small and production-optimized. You're only shipping the compiled frontend and not the entire Node environment.

Make sure to also include an nginx.conf file in your project root. This ensures that API requests like /api/users are correctly forwarded to the backend container. You’ll add that later in the tutorial.

Dockerfile for the Node.js backend

In the root of your backend project (node-api-demo/), create a file named Dockerfile with the following content:

# Use Node.js 22 Alpine image
FROM node:22-alpine

# Set the working directory
WORKDIR /app

# Copy the backend code
COPY . .

# Install dependencies
RUN npm install

# Start the server
CMD ["node", "server.js"]

# Expose the backend port
EXPOSE 4000

This Dockerfile defines everything Docker needs to run your backend service:

Installs dependencies
Runs your server.js file using Node.js
Exposes port 4000 for incoming API requests

If you’d like a more detailed walkthrough of Dockerizing a Node.js backend, check out my step-by-step guide.

Step 3: Build Docker images

Now that both applications have Dockerfiles, it’s time to package them into Docker images. These images will serve as the blueprints for running your containers.

Open your terminal and run the following commands from the root of each project:

Make sure your Docker daemon is running before you begin. If you're not sure, follow this setup guide

# Inside react-demo/
docker build -t react-app .

# Inside node-api-demo/
docker build -t backend-api .

Let’s break that down:

The -t flag assigns a name (or “tag”) to your image. In this case, we're naming them react-app and backend-api.
The . at the end tells Docker to use the current directory as the build context, where it will look for the Dockerfile and app code.

Once these build steps are complete, you’ll have two ready-to-run Docker images:

react-app — a production-ready build of your React frontend, served with Nginx.
backend-api — your Node.js server listening on port 4000.

You’ll see output logs during the build process. Here's what the end of the build typically looks like for each app:

React frontend image build:

Node.js backend image build:

If you see something like this, it means your images were built successfully and are now ready to be run in containers.

Next up, we’ll create a Docker network to allow both containers to communicate.

Step 4: Create a custom Docker network

For the frontend and backend containers to communicate by name, they must be connected to the same Docker network.

Open your terminal (you can run this from any directory) and create a custom bridge network:

docker network create react-backend-net

This command sets up a new bridge network named react-backend-net. If successful, Docker will return a long alphanumeric string that represents the network ID:

You won’t need to interact with this ID directly. What matters is the name of the network (react-backend-net), because that’s what you’ll reference when running containers.

Once both containers are connected to this network, Docker will enable them to resolve each other by container name. For example, the frontend can reach the backend simply using "http://backend:4000" without any IP addresses or exposed ports.

If you get an error saying the network already exists, that means it was previously created. You can safely skip this step and continue with the next one.

Next, we’ll run the backend container and connect it to this custom network.

Step 5: Run the backend container

Start by running the Node.js backend as a container. This will ensure the API is running and ready to handle requests before we launch the React app.

Before running the command below, ensure the following:

You’re not already running the backend server locally on port 4000.
Port 4000 is free (not being used by another process).
The Docker daemon is running properly on your system.

Now, open your terminal and run:

docker run -d \
  --name backend \
  --network react-backend-net \
  -p 4000:4000 \
  backend-api

This command does the following:

--name backend assigns the container a name. Other containers on the same network (like the frontend) can refer to it using this name.
--network react-backend-net attaches the container to the custom Docker network we created in Step 4.
-p 4000:4000 maps port 4000 inside the container to port 4000 on your host, so you can access the API via "http://localhost:4000".

If everything starts correctly, Docker will output a long container ID like this:

This confirms the backend container is running in the background.

Just in case you run into any errors & how to resolve them…

If you run the command and Docker throws an error, don’t worry. These are the two most common ones you might see during this step, and how to fix them quickly:

(1). Port already in use

Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:4000...

This means something (like your local server) is already using port 4000.

Solution:

Stop the process using the port (e.g., your locally running Node server), or
Change the host port in the Docker command:

-p 4001:4000

This maps port 4001 on your machine to port 4000 inside the container.

(2). Container name already in use

Error response from daemon: Conflict. The container name "/backend" is already in use...

Docker doesn’t allow duplicate container names. You can fix this in one of two ways:

Option A: Remove the existing container:

docker rm -f backend

Option B: Run the new container with a different name:

--name backend-v2

Once your backend container is running successfully, you’re ready to launch the frontend container and configure it to communicate with the backend through the Docker network.

Step 6: Configure the React frontend to communicate with the backend container

Before running the React container, we need to update the frontend code so it knows how to communicate with the backend service inside Docker.

During local development, you may have written a request like this:

fetch("http://localhost:4000/api/users")

That works when both apps run on your machine. However, once they’re in containers, this URL will no longer work, because each container has its own isolated environment. Inside the React container, localhost refers to itself, not the backend.

To resolve this, you have two important steps:

(1). Use a relative path in api.js

Update the API request to use a relative path instead of a hardcoded URL:

fetch("/api/users")

This ensures the React app can stay agnostic of the backend's full URL, letting us handle routing through Docker or Nginx.

In your src/services/api.js file, your updated code might look like:

const API_BASE_URL = import.meta.env.VITE_API_BASE_URL || "";

export async function fetchUsers() {
  const response = await fetch(`${API_BASE_URL}/api/users`);
  if (!response.ok) {
    throw new Error(`Failed to fetch users: ${response.status}`);
  }
  return response.json();
}

(2). Set VITE_API_BASE_URLin the .env file

To make this work inside Docker, you should set the environment variable in your React app’s .env file like this:

VITE_API_BASE_URL=

This means: use a relative path like /api (which Nginx will proxy internally to the backend container). In our container setup, the Nginx configuration ensures that requests to /api are forwarded to the backend.

Inside the container, this relies on Docker’s internal networking and DNS resolution. Since the backend container is named backend, Nginx knows how to forward requests to "http://backend:4000".

Once these updates are made:

Your React app will send API requests to /api/users
Nginx inside the container will forward them to "http://backend:4000/api/users"
The backend will respond, and the data will be rendered in your React UI

This setup keeps your frontend clean, avoids hardcoding environment-specific URLs, and works seamlessly inside Docker.

Next, we’ll run the React container and connect it to the backend via the shared Docker network.

Step 7: Run the React container

With your Docker image for the frontend built and the backend container already running, you're ready to launch the React app inside a container.

Run the following command:

docker run -d \
  --name frontend \
  --network react-backend-net \
  -p 3000:80 \
  react-app

Let’s break down what this does:

--name frontend assigns the container a name for internal communication and easier reference
--network react-backend-net connects the container to the same Docker network as the backend, enabling internal communication
-p 3000:80 maps port 80 inside the container (used by Nginx to serve the app) to port 3000 on your machine, so you can access it at "http://localhost:3000"

Once the container is running, visit:

http://localhost:3000

You should see your React app in the browser. If everything is configured correctly, it will fetch the user data from the backend container and display the list.

If you run into any issues, don’t worry. We'll cover debugging in the next step.

Step 8: Debug or verify container communication

If your React app displays an error like "Failed to fetch", it means the frontend was unable to reach the backend API. Here are several ways to diagnose and resolve the issue:

1. Check the backend logs

Run the following command to inspect the backend container:

docker logs backend

This will show whether the request from the frontend reached the backend, and whether the server responded successfully or encountered an error (e.g., route not found or internal server error).

2. Use browser developer tools (DevTools)

In your browser:

Open the Network tab (inside DevTools)
Refresh the page
Look for the request to:

   http://backend:4000/api/users

Inspect the status code, response, and any error message in the preview or console.

This helps you determine if the request was blocked, returned a 404/500, or failed due to CORS or DNS issues.

3. Ping the backend container from inside the frontend container

You can enter the frontend container's shell like this:

docker exec -it frontend sh

Then run:

ping backend

If the containers are correctly connected to the same network, you'll see successful ping responses like:

PING backend (172.18.0.2): 56 data bytes
64 bytes from 172.18.0.2: icmp_seq=0 ttl=64 time=0.102 ms

This confirms that the frontend can resolve and reach the backend container by name.

(Optional) Test the backend API from within the frontend container

Still inside the frontend shell, try:

wget http://backend:4000/api/users

or if curl is available:

curl http://backend:4000/api/users

This helps you verify that a valid HTTP response is returned from the backend endpoint.

Once you confirm the containers can communicate, but the React app still fails to fetch, check for:

typos in the API URL
missing environment variables
CORS issues in your backend (if applicable)

Step 9: Clean up

Before we wrap up this project, let’s clean up everything we created: the containers and the custom network.

This helps avoid conflicts when you’re working on future Docker projects, and keeps your environment tidy.

Follow the steps below to remove both containers and the network.

1. Remove the frontend container

We’ll start by stopping and removing the frontend container.

Run this from any terminal:

docker rm -f frontend

If it works correctly, Docker will stop and remove the container, and you’ll see something like this:

2. Remove the backend container

Next, remove the backend container:

docker rm -f backend

This will stop and delete the backend container. Here’s what it looks like when it works:

3. Remove the custom Docker network

Now let’s remove the network that connected the containers:

docker network rm react-backend-net

If successful, Docker will simply return the name of the network:

This confirms the network has been deleted.

4. Confirm everything is gone

To double-check that no containers are still running, run:

docker ps

You should see no active containers, just an empty table like this:

So what you just built (and why it’s useful)

You’ve just completed a hands-on Docker networking project using a frontend and backend app.

Let’s walk through what you did:

You containerized two apps: a React frontend and a Node.js backend
You created a custom Docker network so the containers could communicate
You updated the frontend to connect to the backend by container name, not localhost
You verified everything worked, using your browser, terminal, and Docker commands

By doing this step-by-step, you've learned how to:

Run separate services inside containers
Connect them on the same Docker network
Avoid common issues developers face when containers can’t reach each other

In the next tutorial, I’ll show you how to simplify this setup using a docker-compose.yml file, so you can launch everything with one command. Follow me on dev.to to get notified.

How to dockerize a node.js application with a custom dockerfile

Deborah Emeni — Thu, 25 Sep 2025 07:58:48 +0000

In this guide, you'll learn how to Dockerize a Node.js application.

If you are a total beginner, start with this: Getting Started with Docker (Understanding virtualization & containers in the simplest way)

Containers let you package your app with everything it needs. But how do you build one from scratch? In this guide, we'll dockerize a Node.js app using a custom Dockerfile.

We'll build a Node.js app that fetches live data from GitHub and serves it via an Express server. Then we'll walk through every line of the Dockerfile that packages it.

Before you begin, make sure you have Docker installed and properly set up. You can follow the steps in this article.

Why should I Dockerize my Node.js app?

Dockerizing your app lets you run it consistently across development, testing, and production. You won’t have to worry about differences across machines. It works the same wherever it runs.

Let's look at a few reasons why it’s a good idea:

Environment consistency: When you package Node.js, your code, and all dependencies into one container, everything runs in a controlled setup. This reduces errors caused by mismatched versions or missing packages.
Simplified deployment: You can move your app from your local machine to a server or cloud provider using the same image. There’s no need to set up the environment from scratch each time.
Team collaboration: Your teammates can run the exact same version of the app with one command. They don’t need to install Node.js, configure ports, or manually set up dependencies.
Better isolation: Containers run independently from your main system. This helps avoid conflicts with other apps or services running on your machine.
Portability: Docker runs your app the same way across all operating systems, like macOS, Windows, and Linux. You don’t need to adjust for each platform.

Most importantly, Docker is useful when you're preparing for CI/CD pipelines, working with cloud platforms, or building apps with a team. It goes beyond packaging your app to also creating a reliable and predictable way to run it anywhere.

What is a Dockerfile?

A Dockerfile is a plain text file that defines how your application will be built and run inside a Docker image. It includes everything the image needs: the base image to start from, any files to copy in, commands to run during setup, and how to start the app when the container runs.

You can think of it as a build script; each instruction adds a new layer that eventually forms the final image.

The Node.js app we'll be dockerizing

We'll containerize a Node.js app built with Express. It fetches data from GitHub using Axios and serves it on the homepage. You’ll see how everything works step by step, from setting up the project to running it locally before we dockerize it.

Start by creating a folder named stars_generator and adding two files inside:

package.json
server.js

1. The "package.json" file

This file defines your app’s metadata, including the dependencies it needs and how to start it. We’ll use express and axios, and tell Node.js to run server.js when the app starts.

Create package.json with the following content:

{
  "name": "docker-node-advanced",
  "version": "1.0.0",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "express": "^4.18.2",
    "axios": "^1.4.0"
  }
}

2. The "server.js" file

This is the main entry point of the app. It uses Express to serve a homepage, fetches GitHub repo data using Axios, and displays the star count in your browser.

Add this code to server.js:

const express = require('express');
const axios = require('axios');

const app = express();
const PORT = process.env.PORT || 3000;

app.get('/', async (req, res) => {
  try {
    const response = await axios.get('https://api.github.com/repos/nodejs/node');
    res.send(`<h1>Node.js GitHub Stars</h1><p>This repo has ${response.data.stargazers_count} ⭐️ stars.</p>`);
  } catch (error) {
    res.status(500).send('Failed to fetch data from GitHub');
  }
});

app.listen(PORT, () => {
  console.log(`Server running on http://localhost:${PORT}`);
});

What each part of this file does:

const express = require('express'); – loads the Express framework to handle HTTP requests.
const axios = require('axios'); – loads Axios to make API calls.
const app = express(); – creates a new Express app.
const PORT = process.env.PORT || 3000; – sets the port from your environment or defaults to 3000.
app.get('/', async (req, res) => { ... }) – defines a route for the root URL (/). When visited, it fetches data from GitHub.
const response = await axios.get(...) – sends a GET request to GitHub’s API for the Node.js repo.
res.send(...) – sends back HTML with the current star count.
catch (...) – catches any errors and returns a 500 status code.
app.listen(...) – starts the server and logs a message with the local URL.

3. Run the application to test it

Before we containerize anything, test the app locally to make sure everything works as expected.

In your terminal:

npm install
node server.js

Then open your browser and go to:

http://localhost:3000

You should see something like this:

If the app crashes or doesn’t respond correctly, fix that first before moving forward. It’s better to start with a working app before introducing Docker.

In the next step, we'll containerize the app.

Containerizing the application

We’re now going to containerize the Node.js app so that it can run consistently anywhere, on your machine, on someone else’s, or on a production server, without needing to worry about system differences. The goal is to package the app and its environment into a reusable Docker image.

Now let's containerize the app.

Step 1: Create a Dockerfile

To containerize the app, you’ll write a Dockerfile that gives Docker step-by-step instructions for building an image.

Create a file named Dockerfile in the root of your project folder (same level as server.js and package.json). Your folder structure should now look like this:

stars_generator/
├── Dockerfile
├── package.json
└── server.js

You can create it by running:

touch Dockerfile

Then, add this content to the Dockerfile:

# Start with Node.js 22 Alpine base image
FROM node:22-alpine

# Set the working directory in the container
WORKDIR /usr/src/app

# Copy package.json and install dependencies
COPY package.json ./
RUN npm install

# Copy the rest of the application code
COPY . .

# Expose the port your app runs on
EXPOSE 3000

# Start the app
CMD ["npm", "start"]

Let’s break it down line by line:

FROM node:22-alpine

This tells Docker to start from the official Node.js v22 Alpine image:

Alpine keeps things lightweight (around 40MB).
Node 22 is a stable version with long-term support (LTS).
The image includes both Node.js and npm.

WORKDIR /usr/src/app

This sets the working directory to /usr/src/app inside the container. All following commands will be executed from this path. It helps keep things clean and consistent.

COPY package.json ./
RUN npm install

The COPY command copies your local package.json into the container.
Then RUN npm install installs your dependencies inside the container.

We do this before copying the rest of the code so Docker can cache this step. That way, if your code changes but your dependencies don’t, Docker won’t re-run npm install.

COPY . .

This copies all the remaining files, including your server.js, into the container.

EXPOSE 3000

This includes files like app.js, routes/, controllers/, etc.

# Expose the port your app runs on
EXPOSE 3000

It informs Docker (and anyone running this image) that the app listens on port 3000.

This doesn't publish the port; it's just documentation within the Docker image. You'll still need to map it using -p or --publish when running the container.

# Start the app
CMD ["npm", "start"]

It defines the default command Docker will run when a container is created from this image.

This runs npm start, which typically starts your Node.js server.
Make sure you've defined a start script in your package.json, like this:

"scripts": {
  "start": "node server.js"
}

Step 2: Add a ".dockerignore" file

This step prevents unnecessary files like node_modules, .env, .git, logs) from being copied into the image. And it should come before building the image for smaller and faster builds including fewer security risks and cleaner containers.

Create a .dockerignore file in your project root with the following contents:

node_modules
npm-debug.log
Dockerfile
.dockerignore
.git
*.md
.env

See the reasons why we added those to the .dockerignore file:

node_modules: You'll run npm install inside the container instead, and so you don't want to copy bulky local dependencies.
npm-debug.log: Log files aren't needed inside the image.
Dockerfile & .dockerignore: These aren't usually needed inside the image.
.git: You don't need Git history or config inside a Docker image.
*.md: Docs aren't needed unless they're part of the app logic.
.env: For security reasons, secrets and environment variables should be passed in at runtime.

Step 3: Build and run the container

Now that the Dockerfile and .dockerignore are ready, let's move on to the following:

Build the Docker image

We're building the Docker image to package our Node.js app, its dependencies, and environment into a single, reusable unit, so that it can run consistently anywhere, regardless of the host system.

In your terminal, from the root of your project, run:

docker build -t <name_of_your_app> .

Replace the placeholder "" with the name of the app, like this:

docker build -t stars_generator .

See what each part of this command means:

-t stars_generator: This tags the image with the name stars_generator, so you can later refer to it easily when running or managing containers (e.g., docker run stars_generator). It's like giving your image a label or shortcut.
.: This tells Docker to look in the current directory (where your terminal is open and where the Dockerfile is located) for everything it needs to build the image, including the Dockerfile and source code.

If the command runs successfully, Docker processes the Dockerfile line by line, creating an image that can later be run as a container.

Make sure your docker daemon is running, otherwise you'll get this error when you run the command:

This error means Docker is installed, but the Docker daemon (background service) isn't running, so your system can't build images or manage containers.

So, to avoid or tackle this error, and make sure your docker daemon is running, follow the steps in this article

Or you can simply do the following to fix it:

1) If you're on macOS and using Docker Desktop:

Open Docker Desktop and wait for it to fully start. You'll usually see the Docker icon in your menu bar with a green dot once it's ready.
Then rerun the command.

2) If you're on linux:

You may need to run:

 sudo systemctl start docker

And possibly:

 sudo usermod -aG docker $USER

3) If Docker Desktop isn't installed, install it from here

When you run the command, you should see the following to show that the build was successful:

So this shows that Docker built your custom image from the Dockerfile without any errors.

What the build output means, line by line

Let's walk through the major steps you see in the terminal

[internal] load build definition from Dockerfile: Docker is reading your Dockerfile and preparing the build context.
transferring dockerfile: 423B: It reads the Dockerfile (423 bytes in size).
load metadata for docker.io/library/node:22-alpine: It pulls the Node.js v22 Alpine base image from Docker Hub.
extracting sha256:...: Docker unpacks (extracts) layers of the Node.js base image.
[2/5] WORKDIR /usr/src/app: Sets the working directory in the container to /usr/src/app.
[3/5] COPY package.json ./: Copies only package.json so Docker can cache dependencies.
[4/5] RUN npm install: Installs the dependencies listed in your package.json.
[5/5] COPY . .: Copies the rest of your project files (like server.js).
exporting to image: Your custom image is finalized and saved with the tag stars_generator.

So, the summary is you now have a reusable Docker image called stars_generator that contains your Node.js app and its dependencies.

Run the container

Now that the image is built, your next step is to run the container. This step is important because it allows you to start and test your Node.js app inside an isolated Docker environment using the image you just built.

So, run this command:

docker run -p 3000:3000 stars_generator

What this command means:

docker run: Tells Docker to start a new container.
-p 3000:3000: Maps port 3000 on your machine (host) to port 3000 inside the container, so you can access your app at http://localhost:3000.
stars_generator: The name of the image you built earlier.

In simple terms: you're launching your app in Docker and making it available at localhost:3000 on your browser.

This is what you should see when you run the command:

What this output means is that your Docker container successfully started and ran the Node.js app inside it. Specifically:

> docker-node-advanced@1.0.0 start
> node server.js

This shows that the container is executing the start script from your package.json, which runs server.js.

Then this line:

Server running on http://localhost:3000

...means your app is now live and listening on port 3000. Since you mapped that port to your local machine with -p 3000:3000, you can open your browser and visit:

http://localhost:3000

to test the app running inside Docker.

But wait, we're not done because you might be asking: "we could already run the node.js app locally so what's the essence of these steps?" you'll find out in the next section.

Why we containerized the Node.js app

You might wonder: we could already run the app with node server.js, so why go through the trouble of Dockerizing it?

Let's see the reasons why:

1. Consistent environment everywhere

Docker lets you package your app with its runtime, dependencies, and system libraries into a single image. That means:

It runs exactly the same on any system like Windows, macOS, or Linux.

No more “works on my machine” issues.

2. Easy sharing and deployment

Once you have a container image:

You can push it to Docker Hub (a public or private container registry).
Anyone can pull and run it without needing to clone your code or install Node.js.

This is especially useful in:

Teams with different dev machines
CI/CD pipelines
Deploying to production (on cloud or Kubernetes)

3. Safe isolation

The app runs inside a container, isolated from the rest of your system. No conflicts with global Node versions or ports. You can stop and delete containers without affecting anything on your machine.

How to push your Docker image to Docker Hub

Since this is a small project and for personal learning, this step is optional. However, it's important to do it at least once to understand the workflow. See the reasons why:

You'll see how teams share and deploy containerized apps.
You'll be able to pull your image from any machine without copying your code.
You'll get familiar with real-world Docker usage which is very useful for DevOps, backend or platform roles.

If you're using the free tier on Docker Hub, just be mindful not to push too many unused images. But for learning? It's definitely not a waste.

Now see the steps to push your Docker image to Docker Hub

Step 1: Create an account on Docker Hub

If you haven't already, go to the Docker Hub website, sign up and create a Docker ID (which is your username).

Step 2: Tag the image with your Docker Hub username

Let’s say your Docker Hub username is your_dockerhub_username. You’ll tag the image like this:

docker tag stars_generator <your_dockerhub_username>/stars_generator

This names your image so Docker knows where to push it.

Make sure your terminal is in the project directory (where your Dockerfile is).

Then tag the image for Docker Hub (replace with your actual Docker Hub username)

This command doesn’t depend on any specific files in the directory, it works because Docker knows the image you built (by name) and assigns it a new tag that includes your Docker Hub namespace.

Why tagging is important:
Docker needs to know where to push the image. stars_generator is your local image name, but /stars_generator tells Docker to push it to your personal namespace on Docker Hub.

Step 3: Log in to Docker from your terminal

Before pushing the image to Docker Hub, you need to authenticate your Docker CLI with your Docker Hub account. Run:

docker login

You’ll be prompted to enter your Docker Hub username and password.

This step is necessary because Docker won’t let you push images to your account unless you’re logged in, it needs to confirm that you have permission to upload to the namespace you tagged the image with (e.g., /stars_generator).

Note: If you're using Docker Desktop, you may already be signed in there, but it's still best to run docker login from the terminal to be sure your CLI is authenticated.

Once you run the docker login command in your terminal, Docker may prompt you to authenticate using a web-based login flow, especially if you're using Docker Desktop or haven’t logged in recently.

You’ll see something like this:

What this means:

Docker generated a one-time device confirmation code (like AKBT-IEBG) that links your terminal session to your Docker account.
It also gives you a link: https://login.docker.com/activate
When you press ENTER, your browser will open that page.
Once there, log in with your Docker Hub credentials and enter the code shown in your terminal.

This links your terminal session with your Docker Hub account, allowing you to push containers.

Once logged in successfully, you should see this confirmation:

Your terminal session will be authorized to push images under your username until the session expires or you log out. You will see "Login Succeeded" in your terminal:

Step 4: Push the image to Docker Hub

Once you’re logged in, it’s time to upload your Docker image to Docker Hub. You’ll use the docker push command:

docker push <your_dockerhub_username>/stars_generator

This tells Docker to push the image you previously tagged (/stars_generator) to your public Docker Hub repository under your account.

You’ll see Docker upload the image layer by layer. It only pushes the layers that don’t already exist in your Docker Hub account, which saves time.

Once it’s done, your terminal should look something like this:

Now, visit your image directly at:

https://hub.docker.com/repository/docker/your-dockerhub-username/stars_generator/

Replace with your actual Docker Hub username.

You’ll see a page like this:

What you’re looking at:

Tag section: Shows which versions of your image are available (e.g., latest)
Push and pull stats: Tells you how recently the image was pushed and how many times it’s been pulled
Docker command prompt: Gives you the exact docker pull command anyone can use to download the image
General settings: Lets you add a description, set categories, and control collaboration

This makes your image publicly accessible, meaning anyone can now pull it using:

docker pull <your_dockerhub_username>/stars_generator

Why this is important:
Pushing to Docker Hub is useful if you want to:

Share your app with teammates
Use your image in a CI/CD pipeline
Deploy it to a cloud platform like AWS
Archive a version of your containerized app

Step 5: Anyone can now run your app

Now that your image is on Docker Hub, anyone can pull and run it, no setup or dependencies required.

All they need to do is run these two commands:

docker pull <your_dockerhub_username>/stars_generator
docker run -p 3000:3000 <your_dockerhub_username>/stars_generator

Replace with your Docker Hub username.

This will:

Download the image from your Docker Hub repository
Run the container and expose port 3000 on their local machine

Once it’s running, they can open:

http://localhost:3000

…and they’ll see your Node.js app in action - without needing to install Node, npm install, or clone any repo.

So, now your app becomes portable and self-contained. Anyone on any OS can run it the same way either on their laptop, a VM, or a production server.

When should you Dockerize and push your app?

You might have this question in mind:
“Should I finish building and testing my app before Dockerizing and pushing it to Docker Hub?”

In real-world projects, the answer depends on your team’s workflow, deployment goals, and stage of development. Let’s break it down.

1. Finish building and testing first, then Dockerize and push

This is the most common and practical approach in real-world scenarios, especially for production services.

You typically:

Build the core functionality of your app.
Test it locally (unit tests, integration tests, etc.).
Confirm that it works as expected outside of a container.
Then, write a Dockerfile and containerize it.
Test the container (does it build, does it start up, are all env vars respected?).
Only then do you push it to Docker Hub or your private registry.

For Example:
Let's say you're building a payment microservice for an e-commerce system. You'd want to make sure the service can talk to your payment gateway, handle retries, fail gracefully, and return correct responses before packaging it in a container. Once it’s reliable, Dockerizing it ensures it behaves the same across staging and production.

2. Dockerize early during development

In some teams, especially those working on cloud-native platforms (like Kubernetes), you might Dockerize the app early, even while it's still being built. This is useful if:

Your team is standardizing development across machines using Docker containers.
You’re testing deployments continuously on a platform like AWS ECS or GKE.
You're writing infrastructure-as-code (IaC) alongside the app.

For Example:
A team building a GraphQL backend might scaffold the app and immediately create a Dockerfile so every developer can spin it up using docker-compose and work in a consistent environment. They may push early test builds to a registry like GitHub Container Registry, even before the app is "done."

So what’s the best approach?

If your goal is to publish a reliable, working app on Docker Hub, it’s better to:

Build it.
Test it.
Then containerize it and push.

This keeps your registry clean and avoids wasting storage on incomplete or broken containers.

But if you're iterating fast, and container consistency is a priority, Dockerizing early can help, as long as you're comfortable handling container-related issues while still building the app.

Getting Started with Docker - How to install Docker and set it up correctly

Deborah Emeni — Sun, 22 Jun 2025 20:30:55 +0000

Before running any docker commands, you need to install Docker and ensure it’s running properly. You’ll need to do the following:

Install Docker on your system
Verify that Docker is running
Run a test container to confirm everything is set up

Step 1: Install Docker

Go to the official Docker website and download Docker Desktop for your operating system.

Follow the steps here to download Docker or click the “Dowload Docker Desktop” button on the website:

Choose the version that matches your operating system:

Windows: Install Docker Desktop for Windows (Follow the steps here)
Mac: Install Docker Desktop for Mac (Follow the steps here)
Linux: Install Docker Engine manually (Follow the steps here)

Step 2: Verify that Docker is running

After installation, Docker needs to be running before you can use it.

On Windows & Mac, open Docker Desktop and wait for it to say "Docker is running."

On Linux, start Docker manually by running:

sudo systemctl start docker

Then, verify it’s running with:

docker info

If Docker is running, you’ll see detailed information about your system. For example, if you run the command, you’ll see something like this in your terminal:

What you can see in the terminal output for the (docker info command) above means a few things:

It is running Docker version 27.5.1.
The Docker daemon (the background service that runs on your machine and manages Docker containers, images, networks, and volumes) is active and responding (shown by the full server information being available).
The Docker daemon (the background service that runs on your machine and manages Docker containers, images, networks, and volumes) is active and responding (shown by the full server information being available).
Docker reports 0 containers running, which is expected if you haven’t started any yet.

Now, see the screenshot below showing a Docker Desktop that shows “Engine running” in the lower left corner, confirming the Docker engine is active:

Step 3: Run a test container

To confirm everything works, run:

docker run hello-world

This will:

Download the hello-world container from Docker Hub
Run it as a container
Display a message confirming that Docker is set up correctly

Note: Docker Hub is an online repository where you can find and download container images for different software.

A Docker image is a lightweight, standalone package that contains everything needed to run a container, including the OS, system tools, and application dependencies. When you run a container, it is created from an image.

If you see an output saying "Hello from Docker!", then Docker is working.

You’ll see something like this your terminal:

Again what this command does:

When you run docker run hello-world, you’re asking Docker to:

download a small test image called hello-world (if it's not already on your machine)
create and start a container from that image
run a program inside the container that prints a message to confirm everything is working

Let’s go into it in more detail, okay?

Step-by-step breakdown of what happened

1. Docker checked for the image locally

The terminal above said:

Unable to find image 'hello-world:latest' locally

That means Docker looked on your computer for the hello-world image but didn’t find it, so it had to pull it from Docker Hub.

2. Docker pulled the image from Docker Hub

latest: Pulling from library/hello-world

Pull complete

Status: Downloaded newer image for hello-world:latest

This shows that Docker successfully downloaded the latest version of the hello-world image from Docker Hub. As I said earlier, Docker Hub is like a public library for container images.

3. Docker ran the container

Once the image was downloaded, Docker created a container and ran it. The container contains a tiny program that simply prints:

Hello from Docker!

This message shows that your installation appears to be working correctly.

This message means Docker is installed correctly, the Docker daemon is running, and Docker can download, create, and run containers successfully.

4. Docker explains what it just did

The message continues:

1. The Docker client contacted the Docker daemon.

2. The Docker daemon pulled the "hello-world" image from the Docker Hub.

3. The Docker daemon created a new container from that image...

4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.

This gives you insight into how Docker works behind the scenes:

The Docker client (you) sends a command
The Docker daemon (background service) does the work: pulling the image, creating the container, and running the code
The output of the container is sent back to your terminal

5. It gives you a next step

At the bottom, Docker suggests:

To try something more ambitious, you can run an Ubuntu container with:

docker run -it ubuntu bash

You have now successfully installed Docker and ensured it's running correctly. Congrats!

Understanding virtualization & containers in the simplest way

Deborah Emeni — Sun, 22 Jun 2025 20:19:06 +0000

What you’ll learn

By the end of this section, you’ll

Understand what virtual machines (VMs) are and why they were created.
Learn the problems VMs solve and their limitations
See why containers exist and how they compare to VMs
Get an introduction to Docker and why it is used
Complete a hands-on project to run an Ubuntu container and execute basic commands inside it.

How were applications traditionally run?

Before we get into virtual machines and containers, let’s step back and talk about how teams used to run software in the early days.

Now, every application, as you might know already, needs to run somewhere, right?

And that means it requires a computer, which in turn needs an operating system (OS) such as Linux, Windows, or macOS.

On top of that, applications rely on what we call “dependencies,” like runtime libraries or language versions, to function properly.

Now, before what we call “virtualization,” which you will soon understand, each workload had its own server.

By “workload”, I mean any software or service running on a server, like a web app, database, or file server, that uses system resources such as CPU, memory, and storage.

To better understand this, let me explain what this looked like in practice:

Let’s say a company wants to run three different parts of its system:

a web application that customers interact with
a database that stores all the data
a file server for internal documents and media

Now, this company wants to keep things simple and avoid problems like one service breaking another.

For instance, the database could need a different version of a library that the web app can’t work with, which is usually called “software version conflicts”.

So what do they do?

They set up three separate physical servers, with one for each.

Meaning that each server has:

its own operating system
its own set of dependencies
and it only runs one service, so that nothing conflicts

Okay, now that you get the “gist”. Next, I’ll tell you what was wrong with this setup.

What were the limitations?

Don’t get me wrong, this setup worked, okay? But it came with serious issues like:

It was expensive: You had to buy and maintain separate hardware for each workload, even if it didn’t use all the resources.
Resources were wasted: Most servers were sitting idle for most of the time, using only 10-30% of their total capacity.
Scaling was hard: If you needed more resources for the web app, for example, you couldn’t just tweak something. You had to buy a whole new server, install everything again, and configure it from scratch.

So, the question became:

How can we run more than one workload on the same machine, without creating conflicts or wasting resources?

That’s what led to the rise of “Virtualization”, which I will define in the next section.

So… what is virtualization & virtual machines (VMs)?

Now that you understand what brought about virtualization. It’s time to understand what it is.

Virtualization allows multiple operating systems to run on the same physical machine.

So, in place of having one OS per machine, you can create multiple virtual machines on a single physical server.

Each virtual machine acts like a separate computer with its own operating system, memory, and storage, even though they all share physical hardware.

Example showing where virtualization is applied

Next, I’ll give you a real-world use case in cloud computing so that you can better understand how virtualization works in practice.

Have you heard of cloud providers? Like AWS, Google Cloud, or Microsoft Azure? They all use virtualization to rent out virtual machines in place of physical machines.

So when you create a cloud server with any of these cloud providers, what’s really happening is you’re getting a virtual machine running inside a massive data center (which is a facility filled with thousands of interconnected physical servers that host VMs for multiple users).

Now, without virtualization, as you can see, cloud computing wouldn’t exist, and companies would have to buy and maintain their own physical servers. If you don’t understand what cloud computing means here, this is how I’d define it:

Cloud computing is the ability to access computing resources like servers, storage, and databases over the internet without owning a physical hardware.

So, what makes all this (virtualization) possible is this special software called a “hypervisor” which allows multiple VMs to run on the same physical machine. See the illustration below:

What problems did virtual machines solve?

As you can see, VMs solved the wasted resources problem of physical servers because with virtualization, one physical server could host multiple VMs, each running different applications.

And with this came several benefits like:

Better resource utilization: A single machine can run multiple applications while maximizing resources.
Cost savings: Fewer physical machines are needed which automatically reduces the cost of hardware and maintenance.
Isolation: Each application runs its own VM, thereby preventing conflicts between applications.

Now, even though VMs obviously improved things, they still had limitations and that’s what I’ll talk about next.

What were the limitations of virtual machines?

There are several reasons why VMs are not always the best solutions, let’s quickly run through them:

Heavy resource usage: Each VM runs a full OS, which takes up a lot of RAM and CPU.
Slow startup: Booting a VM can take minutes, just like starting a computer.
Inefficient scaling: Spinning up new VMs requires significant computing power and takes time.
OS redundancy: If you run 10 Ubuntu VMs, you’re running separate copies of Ubuntu, wasting storage.

Because of these limitations, it led to the need for “Containers” which we would discuss next.

What are containers, and how do they compare to VMs?

Containers solve many of the problems VMs have. In place of running a full operating system for each application, containers share the host OS, making them lightweight, fast, and resource-friendly. See the illustration below:

So, how are containers different from VMs? Look at the table below to understand their differences.

Feature	VMs	Containers (Docker)
Startup time	Minutes	Seconds
Resource usage	High (full OS per VM)	Low (shared OS)
Portability	Limited (OS-dependent)	High (runs anywhere)
Scalability	Slow	Fast (instantly spin up containers)
Isolation	Strong (separate OS)	Strong but lightweight

With containers, applications start in seconds instead of minutes, and they use fewer resources since they don’t need a full OS for each instance.

Next we’ll talk about the platform that makes this possible.

What is Docker, and why use it?

Docker is a containerization platform that allows developers to create, deploy, and manage containers easily. In place of setting up separate VMs, you can package an application with all its dependencies into a lightweight, portable container.

An example use case of Docker can be seen when developer use Docker to make sure that applications run exactly the same way in development, testing, and production environments. It takes out the “works on my machine” problem by making software behave the same way everywhere.

Next, we’ll do a mini project to put all you’ve learned so far into practice.

Mini-project: Run an Ubuntu container using Docker

Now that you understand the difference between VMs and containers, it’s time to get hands-on and run your first container using Docker.

But before we do that, let’s talk about what an Ubuntu container is and why you would want to run an Ubuntu container in the first place.

What is an Ubuntu container?

A containerized version of Ubuntu, also known as Ubuntu container, is a lightweight, minimal version of Ubuntu that runs inside a container. It does not include a full desktop environment, but it does have all the essential Linux utilities needed to run software.

When you install Ubuntu on a computer, it comes with:

The Linux kernel (which interacts with the hardware).
System files and utilities
Preinstalled software

When would you want to run an Ubuntu container?

Let’s see some reasons why you would want to run an ubuntu container in a practical scenario:

1. Testing software in a clean environment

Let’s say you’re developing an application and need to test it on Ubuntu 22.04, but your computer runs Windows or macOS. Instead of setting up a virtual machine, you can launch an Ubuntu container in seconds and test your application inside it.

2. Running Linux tools on a non-Linux system

If you use Windows or macOS, you may sometimes need access to Linux commands or tools that are only available on Ubuntu. Running an Ubuntu container gives you access to an Ubuntu terminal without installing Ubuntu on your machine.

3. Experimenting with a different Linux distribution

You might be working on a server that runs Ubuntu, but your computer runs another Linux distribution like Fedora or Arch. Running an Ubuntu container allows you to test commands in an Ubuntu-specific environment before applying them to a real server.

4. Learning Linux without installing a new OS

If you want to practice Linux commands but don’t want to reinstall your operating system, running an Ubuntu container gives you a safe place to try out Linux without affecting your main system.

What other containers can you run?

Ubuntu is just one example of a container you can run with Docker. There are many different container images available for different purposes, including:

Alpine Linux: A lightweight Linux container for minimal environments.
Nginx: A web server container to serve web pages.
PostgreSQL: A database container for managing data.
Node.js: A container with Node.js preinstalled for JavaScript development.
Python: A container with Python and all necessary dependencies for scripting.

You can pull and run any of these containers using Docker, just like you will with the Ubuntu container.

Note: Follow the steps in this article to properly install and setup Docker before you go on

Running an Ubuntu container using Docker

Let’s now run an Ubuntu container and interact with it as if it were a real Ubuntu system.

Step 1: Download the Ubuntu container image

Before you can run an Ubuntu container, you need to download the official Ubuntu image from Docker Hub.

To pull the Ubuntu image, open your terminal and run:

docker pull ubuntu

This command downloads the latest Ubuntu container image from Docker Hub.

Understanding the output of docker pull ubuntu

Once you run the command, you’ll see several lines printed in the terminal. Let’s break down what each part means so you know exactly what’s happening.

docker pull ubuntu command

Using default tag: latest

You didn’t specify which version of Ubuntu you want, so Docker used the default tag, which is latest. That means it will pull the most up-to-date version available.

If you wanted a specific version, you could run:

docker pull ubuntu:22.04

latest: Pulling from library/ubuntu

This shows where the image is coming from.

library/ubuntu is the official Ubuntu image maintained by Docker.
It lives on Docker Hub, which is Docker’s public registry of container images.
2f074dc76c5d: Pull complete

Docker images are built in layers. Each one adds something on top of the previous layer.

This message means a specific layer of the Ubuntu image has been successfully downloaded.

Digest: sha256:...

This is the unique ID (checksum) of the image you pulled. Think of it like a fingerprint for this exact version. It helps Docker verify the integrity and version of the image.

Status: Downloaded newer image for ubuntu:latest

Docker checked your system to see if you already had the image.

In this case, it didn’t find it or found an older version, so it downloaded the newer one.

If the image was already up to date, you would see:

Status: Image is up to date for ubuntu:latest

docker.io/library/ubuntu:latest

This confirms the full path of the image you now have locally:

It came from docker.io (Docker Hub)
It’s the official library/ubuntuimage
The tag is latest

Now that the Ubuntu image is on your system, you're ready to use it to run your first container, which we’ll do next. You don’t need to download it again; Docker will use this image from your local machine.

Step 2: Run the Ubuntu container

Once the image is downloaded, you can start an Ubuntu container.

Run this command:

docker run -it ubuntu

What does this command do?

docker run tells Docker to start a new container.
-itmakes it interactive and gives you access to a terminal inside the container.
ubuntu tells Docker to use the Ubuntu image (if it wasn’t already downloaded, Docker pulled it from Docker Hub automatically).

After running this command, your terminal will change. You are now inside the Ubuntu container, running a Linux shell.

Your terminal will look like this:

What you see in your terminal root@6f63eabbab0e:/# is the Ubuntu container's shell prompt. You are now inside the container, running Ubuntu as the root user.

Let’s break it down:

root → You are logged in as the root user (the default in most containers).
@6f63eabbab0e → This is the short container ID assigned by Docker. It uniquely identifies your running container.
:/# → You are currently at the root directory (/) inside the Ubuntu file system. The # symbol confirms you're logged in as root.

From this point, you can run Linux commands inside the container as if you were using a real Ubuntu server. You're not in a simulation, you’re in a real Ubuntu environment, isolated from your host machine.

Step 3: Check the OS version inside the container

You are now inside a working Ubuntu container, but keep in mind that this is a minimal version of Ubuntu. It’s stripped down to keep the container lightweight, so some common commands are not included by default.

To check the Ubuntu version running inside the container, you can use this built-in command:

cat /etc/os-release

This command reads a system file that contains the OS version details. You should see output similar to this:

Here’s what each part means:

PRETTY_NAME="Ubuntu 24.04.2 LTS"

This is the full name of the OS version, written in a human-readable way. In this case, it's Ubuntu version 24.04.2, Long-Term Support (LTS).
NAME="Ubuntu"

This confirms the base distribution is Ubuntu.
VERSION_ID="24.04"

This is the base version number of the operating system. It's commonly used by scripts or automation tools.
VERSION="24.04.2 LTS (Noble Numbat)"

This gives both the version number and the codename ("Noble Numbat") assigned to this Ubuntu release.
VERSION_CODENAME=noble and UBUNTU_CODENAME=noble

These provide the codename in a more machine-friendly format.
ID=ubuntu and ID_LIKE=debian

These are used by tools to identify what kind of system they're running on. ID_LIKE=debian means that although this is Ubuntu, it behaves similarly to Debian.
The HOME_URL, SUPPORT_URL, and BUG_REPORT_URL lines point you to official Ubuntu resources for learning more or reporting issues.
LOGO=ubuntu-logo

This is mostly used in graphical environments or tools that display branding.

This output confirms that your Ubuntu container is based on Ubuntu 24.04.2 LTS, and you're working inside a clean, isolated Linux environment (even if your main system is running something else like Windows or macOS).

If you prefer using the lsb_release command (which gives similar OS version details in a cleaner format), you'll need to install it manually, because it isn’t included in the minimal Ubuntu image.

So, run this in your container:

apt update && apt install -y lsb-release

Let’s break this down:

apt update tells Ubuntu to refresh its list of available packages. Think of it as checking for the latest versions and availability of software.
apt install -y lsb-release installs the lsb-release utility. The y flag tells Ubuntu to automatically confirm that you want to proceed, so it won’t stop and ask you to type “yes.”

After running the command, you’ll see an output like this:

You’ll see a long list of messages in your terminal. That’s completely normal, let’s quickly walk through what’s happening so you know what to expect.

First, Docker connects to Ubuntu’s package servers and pulls the most up-to-date list of software. You’ll see lines like:

Get:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease

These are the repositories being contacted. You don’t need to interact with any of this, just let it run.

Once the package list is refreshed, Ubuntu starts installing the lsb-release tool. You’ll see a few confirmations like:

The following NEW packages will be installed: lsb-release

That just tells you this package wasn’t already on the system and is being added now.

It then downloads the package and unpacks it:

Unpacking lsb-release...
Setting up lsb-release (12.0-2)...

This part completes in a few seconds. Once you see the “Setting up” line, you’re ready to use the command.

Now you can type:

lsb_release -a

and you’ll get a clean, structured summary of the Ubuntu version your container is running. Let’s move on to that next.

Here’s what that command does:

lsb_release is a small tool that prints version info about the current Linux distribution.
The a flag means "all," so you’ll see a full breakdown: the distribution name, description, version, and codename.

This output will be very similar to what you saw earlier with cat /etc/os-release, but a bit more focused and formatted for readability.

If you're in a situation where you're writing shell scripts or doing system automation and you only need the codename or release number, lsb_release is a quick way to get exactly that.

Either method is valid, you can stick with cat /etc/os-release if you don’t want to install anything extra, or use lsb_release -a if you prefer its format.

Step 4: Install software inside the container

You can install software inside the Ubuntu container just like you would on a normal Ubuntu system.

For example, to install curl, run:

apt update && apt install -y curl

What’s happening here?

apt update updates the list of available packages.
apt install -y curl installs curl without asking for confirmation (y).

Once installed, you can run:

curl --version

This verifies that curl is now available inside the container.

Step 5: Exit the container and restart it later

After you’re done working inside the container, you can exit it by typing:

exit

This will stop the running container and return you to your regular terminal prompt. Exiting doesn’t delete the container, it just stops it temporarily.

Check for stopped containers

To see a list of all containers (including the ones that have stopped), use:

docker ps -a

This command shows both running and exited containers. The output in your screenshot looks like this:

Let’s break this down line by line:

CONTAINER ID: This is the unique ID Docker assigns to each container. You can use this ID to start, stop, inspect, or remove a container. In the screenshot above, 6f63eabbab0e refers to your Ubuntu container.
IMAGE: This shows which Docker image was used to create the container. In this case, it shows you used the ubuntu image and also ran hello-world earlier.
COMMAND: This is the default command that runs when the container starts. For Ubuntu, it’s "/bin/bash", which opens an interactive shell. For hello-world, it’s "/hello", which just prints a message and exits.
CREATED: This tells you how long ago the container was created. It helps you keep track of how old a container is, especially if you’re managing several. For example, 26 hours ago shows your Ubuntu container was created a day ago.
STATUS: This shows the current state of the container. If it says Exited, the container is stopped. If it says Up, then the container is running. You can also see how recently it exited (like About a minute ago).
PORTS: This column lists any port mappings between your host machine and the container. For example, if a web server container exposes port 80, this column would show which host port it's connected to. In your case, the Ubuntu container has no ports exposed, so this is blank.
NAMES: Docker assigns a random, readable name to each container if you don’t give it one yourself. In the screenshot, the Ubuntu container was named keen_meninsky. You can rename containers or assign a custom name when creating one using the -name flag.

Restart and attach to a stopped container

If you want to restart the Ubuntu container and attach to it, use:

docker start -ai <container_id>

Replace <container_id> with your container ID, which in this example is 6f63eabbab0e, so you would run:

docker start -ai 6f63eabbab0e

start brings the container back to life.
a means “attach” — so your terminal will connect to the container’s input and output.
i means “interactive” — so you can type commands and see results like before.

Once this command runs, you’re back inside the same container, with everything still intact. This is helpful if you've installed tools or created files in your container and want to pick up where you left off.

The screenshot below confirms this worked. You're back at the container prompt:

You’ve covered a lot already, and it’s the kind of foundation that sets you up for everything else we’ll do with Docker.

Here’s what you’ve done:

You learned how applications used to run, and why virtualization became necessary.
You understood what virtual machines are, where they’re useful, and where they fall short.
You broke down why containers were introduced, and how they solve those limitations.
You saw the difference between VMs and containers using clear examples.
You got a proper introduction to Docker (what it is and how it fits into the container world).
You ran your first hands-on container, checked the OS inside it, installed software, exited it, and brought it back.

If you followed along with the mini-project, you now know how to pull images, run containers interactively, install software inside them, and restart them after they’ve exited. That’s a major first step.

Build a Discord command bot in minutes with Appwrite Cloud Functions

Deborah Emeni — Wed, 29 Nov 2023 10:42:20 +0000

If you’re a developer looking for a secure and efficient solution to build your Discord bots rapidly, then you’ve come to the right place. In this article, we’ll discuss an excellent tool that you can use to suit your needs called Appwrite Functions.

Appwrite Functions is a feature provided by Appwrite that enables developers to create and automatically run custom backend code in response to events triggered by Appwrite or according to a predefined schedule. With Appwrite Functions, you can access several benefits, including scalability and enhanced security.

Why use a template for your Discord bots? Templates save time and effort by providing pre-built code for custom commands and features. Appwrite Functions offers a Discord command bot template, simplifying the development process. You can customize the template to meet your needs and seamlessly integrate unique functionalities.

This tutorial will walk you through the process of building your Discord command bot in minutes using Appwrite Functions.

So, without further ado, let's dive in!

Prerequisites

Ensure that you have the following to follow along with the tutorial:

Creating your Appwrite project

Log into your Appwrite Cloud account and create your Appwrite project to gain access to Appwrite's services. You’ll also gain access to your project ID and API key, which are required for using the Appwrite SDK. Set a name for your project, such as 'Discord Command Bot,' and then click the Create Project button:

Creating an Appwrite Function with the Discord command bot template

Here, we’ll easily create an Appwrite Function to automate your backend tasks and extend Appwrite with custom code.

Navigate to the side menu on your project dashboard and click on Functions to access the template:

Click on the Templates tab on the Functions page:

Search for the Discord Command Bot template in the Search templates box as follows:

Click the Create function button:

Select Node.js-18.0 as the runtime for your discord command bot and click Next:

Before you proceed, you’ll need to retrieve several values from your Discord Developer P ortal, namely the public key, ID, and Bot token of your application in your Discord Developer Portal. They will serve as the values for the environment variables passed to your function at runtime.

On your Discord Developer Portal, click on New Application:

Enter the name of your application and click on the Create button:

Once you do that, your application ID and public key will be generated. Copy both of them:

Then, go back to your Functions in Appwrite and paste your copied Application ID and Public key as the values of your environment variables that will be passed to your function at runtime:

Go back to your Discord Developer Portal to get your DISCORD_TOKEN variable, and on the slide menu, click on Bot, then click on the Reset Token button:

Once you do that, you’ll see the following message: click Yes, do it!

Once you do that, you’ll get a 6-digit authentication code in your 2FA authentication app or whichever verification method you use. After entering the 2FA code, click on Submit:

Once you do that, your token will be generated. Click the Copy button to copy your token:

Go back to your Appwrite Functions, paste your copied token as the value for your DISCORD_TOKEN variable, and click the Next button:

This will take you to the next step of connecting your function to a new repository or an existing one within a selected Git organization. Select Create a new repository and click Next:

You need to select a Git repository that will trigger your function deployments when updated. Choose GitHub from the options and click Next:

Grant Appwrite permission to your GitHub account by clicking the Install & Authorize button:

Once that is done, you will be redirected to Appwrite to continue your installation. Now that Appwrite has access to your GitHub, you can go back to choose the Create a new repository option, and this time, you will see the following:

As you can see, a discord-command-bot repository has been automatically created for you. You can change the repository name if you so choose. You can leave your repository private and click Next. Afterward, your repository will be successfully created.

Next, name your production branch or leave it as main, then click the Create button:

Once you do that, you will see an alert saying that your Discord Command Bot has been created:

Your deployment will automatically display the deployment ID, build time, size, source, and link to your domains. Copy the link to your domains:

Configuring an interactions endpoint for your app

Go back to your Discord developer portal from your side menu, click on General Information, and scroll down to the Interactions Endpoint URL to configure an interactions endpoint to receive interactions via HTTP POSTs rather than over Gateway with a bot user:

Paste the link to your domains that you copied to configure an interactions endpoint by adding the https:// in front of it and the “interactions” endpoint at the end. Your link should look like this: https://[your domains]/interactions. Afterward, ensure that you save your changes by clicking the Save Changes button:

Generating an invite link for your app

Next, generate an invite link for your application by picking the scopes and permissions it needs to function. To do this, go to OAuth2 on the side menu, and in the dropdown menu, click on URL Generator. From the SCOPES section, select bot:

Under the Bot Permissions section, select Administrator:

Once you select your scope and bot permissions, a URL will be generated. Scroll down to the bottom of the page and click the Copy button to copy your generated URL:

Adding your application to your Discord server

Go to your favorite browser and paste the URL in the search bar. This will open Discord with a message telling you that an external application wants to access your Discord account with information about what you are permitted to do, which includes adding a bot to a server and creating a command. Remember to have a server created and ready to go.

Select the server that you would like to add from your list of servers and click the Continue button:

Grant administrator permission to your server and click the Authorize button:

Once you do that, you’ll be required to confirm that you are not a robot:

Once you’ve been verified, your application will be successfully authorized and added to the server you created on Discord. Click on the Go to Appwrite Bot Server button to go to your server:

This will take you to your server on Discord:

Now, you can add commands to your Discord command bot using Appwrite functions.

Customizing the template by adding Commands to your Discord command bot

With the Discord command template provided by Appwrite functions, you have access to a variety of customization options. You can do the following:

Add, remove, or modify commands.
Customize the responses of the Discord bot to commands.
Set permissions for who can use the commands.
Register the Discord bot to listen for and respond to Discord events, including reactions when a member leaves the Discord server or messages.

Not only do you have access to several customization options, but you also benefit from the speed that the Discord command bot template provides. For instance, instead of spending time developing the Discord bot from scratch, the Discord command bot template offers you a pre-built foundation with code that you can use to get started quickly. If you want to add more features to your discord bot, such as commands, you can easily customize the template by creating a new function and registering it with the Discord bot.

Let’s see this in action by customizing the template by adding a command to your Discord server that tells jokes.

In your Discord command bot GitHub repository, you have a setup.js file that registers the command on Discord by defining a name and description for the command. By default, the template already registers a “Hello World Command”.

Here’s the link to the complete source code.

Let’s create another command by registering the command in the setup.js file. Modify the code in your setup.js file with the following:

    import { fetch } from 'undici';
    import { throwIfMissing } from './utils.js';
    async function setup() {
      throwIfMissing(process.env, [
        'DISCORD_PUBLIC_KEY',
        'DISCORD_APPLICATION_ID',
        'DISCORD_TOKEN',
      ]);
      const registerApi = `https://discord.com/api/v9/applications/${process.env.DISCORD_APPLICATION_ID}/commands`;
      const commands = [
        {
          name: "hello",
          description: "Hello World Command"
        },
        {
          name: "jokes",
          description: "Tell a random joke"
        }
      ]

      const responses = await Promise.all(commands.map((command) => {
        return fetch(registerApi, {
          method: 'POST',
          headers: {
            Authorization: `Bot ${process.env.DISCORD_TOKEN}`,
            'Content-Type': 'application/json',
          },
          body: JSON.stringify(command),
        });
      }))
      if (responses.some((response) => !response.ok)) {
        throw new Error('Failed to register command');
      }
      console.log('Command registered successfully');
    }
    setup();

Here’s a detailed explanation of what this code does:

The throwIfMissing() function checks if the environment variables that you set in your Appwrite function exist — here, this will include your DISCORD_PUBLIC_KEY, DISCORD_APPLICATION_ID, and DISCORD_TOKEN.
The registerApi saves the URL to add the command to the Discord bot.
The commands variable is an array of the commands you want to add to Discord.
The responses variable stores the response from Discord after a request is made to register the command on your Discord server. If it fails, it throws an error; if it is successful, it logs a message to the console saying, “Command registered successfully.”

Next, go to the main.js file, which contains the code that handles the request to add a command to Discord and modify the code with the following:


    import {
      InteractionResponseType,
      InteractionType,
      verifyKey,
    } from 'discord-interactions';
    import { throwIfMissing } from './utils.js';
    export default async ({ req, res, error, log }) => {
      throwIfMissing(process.env, [
        'DISCORD_PUBLIC_KEY',
        'DISCORD_APPLICATION_ID',
        'DISCORD_TOKEN',
      ]);
      if (
        !verifyKey(
          req.bodyRaw,
          req.headers['x-signature-ed25519'],
          req.headers['x-signature-timestamp'],
          process.env.DISCORD_PUBLIC_KEY
        )
      ) {
        error('Invalid request.');
        return res.json({ error: 'Invalid request signature' }, 401);
      }
      log('Valid request');
      const interaction = req.body;
      if (
        interaction.type === InteractionType.APPLICATION_COMMAND &&
        interaction.data.name === 'hello'
      ) {
        log('Matched hello command - returning message');
        return res.json(
          {
            type: InteractionResponseType.CHANNEL_MESSAGE_WITH_SOURCE,
            data: {
              content: 'Hello, World!',
            },
          },
          200
        );
      }
       if (
        interaction.type === InteractionType.APPLICATION_COMMAND &&
        interaction.data.name === 'jokes'
      ) {
        log('Matched jokes command - returning message');
        const response = await fetch("https://v2.jokeapi.dev/joke/Any?type=twopart");
        const data = await response.json();
        return res.json(
          {
            type: InteractionResponseType.CHANNEL_MESSAGE_WITH_SOURCE,
            data: {
              content: `${data.setup} ---------- ${data.delivery}`,
            },
          },
          200
        );
      }
      log("Didn't match command - returning PONG");
      return res.json({ type: InteractionResponseType.PONG }, 200);
    };

The code does the following:

It verifies the data coming from the Discord server.
It gets the name of the registered command, and it responds with the appropriate data.

After customizing the template, commit your changes and push them to GitHub.

Once you commit and push your changes, your building process will start, and your code will be deployed on Appwrite. Go to your Appwrite Functions and click on the Deployments tab; you will see your build time and other information.

Scroll down on your Deployments tab to see the status of your deployment:

Finally, you can test this out to see if your command was successfully added to your Discord server by going to your discord server and typing “/jokes” as follows:

You can see the description of your command that you set below your command name:

Once you hit enter, it starts sending your command to the Appwrite Function for execution and displays a response:

Conclusion

This tutorial demonstrated the use of Appwrite Cloud Functions, which provides a Discord command bot template for building a Discord command bot. You learned how to customize the template and add commands to your Discord server.

Resources

You may find the following resources useful:

Demystifying AWS VPC Network Firewall using Terraform

Deborah Emeni — Tue, 24 Oct 2023 21:10:32 +0000

This post was originally published on Hackmamba.

Many organizations that deal with sensitive data, such as financial information, may suffer from cyber attacks such as malware infections, data breaches, distributed denial-of-service (DDoS) attacks, and so on, which could result in financial loss. Fortunately, organizations can avoid such situations by creating a Virtual Private Cloud (VPC) with a tool like Amazon Web Services (AWS), allowing them to monitor their network for security threats and isolate it from the public internet.

Organizations must ensure their infrastructure is always secure by implementing security measures such as firewalls, data encryption, access control, among other features. Organizations can use Terraform, a powerful and reliable tool, to automate the deployment of security groups; this tool can help control access to an organization’s resources while ensuring that the security groups remain compliant with their security policies.

When manually building infrastructure, the organization risks many human errors. However, combining AWS and Terraform reduces the risk of human errors and enables organizations to build a secure and scalable infrastructure.

This article will help you understand the AWS VPC Network Firewall and how it improves cloud security. It will also include a hands-on demonstration of using Terraform to manage and automate your project’s AWS VPC Network Firewall configurations.

What is AWS VPC Network Firewall?

To understand the AWS VPC Network Firewall, it's essential to grasp the concept of a firewall. A firewall is a network security device that filters incoming and outgoing network traffic based on security rules, acting as a barrier between a private network and the public internet to block bad or malicious traffic.

Depending on an organization's or individual's needs, a private network or public internet can be chosen. If there are concerns about system security, cost, control, or performance, a private network becomes the preferred option due to its significant benefits over the public internet. A private network is more secure than the public internet, using private IP addresses and enabling control through security devices like firewalls to manage network traffic.

Unauthorized access to a private network can severely impact an organization or individual, leading to financial losses from cyber attacks such as data breaches and disruptions, rendering the network unavailable. Implementing security measures such as firewalls and intrusion detection systems is crucial to protect private networks.

AWS VPC Network Firewall secures and protects your AWS VPC from unauthorized access. As an AWS service, it's managed by Amazon, relieving you of infrastructure management. This service watches incoming and outgoing traffic, identifying and blocking malicious activity to maintain VPC security.

Benefits of AWS VPC Network Firewall
AWS VPC Network Firewall is an excellent choice for organizations due to its numerous benefits.
Let’s explore a few of them.

Scalability: Applications can experience surges or drops in traffic depending on the number of active users. If your app can't handle more users, it might slow down or crash. To prevent this, monitor for issues and consider adding extra resources, like more servers. It's hard to do this yourself, so the AWS VPC Network Firewall can help. If users suddenly increase, it adjusts automatically, saving time and money.
Performance: If your app can't handle many users, it becomes slow and might not work well when many people use it simultaneously. This can make your users dissatisfied, make your app less safe, and even cost you money. To avoid this, you can use AWS VPC Network Firewall. It helps your app handle lots of traffic without slowing down using rule-based filtering and stateful inspection techniques.
Cost: Using AWS VPC Network Firewall is a cost-effective choice as it allows you to only pay for the resources you use. This means that you are charged based on the amount of your application’s traffic and the number of firewalls deployed. Additionally, Since AWS VPC Network Firewall is a managed firewall service, it saves the cost of managing and maintaining your firewall by yourself.
Security: AWS VPC Network Firewall provides features such as intrusion prevention (which helps detect and block malicious traffic), web filtering (which can help block traffic), and more for protecting your VPCs from malware and brute force attacks.

Features of the AWS VPC Network Firewall
What makes AWS VPC Network Firewall stand out are its extensive features. Let’s take a look at them.

Stateful firewall: This feature of AWS VPC Network Firewall keeps track of the connections between your network and other networks to allow or block traffic based on the type of traffic and the direction of the connection. A few examples of what the firewall tracks include source and destination IP addresses, ports, and protocol type.
Automated scaling: AWS VPC Network Firewall offers automatic scaling for the firewall capacity of your network to scale up or down based on the traffic load.
Intrusion prevention: This feature utilizes signature-based detection to inspect network traffic patterns for matches against known threat signatures to protect your VPC from unauthorized access or malicious activities.
Web filtering: This feature blocks unencrypted web traffic to known malicious websites and monitors fully qualified domain names (FQDNs) for encrypted web traffic using the Server Name Indication (SNI) extension that blocks access to specific sites.
Central management: This feature centrally manages and enforces firewall policies across multiple VPCs to ensure the same security policies protect all the resources.

How does AWS Network Firewall work?
Because it is an AWS service, AWS Network Firewall requires the use of an Amazon VPC. When you create an AWS account, it will contain a default VPC for each AWS region, and if you choose to, you can create more VPCs.

After creating your VPC, you can add subnets (where each subnet must reside within one Availability Zone to allow the launching of AWS resources in separate Availability Zones) and then deploy AWS resources like Elastic Compute Cloud (EC2 ) instances in your VPC. In addition, you’ll need to configure the route tables for your VPC to send Network traffic through the Network Firewall endpoints before the AWS Network Firewall is enabled.

So, the AWS Network Firewall protects the subnets you’ve added to your VPC by filtering the traffic between the subnets and locations outside your VPC.

The following is an illustration from AWS Documentation of how AWS Network Firewall works:

The AWS Network Firewall filters the traffic using r ule groups, either stateless (evaluating packets in isolation) or stateful (evaluating packets in the context of their traffic flow). You can configure these rules inside a firewall policy. The configuration involves several settings, including specifying the subnets for the firewall endpoints for each Availability zone defined and more.

You can write your stateful rules in Suricata-compatible format, and the Network Firewall will process the rules using a Suricata rules engine.

Then, the AWS Network Firewall uses a firewall to connect the inspection rules configured in the firewall policy to the VPC.

Overview of Amazon Virtual Private Cloud (VPC)

You’ve seen the term Amazon Virtual Private Cloud (VPC) mentioned a couple of times, and from all that has been explained, you may now have some clue about what it is. For clarity, let’s take a look at a brief overview.

What is Amazon VPC?
Amazon Virtual Private Cloud (Amazon VPC) is an AWS service that lets you establish a secure and isolated virtual network for deploying AWS resources. This network resembles a traditional on-premises network but leverages AWS's scalable infrastructure.

Core features of Amazon VPC
Amazon VPC consists of primary features, as shown in the illustration below from the Amazon VPC documentation:

The core features include the following:

VPCs: VPCs (Virtual Private Clouds) are isolated sections of the AWS cloud that you can use to launch AWS resources. Each VPC has its IP address range, subnets, and routing tables.
Subnets: Subnets are smaller divisions of a VPC. You can launch AWS resources, such as Amazon EC2 instances, into subnets. Subnets must be located within a single Availability Zone.
Route tables: Route tables control how traffic is routed within a VPC. Each route table contains a list of routes, specifying the destination IP address range and the next hop router.
Internet gateways: Internet gateways allow traffic to flow between a VPC and the internet. You can also use internet gateways to connect your VPC to other VPCs or your on-premises network.
VPC Peering: VPC Peering allows you to connect two VPCs together without using an internet gateway or VPN. This can be useful for connecting VPCs that are in the same region or different regions.

Types of Amazon VPCs
There are two types of Amazon VPCs: default VPCs and non-default VPCs. Let’s explore them below.

Default VPCs: These VPCs are created automatically when you create an AWS account. They are pre-configured with a single subnet in each Availability Zone in the Region. You can use a default VPC to launch your Amazon EC2 instances, Amazon Relational Database Service (RDS) instances, and other AWS resources. To create a default VPC, check the user guide provided by Amazon VPC.

The illustration below shows the components of a default VPC from the Amazon VPC documentation.

Non-default VPCs: These types of VPCs are VPCs that you create yourself. You have more control over the configuration of a non-default VPC, such as the IP address range, the number of subnets, and the routing configuration. You can also create non-default VPCs in multiple Regions.

Advantages of using Amazon VPC
Amazon VPC offers several advantages over a traditional data center, including the following:

Cost-effectiveness: Amazon VPC is a cost-effective way to host your applications and data. You only pay for the resources that you use.
Scalability: Amazon VPC is scalable, so you can easily add or remove resources. You can also create multiple VPCs to meet the needs of different applications or workloads.
Security: Amazon VPC provides high security for your data and applications. Your VPC is isolated from other VPCs and the public internet, and you can control who can access your resources.
Reliability: Amazon VPC is a reliable platform that is backed by Amazon's infrastructure. Your data is stored in multiple Availability Zones, protecting it from unplanned outages.

Things to consider when setting up an Amazon VPC
Here are some things to consider when setting up your Amazon VPC.

Determining IP address ranges: When setting up your VPC, you need to consider how the resources within your VPC would communicate with each other and how they would communicate with resources over the internet.

So, depending on the size of your VPC, which also depends on the number of resources that you plan to deploy, you’d need to select a range of IP addresses (such as 172.16.0.0/16, 192.168.0.0/16) that would be used for this purpose. You can use the Classless Inter-Domain Routing block (CIDR) to represent the range of IP addresses.

Selecting Availability Zones (AZs): You must consider your application's availability and fault tolerance when setting up your VPC. When you create your VPC, you can deploy your resources in multiple Availability Zones, isolated locations within a highly available, fault-tolerant region with its own networking infrastructure, power, and cooling.

Doing so would ensure that your application does not suffer a single point of failure. For instance, if one Availability Zone goes down, the other Availability Zones will still have your application and data.

Determining Internet Connection for Resources: You need to decide how your resources in the VPC will connect to the Internet. You can choose to have a public subnet directly connected to the internet or a private subnet connected to the internet through a VPN or AWS Direct Connect.
Creating the VPC: Once you have considered the above factors, you can create your VPC. When creating your VPC, you need to specify the IP address range, the number of AZs, and the internet connectivity options.
Consideration of the Applications’ Architectural Design: When designing your VPC, you need to consider the architectural design of your applications. For example, if you have an application that needs to be highly available, you must place its components in multiple AZs.

How to utilize Terraform for AWS VPC Network Firewall management

Terraform is an Infrastructure as Code (IaC) tool used to automate the creation and management of AWS VPC Network Firewall rules. It streamlines the process by providing automated provisioning, ensuring consistent and repeatable deployments, enabling version control for tracking changes and easy rollbacks, and aiding in audibility to maintain compliance with security policies.

Let’s do a quick demo showing how to use Terraform to create and manage AWS VPC Network Firewall rules.

Prerequisites
To follow along, you’ll need the following:

Knowledge of Terraform
Terraform installed (check the documentation to install Terraform for your operating system)
AWS Command Line Interface (CLI) installed (install here)
Terminal

Creating project directory
Now, let’s create a directory called terraform_project that will house the configuration file called “conf.tf”. In your terminal, run the following commands:

mkdir terraform_project
cd terraform_project
touch conf.tf

Since we want to utilize Terraform for AWS VPC Network Firewall management, we’ll need to define and configure AWS as our Cloud Provider for the project. In your conf.tf file, add the following configuration:

provider "aws" {
  region = "us-east-1"
}

In the configuration above, you set the AWS provider to operate in the “us-east-1” region. AWS has its own set of regions available for deployment, so you can use any desired region of your choice. You specified the region by setting the ‘region’ parameter. This means that any resources you define within this configuration will be created in the “us-east-1” region.

Before proceeding, ensure that you have added your Access and Secret access keys from your AWS account.

Defining the Resources
Since you have defined the cloud provider, you can now define the resources you want to create using the AWS provider. As mentioned earlier in the article, AWS provider offers a wide range of resources that you can create using Terraform such as S3 Bucket ("aws_s3_bucket" type), AWS EC2 instance (“aws_instance” resource type), VPC ("aws_vpc" resource type), Security Group ("aws_security_group" resource type), Subnet ("aws_subnet" resource type) and more.

Let’s create an AWS EC2 instance, which would involve setting up networking components like VPC, subnets, and security groups. Using Terraform’s configuration language within the conf.tf file defines the following:

resource "aws_vpc" "sample_vpc" {
   cidr_block = "10.0.0.0/16"
}
resource "aws_subnet" "sample_subnet" {
   vpc_id     = aws_vpc.sample_vpc.id
   cidr_block = "10.0.0.0/24"
}
resource "aws_security_group" "sample_security_group" {
   name_prefix = "sample-security-group"
   ingress {
     from_port = 22
     to_port   = 22
     protocol  = "tcp"
     cidr_blocks = ["0.0.0.0/0"]
   }
 }
 resource "aws_instance" "sample_instance" {
   ami           = "ami-0c55b159cbfafe1f0"
   instance_type = "t2.micro"
   subnet_id     = aws_subnet.sample_subnet.id
   security_groups = [aws_security_group.sample_security_group.name]
      tags = {
        Name = "SampleInstance"
      }
 }

From the configuration above, you are setting the following:

The aws_vpc resource creates a VPC with the specified CIDR block.
The aws_subnet resource creates a subnet within the VPC.
The aws_security_group resource defines a security group that allows incoming secure shell (SSH) traffic (port 22) from anywhere (0.0.0.0/0).
The aws_instance resource uses the previously created VPC, subnet, and security group to launch the EC2 instance. Note that AWS provides several EC2 instance types, differing in terms of memory, CPU, storage, and more. Here, you’re using the "t2.micro" instance that is designed for small to medium workloads.

Initializing your Terraform configuration
Now that you have defined the resources, you need to initialize your Terraform configuration to ensure that the necessary components, such as plugins and state dependencies, are in place.

Within your project directory, run the following command to initialize Terraform:

terraform init

After running the command, Terraform will be initialized:

Previewing changes
Based on your configuration, you can take this precautionary step for early detection of errors in your configuration to reduce the chances of failed deployments and to see what changes Terraform will make to your infrastructure.

terraform plan

After the command runs successfully, Terraform will use the selected providers to generate the execution plan, as shown in the screenshots below:

Applying changes
Executing the planned infrastructure modifications to create or update resources based on your Terraform configuration is necessary. Run the following command in your project’s directory:

terraform apply

Conclusion

You’ve come to the end of this article, where you learned about AWS VPC Network Firewall, including what it is, its benefits, features, and how it works. You also learned about Amazon VPC, including what it is, its core features, types, and advantages. The article also included a demo showing how to set up and use Amazon VPC and Terraform for AWS VPC Network Firewall Management.

Resources

You may find the following resources helpful:

How Appwrite Cloud frees up time for freelancers to drive creative solutions

Deborah Emeni — Thu, 03 Aug 2023 13:37:19 +0000

Juggling multiple projects, freelancers worldwide often struggle to meet tight deadlines and maintain high-quality work. This struggle not only strains them but also hampers their productivity. For example, freelancers could spend more time on essential tasks for another project instead of spending time on mundane tasks for one job.

Fortunately, tools like Appwrite Cloud exist to help freelancers save time and automate tedious tasks, thereby improving their productivity. Appwrite Cloud offers numerous benefits, allowing freelancers to focus on delivering high-quality work to their clients.

This article discusses the time-saving features in Appwrite Cloud and how you can leverage them as a freelancer.

Why is a time-saving tool essential for freelancers?

Most freelancers strive for maximum productivity to deliver high-quality work and satisfy their clients. They often use time-saving tools to handle repetitive and common tasks such as project management, organization, communication, file sharing, collaboration, task tracking, etc.

Let's explore the importance of a time-saving tool in the life of a freelancer through this illustration below.

John is a freelance programmer who handles multiple projects for different clients simultaneously. These projects require his attention and skill. On a regular day, John faces challenges in manually tracking tasks, managing deadlines, and organizing his workflow.

Luckily for John, he discovered a project management tool that could help him streamline processes, centralize project information, automate repetitive tasks, track his time, set reminders, collaborate with clients, and prioritize work effectively.

As a result, John can now spend more time on coding and problem-solving, allowing him to deliver high-quality work promptly and maximize his productivity.

Overview of Appwrite Cloud

Appwrite Cloud is the open-source version of Appwrite, a backend as a service (BaaS) platform that offers a range of features and tools, including time-saving capabilities for freelancers. As a BaaS platform, Appwrite handles the entire backend infrastructure, such as REST API management and server provisioning, relieving freelancers from these responsibilities.

Freelancers can use the Appwrite Cloud platform to easily incorporate features like user management, file storage, and push notifications into their applications without developing them from scratch. Moreover, Appwrite Cloud offers a collaborative environment for freelancers to chat, work together on their apps, and conduct code reviews or issue tracking.

How Appwrite Cloud benefits freelancers

Let’s explore some of the time-saving features Appwrite Cloud offers and how they significantly enhance the productivity of freelancers.

Automated deployment
Freelancers working on multiple projects simultaneously may experience delays in manually deploying their code changes. The manual deployment process can hinder progress when new features must be deployed to production and when obtaining user feedback is crucial. As a result, freelancers might deploy features to production that could adversely affect the user experience, leading to a loss of user trust. This, in turn, would require additional time to fix and update, potentially leaving the client dissatisfied with the freelancer's work quality.

To avoid such situations, freelancers can utilize Appwrite Cloud's a utomated d eployment feature, which automatically deploys their code changes to production. This enhances freelancers' productivity by saving time on manual deployment, allowing them to focus on tasks like debugging or building new features.

The automated deployment feature also prevents deploying bad code changes to production by automatically running a series of tests before deploying the changes.

Scalability
Several issues may arise when a freelancer fails to handle a traffic spike in an application on time. The application's performance can be affected, leading to downtimes that impact the user experience. Additionally, the business and the client may suffer financial losses, which would reflect poorly on the freelancer.

However, freelancers can leverage Appwrite Cloud’s auto s caling feature, which enables simple scaling of applications up or down based on traffic, without worrying about infrastructure or manual adjustments.

Freelancers have several options for utilizing the auto scaling feature, including:

Manual scaling: Freelancers can manually adjust the allocation of resources to their applications, allowing them to handle sudden traffic spikes by increasing or decreasing the resources as needed.
Automatic scaling: Freelancers can let Appwrite Cloud automatically scale their applications up or down dynamically, based on the workload.
Load balancing: Appwrite Cloud offers load-balancing capabilities to enhance application availability and performance, enabling the distribution of applications across multiple servers.

Real-time monitoring
Real-time updates play a vital role in ensuring high-quality work delivery and optimal application performance. When developers don't receive immediate notifications about errors in their application, they may unknowingly continue working with incorrect information, leading to potential issues in other parts of the application and causing delays in resolving the problem. Consequently, this could waste time and effort and hinder development. Fortunately, with Appwrite Cloud, freelancers can swiftly identify and address issues by promptly receiving error notifications — improving efficiency, client satisfaction, and meeting project deadlines.

Appwrite Cloud offers a r eal-time feature that allows developers to seamlessly integrate real-time updates into their applications. By utilizing Appwrite's real-time feature, freelancers can create their dashboard to receive timely updates and notifications whenever changes occur, facilitating efficient workflow, troubleshooting, and collaboration.

Automated workflows
Freelancers regularly perform repetitive tasks such as writing code for similar functionalities across various projects, updating and troubleshooting code, conducting software testing, deploying applications, and communicating with clients or team members. The manual execution of these tasks can be challenging, prone to errors, time-consuming, counterproductive, and inefficient.

Automating these repetitive tasks is highly productive, resulting in better accuracy, improved quality of work, and time savings. Freelancers can easily automate these tasks using the Automated Workflow feature provided by Appwrite Cloud, which seamlessly integrates with popular CI/CD tools like GitLab and GitHub Actions.

Freelancers can utilize Appwrite Cloud's serverless functions to configure workflows that perform actions, such as emailing the client whenever a project update is made.

Database management
Freelancers may encounter challenges in creating, managing, and accessing databases for their projects, which can hinder their productivity. These challenges may arise from inefficient database systems or a lack of seamless access to databases, resulting in time-consuming processes for organizing and storing data. Consequently, the quality of work delivered may also be affected.

Nevertheless, freelancers can effectively and securely address these challenges by utilizing the Database Management feature offered by Appwrite Cloud. This feature facilitates the creation, management, and access of databases, enabling freelancers to handle their projects in a timely manner and efficiently.

Check out this article to create an account and explore Appwrite.

Conclusion

This article explored how Appwrite Cloud saves freelancers time and highlighted the time-saving features of Appwrite Cloud and how they greatly enhance freelancers' productivity.

Resources

You may find the following resources useful:

Building Trust in BaaS Platforms: The Essential Guide to Better Data Management

Deborah Emeni — Mon, 27 Mar 2023 10:05:26 +0000

In recent years, there has been a spike in the number of Backend-as-a-Service (BaaS) platforms like Appwrite, Firebase, AWS Amplify, and so on that render many essential services, such as database management, push notifications, cloud storage, user authentication, hosting, and more. Businesses have taken advantage of BaaS platforms to handle server functionalities and backend tasks so that their applications can continue to run smoothly, saving development costs and time.

As the number of BaaS platforms increases, businesses have more providers to select from. Security is a major concern for businesses that handle user data (that includes personal information like addresses, credit card information, and so on). Businesses require specific details on the critical factors to consider when selecting a BaaS provider.

This article will provide information about the essential factors when choosing a BaaS provider or platform for businesses. It will teach you the best practices for secure data management with BaaS providers.

Overview of BaaS Platforms

BaaS platforms are cloud architectures that handle server functionalities and repetitive backend tasks for businesses. They provide Application Programming Interfaces (APIs) and Software Development Kits (SDKs) through which applications connect to and gain access to third-party cloud services. As a result, developers can focus on the client side of the applications.

Use Cases of BaaS Platforms in Businesses

There are numerous BaaS use cases in various business sectors ranging from financial tech (fintech) to gaming or even healthcare. The server side of applications requires significant effort including the database management, user authentication, push notifications, and API management.

Let's take a look at some of these use cases.

1. Database Management
BaaS platforms offer services that store and manage business data in the cloud and allow the application to scale while synchronizing and maintaining a unified user profile. BaaS platforms have several database service mechanisms where good BaaS platforms often provide services such as creating, listing, getting, updating, and deleting documents among other things.

2. User Authentication
Businesses can use BaaS platforms to authenticate and manage user accounts, such as creating, registering, updating, verifying, and deleting user accounts. BaaS platforms provide services such as email/phone verification, two-factor authentication, password recovery, session creation, and more.

3. Notifications
BaaS platforms provide cost-effective cloud infrastructures with an easy-to-use API for sending users' notifications (push/email/SMS). As a result, businesses can avoid the stress and time required to build and manage their notification infrastructure. Businesses concerned about user engagement and sending real-time notification updates to their users can use BaaS platforms.

4. API Management
Most applications rely on multiple APIs to provide third-party services. Businesses can use BaaS platforms to securely create and manage their APIs, including controlling access, checking and tracking usage, setting usage quotas (rate limits), and reducing the effort and time required to build and manage the infrastructure. BaaS platforms offer various API management services, such as rate limits and a dashboard for tracking API usage and downtime.

How BaaS Platforms are Critical for Better Data Management

BaaS platforms have improved data management for businesses due to the vital and critical benefits they offer. Businesses have built more cost-effective, scalable, accessible, and real-time data solutions using BaaS platforms. Businesses use BaaS security measures to enable data backups, disaster recovery, and security.

Furthermore, BaaS providers offer cross-platform support, which means they support multiple Operating Systems/tools/platforms. Businesses can integrate with multiple tools and platforms and easily migrate data from one platform to another (web to mobile) without affecting application performance.

Essential factors to consider when selecting BaaS platforms for businesses

When selecting a BaaS platform, businesses must consider several important factors. Understanding these factors will enable businesses to develop trust in BaaS platforms.

Let's take a closer look at some of these factors.

1. Security measures and updates
Before choosing a BaaS platform for your business, you must consider the security procedures it employs. These procedures include the authentication, data encryption, access control, and so on that the BaaS platform employs.

In application downtime or disaster, you should note how the BaaS platform handles disaster recovery to avoid losing your data. Appwrite, for example, uses JSON Web Tokens (JWT), social logins, API keys, and OAuth2 for authentication and authorization.

2. Data Transparency
Data is intricate and should be handled securely and with care when developing and managing applications. Before selecting a BaaS platform, consider its policies to handle data in terms of transparency, validation, tracking, updates, formatting, security, accessibility, ownership, and usage.

You can use Appwrite to access various data access methods such as queries, formatting (JSON), real-time data updates, and CRUD operations.

3. Monitoring performance
Monitoring the performance of running applications is critical and is best practice to avoid situations such as downtime. Not only that, but the ability to note any issues or errors the application may encounter and test the application's performance in real time is critical.

You should ensure that the BaaS platform you select can monitor application performance in real time. Appwrite provides a Health API service for monitoring the application's performance and an Error Reporting service for reporting performance errors.

4. Data/Industry regulations compliance
When choosing a BaaS platform, it is critical to ensure that the BaaS complies with data regulations governing security, storage, and data collection. Ensure that the BaaS platforms have security standards certifications such as ISO/IEC 27001 and that they comply with the industry regulations where the business operates.

Appwrite processes data under the General Data Protection Regulation (GDPR) and has a detailed Privacy Policy for data retention, transfer, disclosure, and security.

5. Scalability
As the business expands and the traffic volume spikes as the number of users increases, it is essential to verify the BaaS platform's ability to handle the increased load without affecting the application's performance. It is also critical to ensure that the BaaS platform can efficiently distribute multiple requests across multiple instances.

Appwrite handles business traffic by automatically scaling applications with Kubernetes and distributing incoming requests across multiple instances in a way that does not affect application performance.

Security Best Practices for Businesses using BaaS platforms

To secure data management for businesses, it's essential to know these best practices described below.

1. Regular security audits
It is best practice to analyze the BaaS platform's security policies and methods to ensure that they comply with security regulations to ensure your business's data privacy and security (e.g., GDPR). You can identify potential security risks and vulnerabilities that can be exploited by conducting regular analyses.

Appwrite performs regular security audits using various mechanisms such as penetration testing, log analysis, and vulnerability scanners. Then, it provides a report on the results of each security audit, including the vulnerabilities discovered and ways to address them.

2. Executing proper access control procedures
Businesses need to consider the type of access control procedures used by the BaaS platform. Some of the access control procedures include software updates, data encryption, and the type of user authentication and authorization used.

Appwrite secures data for businesses by utilizing access control procedures such as JSON Web Tokens (JWT) for authentication and Hypertext Transfer Protocol Secure (HTTPS) for encryption.

3. Using secure APIs
To avoid a security breach in business sensitive data, it is best practice to ensure that the application's underlying APIs are secure. The BaaS platform should use a standard authentication method, such as Appwrite, which uses OAuth or multifactor authentication and monitoring logs to report security issues and validation to secure the APIs.

Appwrite secures APIs with various tools, including regular software updates, SSL/TLS encryption, validation, monitoring logs, rate limiting, etc.

4. Regular security updates
Typically, BaaS platforms send notification alerts to users when a security update is required. Pay attention to any BaaS platform alerts to ensure that your software is up to date. This decreases the risk of security vulnerabilities in your application.

5. Evaluating third-party security
Third-party services may introduce security vulnerabilities into a business application. Before integrating a third-party service into your application, ensure that it meets security standards. Ensure that its source is trusted, and perform a proper evaluation, which may include going through the documentation in search of any known vulnerabilities.

Conclusion

In this article, we explored an overview of BaaS platforms such as Appwrite, use cases of BaaS platforms in businesses, how BaaS platforms are critical for better data management, essential factors to consider when selecting BaaS platforms for businesses, and security best practices for businesses using BaaS platforms. Appwrite is a good BaaS platform and you can quickly get started with it.

Resource

Check out Appwrite Documentation

A Beginner's Introduction to DevOps

Deborah Emeni — Mon, 27 Mar 2023 09:57:01 +0000

We live in a society where customer satisfaction and application performance are critical to the success of any software product designed for human consumption. Since developers build applications, they are prone to errors, which may affect code quality. As a result, necessary updates and changes may need to be integrated into the software in real time without interfering with the application's performance.

Many organizations have realized in recent years that using the set of practices involved in DevOps is the key to a faster release cycle of quality software, including collaboration between Developers and Operations teams.

This article will teach you about DevOps, the Software Development Life Cycle (SDLC), its limitations, and the advantages of DevOps over the traditional system. You'll also learn DevOps practices, terminology, and case studies.
Table of Contents

What is DevOps?
Understanding the SDLC
A system architecture for an e-learning platform
Limitations of the Traditional SDLC
Benefits of DevOps over the traditional system
Set of Practices and Terms used in DevOps
Case Studies: Examples of Companies implementing DevOps practices
● Google
● Microsoft
● Netflix
● Amazon
Conclusion
Resources

What is DevOps?

DevOps is a set of practices that fosters collaboration between IT teams and software developers to automate the operations involved in both development and IT to achieve faster software deployment and code quality.

I know, that's a lot of buzzwords, but don't worry, you'll understand better as you read on

You must first understand the Software Development Life Cycle process before diving into DevOps.

Understanding the SDLC

The Software Development Life Cycle comprises processes/stages that follow a hierarchical pattern. These stages are as follows:

Planning Stage —- This is where the project's scope (features and functions) and everything needed to develop the project are defined.
Analysis Stage — This is where we analyze the requirements that have been gathered, which could be the components (i.e. the building blocks that would make up the system). For example, the hardware, software, data exchange mechanisms, data itself, interfaces that promote interaction between the components and sub-systems, and network infrastructure for the application's running and deployment.

Then, we define their use cases and apply our findings to conceptualize the system architecture, which defines how the system components will collaborate to achieve the project's end goal.

See the illustration below to better understand what a system architecture entails:

A system architecture for an e-learning platform

Design stage —The next stage is to create the detailed design from the software architecture, which includes the user interface and database schema.

Below is an illustration of a System design:

Implementation stage - This is where the integration of the components that have been designed happens — the writing of the code and unit tests.

Testing stage - This is the stage where the software is tested to find errors or bugs before production.

Deployment stage - This is the stage at which the application is made available for users to install.

Maintenance stage - This is where the software is monitored to detect any problems, fix them, or make necessary updates.

The stages listed above can be carried out using different methodologies such as, the Waterfall model (which employs a sequential approach) and the Agile methodology (which uses an iterative approach).

The Waterfall model requires completion of each stage before moving on to the next, whereas Agile focuses on continuous feedback and communication between a team of developers and customers to ensure that the software developed meets the needs and requirements of the customers.

I'll emphasize the Agile methodology because it is the method DevOps uses to carry out SDLC. What kind of practices happens in the Agile methodology?

Continuous planning and prioritization of work
Continuous integration and testing of code
Continuous delivery of working software
Regular retrospectives to evaluate and improve the process

Limitations of the Traditional SDLC
The Waterfall model was the traditional SDLC methodology before DevOps, and it has several limitations, including the following:

Lack of flexibility: When changes are made to the software later on, there is no going back to make any updates because the model uses a linear approach. It would mean restarting the entire SDLC from scratch.

Lack of customer involvement: When developing a product for human consumption, including customers in the SDLC process is critical to ensure that the product meets the customer's needs. I mean, what's the point of a product that customers don't find useful, right?

Slow development: The Waterfall model's architecture makes it much more challenging to develop and make software available to users.

High Cost: With the sequential approach to SDLC, a significant amount of testing, which is costly, will be required because there is no going back if an error occurs in the future.

Because of these limitations, many organizations have shifted to an iterative approach to SDLC management. DevOps, on the other hand, employs a flexible and iterative approach — the Agile methodology.

Let's look at some of the advantages DevOps provides to compensate for the limitations of traditional SDLC.

Benefits of DevOps over the traditional system
DevOps has several benefits compared to traditional SDLC (Software Development Life Cycle). They include the following:

Flexibility: DevOps is designed to be flexible, compensating for the lack of flexibility caused by traditional SDLC methodologies. The speed with which DevOps responds to and updates software demonstrates flexibility.

Collaboration: DevOps encourages teamwork and collaboration between development and operations teams.

Quality: DevOps enhances code quality through practices such as automated testing and infrastructure as code.

Customer satisfaction: DevOps allows customers to participate in the SDLC process, resulting in software that meets the customer's needs.

Cost: DevOps significantly reduces costs by employing an iterative rather than a sequential approach, which typically necessitates rework.

Faster release cycles: DevOps reduces the time it takes to release software by automating collaboration between development and operations teams.

Next, let's look at some of the DevOps practices and terminologies

Set of Practices and Terms used in DevOps

As you learn DevOps, you will come across these terms, and it is good practice to understand what they mean and how they work. Let's look at some of them:

Continuous Integration & Continuous Delivery (CD/CD):: DevOps ensures that every piece of code tested by developers is integrated as soon as possible. Also, DevOps ensures software changes are automatically built, tested, and deployed to production.

Infrastructure as Code (IAC): To configure and manage infrastructures, DevOps employs tools such as Ansible, Puppet, and Chef.

Automated Testing: DevOps automates testing such as unit testing, integration testing, and acceptance testing.

Monitoring and Logging: DevOps also monitors the system's performance to identify any problems or errors affecting the system's overall performance.

Collaboration and Communication: DevOps promotes teamwork and communication between development and operations teams.

Agile Methodologies: DevOps employs Agile methodologies such as Scrum to respond to software changes and provide continuous customer feedback.

Automation Tools: I've mentioned several times that DevOps uses automation. Several tools, including Jenkins, Docker, and Kubernetes, automate the process.

Version Control: A code change management system that allows developers to track changes and collaborate with others.

Before learning a new technology, it is critical to understand real-world applications of the technology. It provides a better idea of what you can create with that technology!

Next, we'll look at some real-world DevOps use cases.
Case Studies: Examples of Companies implementing DevOps practices
These organizations use DevOps practices such as Continous Integration and Delivery, automated testing, and containerization to easily and quickly deliver new services and features, as well as high-quality software.
Google

Microsoft

Netflix

Amazon

Conclusion
In this article, we explored what DevOps is, the limitations of the traditional SDLC and the benefits DevOps offers. We also covered the set of practices in DevOps and showed some real-world case studies.

Resources
You may find the following resources useful:
Continuous Integration & Continuous Delivery (CI/CD)
Infrastructure as Code (IAC)
Automated testing
Monitoring and Logging
Collaboration and Communication
Agile Methodologies
Automation Tools
Version Control

Picking the Right Backend for your AI Project

Deborah Emeni — Mon, 30 Jan 2023 19:19:25 +0000

This article was originally posted on Hackmamba.

One of the primary considerations when developing an application is choosing the right infrastructure. A standard application that provides services to users would require a proper backend infrastructure to handle user data, security, servers, and APIs.

Backend developers must also find the right tools for each project, including programming languages, servers, databases, frameworks, etc. There are numerous advantages to selecting the right backend infrastructure for a project and it will have a significant impact on the user experience and the project's success or growth.

This article will teach you about the various backend infrastructure types suitable for an AI project.

Overview of Backend Infrastructures

Backend Infrastructures are all the components and tools required to design the backend architecture for your AI application by handling backend services such as storage, authentication, database, security, and so on, allowing you to focus on your application's frontend functionality.

Your AI application's foundation is based on your backend infrastructure, which can vary in type depending on your AI application's requirements while considering factors such as cost-effectiveness, budget, bandwidth, volume of work, project type, project timeline, or scalability.

To choose the best backend infrastructure for your project, you must weigh the pros and cons of each backend infrastructure.

Use Cases of Backend Infrastructures

There are several use cases of backend infrastructures for your project, including:

Maintenance: Backend Infrastructures handle the backend maintenance of your application, from updates to bug fixes, saving you the stress of doing it yourself. This use case must be considered because it prevents downtime in your application.
Security: The safety of your users' data should be a top priority. Backend infrastructures can protect your servers and databases from cyber-attacks.
Speed and Scalability: Backend infrastructures provide speed which improves user experience. Backend infrastructures can provide scalability for your application depending on the type of project and the rate at which it grows in traffic volume.

Infrastructure Requirements for AI Project

The performance of your AI projects is highly dependent on your infrastructure as it handles a vast amount of the workload, like deep learning and algorithms. Choosing an Infrastructure for an AI project depends on the following project's requirements:

Storage Capacity: The amount of data generated or real-time data required varies depending on the scope of your AI project. Your database will grow as your AI project grows and this should be considered when selecting the right infrastructure. The level of data generated by your AI project should influence your infrastructure selection.
Security: Security is a vital requirement when choosing an Infrastructure. AI project inferences are highly dependent on user data, so you should consider cases like data breaches and bad data which could result in incorrect inferences. Your choice of Infrastructure should be capable of providing a high level of security for your AI project.
Computing Capability: AI projects require high computing power and speed to process large amounts of data, which includes algorithms, machine learning, and neural networks. The infrastructure should be capable of providing high computing power, such as CPUs for handling workloads and cloud-based GPUs (Graphic Processing Units) for deep learning.
Cost-effective solutions: As the complexity of AI projects grows, so does the cost of managing them and the demand for infrastructure services such as storage, servers, and networks. Consider the long-term cost-effectiveness of your infrastructure before selecting it for your AI project.
Networking Infrastructure: The expansion of your AI project involving deep learning algorithms will be powered by multiple containers that depend primarily on communication. Your infrastructure should be capable of providing a viable and scalable networking service while taking into account network speed, scalability, high bandwidth, reliability, and low latency.

Types of Backend Infrastructure Suitable for AI Projects

Cloud infrastructures (Hybrid Cloud) are the foundation of AI projects to handle the projects' flexibility and adaptability. The workload and demand on the volume of data increase as the project scales and cloud infrastructures can handle the workload and meet the demands at a reasonable cost without affecting performance. Some of these infrastructures include:

Infrastructure-as-a-Service (IAAS)
Platform-as-a-Service (PAAS)
Backend-as-a-Service (BAAS)

Let’s look at each of these infrastructures in detail.

IAAS

IAAS is a cloud infrastructure service that consists of resources that provide services such as networking, computing, and storage. These resources help you cut costs by allowing you to pay for only the resources you need for your AI project. IAAS providers like Azure and AWS support AI projects.

Pros and Cons of IAAS

It allows you to migrate your project to the cloud and eliminate the need to manage your backend on premises.
Provides services that can scale and handle a large workload depending on your project's requirements without affecting application performance.
The infrastructure configuration is less transparent and visible, preventing easy monitoring.
Downtime at an IAAS provider can impact your project's workload.

PAAS

PAAS is an infrastructure comprised of storage, servers, development tools, and networks that support the entire application life cycle from development to testing, deployment, and updating. As with IAAS, you can pay for only the resources you need as you continue to use the platform. Google App Engine is a PAAS suitable for AI projects.

Pros and Cons of PAAS

Provide free access to various project testing options, such as operating systems, languages, databases, or development tools.
Provides services to developers that enable provisioning, application development, and easy collaboration.
The migration from one PAAS provider to another is complex because the project depends heavily on the provider's platform.
Data security can be at risk as the entire project is on the cloud.

BAAS

BAAS is an infrastructure that handles an application's backend or server side, such as pre-defined authentication, database, hosting, cloud storage, and so on, allowing developers to focus on building the application's front end.

Pros and Cons of BAAS

BAAS can seamlessly handle the backend of your application.
It saves time and money that would otherwise be spent on backend developers.
Increased data and project expansion can cause infrastructure costs to skyrocket.
Migrating from one BAAS provider to another is difficult because the backend is built on the provider's architecture.

Appwrite for AI Projects

Appwrite is an open-source BAAS that offers a set of easy-to-use APIs for outstanding backend infrastructure services, making it suitable for AI projects. Let's take a look at some of these services.

Authentication

Appwrite offers a Service for authenticating, updating, creating, and retrieving user accounts; as well as built-in integration with multiple OAuth providers, including GitHub and Google. This service allows you to manage user registration and log in for your applications easily.

Database

The database service enables you to manage your data collection in a structured or flexible manner. Each document can have multiple child components with functionality that allows you to set permissions for users and teams.

Security

Appwrite supports several encryption methods that provide security services for your project by providing end-to-end protection and preventing cyber-attacks on your application's data.

Storage

The Appwrite Storage service allows you to manage file uploads and downloads with the ability to assign read or write permission using an access control method.

Conclusion

Choosing the right backend infrastructure is critical to the growth and performance of an AI project. In this article, you learned about backend infrastructures, their use cases, their requirements for an AI project, types, and the Appwrite services for building AI projects.

Resources

Check Appwrite Documentation for more information on getting started with its services.

Forem: Deborah Emeni

How to run a multi-container app with Docker compose

What you'll learn

Why managing containers manually doesn’t scale

What is Docker Compose?

Understanding the docker-compose.yml structure

Setting up the demo project

Prerequisites

What we’re building

Project structure

Step 1: Creating our docker-compose.yml file

Create the file

Step 2: Start with the smallest working Compose file

Run it

Step 3: Add MongoDB as a service

Update your docker-compose.yml

What changed?

Run it again

Step 4: A quick recap of what just happened

Step 5: Testing the application

Step 1: Check the health endpoint

Step 2: Save some data

Step 3: Read the data back

Step 6: Adding persistent storage with volumes

Stop everything

What are volumes?

Update your docker-compose.yml

Test it

Step 7: Managing configuration with a .env file

What is a .env file?

Create a .env file

Update docker-compose.yml

Run it again

Everyday Docker Compose commands

Start all services

Start in the background (detached mode)

Rebuild after code changes

View logs

Check running containers

Stop everything

Reset everything (including database data)

Wrap up

What’s next in the Docker learning series?

Run a MySQL container with persistent storage using Docker volumes

What you'll learn

Why do containers lose data?

A quick example to visualize this

What are Docker volumes?

Why are Docker volumes used?

Bind mounts vs named volumes - what’s the difference?

Bind mounts

Named volumes

See this table below that summarizes this comparison

How do containers store and retrieve data using volumes?

What exactly happens when a volume is mounted?

What does the data flow look like, in simple terms?

Hands-on project: Run a MySQL container with persistent storage using Docker volumes

Before you begin

Step 1: Pull the MySQL image

Step 2: Create a named Docker volume

Step 3: Run a MySQL container using the volume

Step 4: Connect to MySQL inside the container

Step 5: Stop and remove the container

Step 6: Start a new container using the same volume

Connect back into MySQL

Verify that your data is still there

Step 7: Inspect the volume (optional)

So what you just learned (and why it’s useful)

Docker networking: How to connect containers in a full-stack project

What you'll learn

Why Docker networking is important in multi-container applications

Okay, let me give you a common scenario.

So, what's the solution?

How Docker networking works

The 3 types of Docker networks

1. Bridge network (default for single-host setups)

2. Host network

3. Overlay network

Exposing container ports with -p

How containers communicate with each other internally

Step 1: Creating our `docker-compose.yml` file

Update your `docker-compose.yml`

Step 7: Managing configuration with a `.env` file

What is a `.env` file?

Create a `.env` file

Update `docker-compose.yml`

Exposing container ports with `-p`