Forem: Omolara Adeboye

Deploying A MERN Stack To-Do App on AWS: A Step-by-Step Guide

Omolara Adeboye — Wed, 02 Oct 2024 16:00:21 +0000

Project Overview
Prerequisites
Steps
- Step 0 Prepare Prerequisites
- Step 1 Configure the backend
- Step 2 Test the Backend code with Postman
- Step 3 Create the frontend
- Step 4 Create the React Components
- Step 5 Run the app
Conclusion

Project Overview

The MERN stack is a popular web development framework that combines four technologies: MongoDB, Express.js, React and Node.js. This stack allows developers to build robust, scalable web applications using Javascript accross both the client and server sides.

The application we will be deploying is a single-page application (SPA), a to-do app having dynamic content.It can be used to add to-do items, display and delete to-do content.

Prerequisites

AWS account
AWS CLI
Basic knowledge of React, Node.js, Express.js
Basic knowledge of Linux

Steps

Step 0 Prepare Prerequisites

Launch an EC2 instance named mern-server. We will be using an ubuntu instance LTS 24.04.

SSH into the server from your local machine or development environment

ssh -i /path/to/key ubuntu@[public-IP]

Step 1 Configure the backend

Update and upgrade the ubuntu instance:

sudo apt update -y && sudo apt upgrade -y

Install Node.js and npm on the server:

sudo apt install -y nodejs npm

Check node and npm version

node -v
npm -v

Set up application code

First, we will create a new directory for the To-do app project and verify its creation with the ls command:

mkdir Todo && ls

Change directory to the newly created directory:

cd Todo

Initialise the project by running:

npm init

Press enter/return key to accept the default values. Finally enter yes to accept.

This command creates a new package.json file within our folder that
contains the application code and the dependencies it requires to run.

The content of the package.json is similar to the image below:

Install Expressjs and the dotenv module Expressjs is a framework for Node.js. It helps to simplify development. We will install it with npm:

npm install express dotenv

Create the index.js file which is the entry point of the application:

vi index.js

We will write the following code into the index.js file. Open the file with an editor of choice and enter the code:

// Importing dependencies
const express = require('express');
require('dotenv').config();

// Creates the express app
const app = express();

// Setting the port
const port = process.env.PORT || 5000;

// CORS configuration
app.use((req, res, next) => {
  res.header("Access-Control-Allow-Origin", "*");
  res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
  next();
});

// Routes handling
app.use((req, res) => {
  res.send('Welcome to Express');   
});

// Starting the server
app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

The code sets up a simple web server using Node.js and Express.js. Comments have been included in the code to provide explanations of each sections. When a request is made to the server, CORS headers are added to the response. Irrespective of the request URL or method, the server reponds with, "Welcome to Express". The server listens on the specified port or defaults to 5000 if the port is not provided.

Save the file.

To test our server, we will run:

node index.js

The image below shows the running server.

Note the port the server is running on and open the port in the security group with the following command:

Retrieve the groupId of the security group by running the following command:

aws ec2 describe-security-groups --query "SecurityGroups[*].{ID:GroupId,Name:GroupName}"

Retrieve groupId for a specific security group

aws ec2 describe-security-groups --group-names "my-security-group" --query "SecurityGroups[*].GroupId" --output text

Create an ingress rule for the port from anywhere 0.0.0.0/0

aws ec2 authorize-security-group-ingress --group-id [GroupId] --protocol tcp --port [port] --cidr 0.0.0.0/0

Alternatively, set the security group ingress via the AWS console.

Retrieve your server's public IP with the instance metadata

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` && curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/public-ipv4

Next, access your server's public IP in your browser:

http://[public-ip]:5000

Hints/ Troubleshooting :

If you get "ERR_SSL_PROTOCOL_ERROR" , check that you are running the application of http not https.
When you run node index.js you will not be able work on that terminal, resolve this by ssh into the instance on another terminal. Alternatively, run the server with the ampersand symbol to run it in the background: node index.js &

Routes Our to-do app will be able to perform the following three actions which will be associated with specific endpoints.

Create a new task
Display list of all tasks
Delete a completed task

We will be using standard HTTP request methods: POST, GET. DELETE

We will create routes that will define the endpoints that the to-do app requires.

In a new terminal,create a folder called routes and cd into it.

mkdir routes && cd routes

We will create a file named api.js inside the folder, opening it with an editor of choice.

vi api.js

Enter the following code:

// Import Express and Create a router
const express = require('express');
const router = express.Router();

// Define routes

// Handles GET requests to retrieve a list of todos
router.get('/todos', (req, res, next) => {
  // Return the result as a JSON array
});

// Handles POST requests to create a new todo
router.post('/todos', (req, res, next) => {
  // Return the result as a JSON array
});

// Handles DELETE requests to delete a todo by ID
// `:id` is a route parameter, passed in the URL
router.delete('/todos/:id', (req, res, next) => {
  // Return the result as a JSON array
});

// Export the Router
module.exports = router;

The above code defines an Express.js router for handling CRUD (Create, Read, Delete) operations on our to-do app. Comments are included in the code to explain the functionality. To use this router, you would typically import it in your main server file and use it like this:

const todoRoutes = require('./api');
app.use('/api', todoRoutes);

Model
We will create a model since our app is using Mongodb, a NoSQL database. A model makes a javascript applications interactive. The model will be used to define the database schema. A schema is a layout of how the database will be constructed. We will install mongoose to create a schema and a model. mongoose is a node.js package that makes working with mongodb easier. it is an Object Data Modeling (ODM) library for MongoDB and Node.js

Install mongoose inside the Todo folder:

cd Todo && npm install mongoose

Create a models directory, cd into it and create a file named todo.js within the folder:

mkdir models && cd models && vi todo.js

Enter the following code into the todo.js file:

// Import mongoose
const mongoose = require('mongoose');
const Schema = mongoose.Schema; // retrieves the mongoose schema constructor

// Defines the todo schema
const TodoSchema = new Schema({
  action: {
    type: String,
    required: [true, 'The todo text field is required']
  }
});

// Create model for todo
const Todo = mongoose.model('todo', TodoSchema);

// exports the todo model to be used in other parts of the app.
module.exports = Todo;

The code defines a mongoose model for our todo app in a MongoDB database. Comments are included in the code to explain its functionality.

Since our MongoDB models has been set up, we need to update the routes in the api.js

Delete the previous content of the api.js file and enter the following:

// Import Dependencies 
const express = require('express');
const router = express.Router();
const Todo = require('../models/todo');

router.get('/todos', (req, res, next) => {
  Todo.find({}, 'action') // finds all the documents, selecting only the action field 
    .then(data => res.json(data))
    .catch(next);
});

router.post('/todos', (req, res, next) => {
  if (req.body.action) {
    Todo.create(req.body)
      .then(data => res.json(data))
      .catch(next);
  } else {
    res.json({
      error: "The input field is empty"    
    });
  }
});

router.delete('/todos/:id', (req, res, next) => {
  Todo.findOneAndDelete({"_id": req.params.id})
    .then(data => res.json(data))
    .catch(next);
});

module.exports = router;

This code sets up a RESTful API for a todo application, allowing clients to:

Retrieve all todos (GET /todos)
Create a new todo (POST /todos)
Delete a specific todo (DELETE /todos/:id)

It interacts with the MongoDB database using the Mongoose model we defined earlier. The routes now have actual implementations, unlike in the previous version where they were empty.

MongoDB Database We will use mongoDB to store our data. We will use a database as a service solution that used to be called mLab. mLab was a popular cloud database service for MongoDB, but it has since been acquired by MongoDB and merged into MongoDB Atlas, their cloud-based service. Since mLab no longer offers services independently, the process for creating a free MongoDB cluster now involves using MongoDB Atlas.

Sign up for MongoDB Atlas. Once signed in you will be taken to the MongoDB Atlas dashboard. And then, the Deploy your cluster page

Or follow the steps below:

Create a New Project: Click on New Project to create a new project where your cluster will reside. Name the project and click "Next".

Create a Free Cluster: In the newly created project, click "Build a Cluster", Choose "Shared Clusters" and select the "Free tier" (M0 cluster).You have options to configure the cloud provider (Choose AWS) and the region closest to you. Ensure you select the regions that have free tier available. We will choose the option of allowing access to the MongoDB database from anywhere.This is ideal for testing cases. [Note that stricter maeasures should be applied for production environment]. Also change the time of deleting entry from 6hours to 1 week.

Click on "Create your deployment", will display a page similar to the image below:

Note that, I have blocked out personal details,

From the pop-up you can set up your Network Access and Database user.

Set up MongoDB User: Click "Database Access" and then Click "Add New Database User". Choose "Password" for the authentication method and set the username and password.

I used a weak password for testing purposes. It is ideal to use strong passwords for production environments.

Set IP Whitelist: This allows your local machine or application to connect to MongoDB Atlas cluster. Click "Network Access", then click "Add IP Address". You can allow access from anywhere by entering 0.0.0.0/0. This is ideal for testing cases. [Note that stricter maeasures should be applied for production environment].

![image: Network access from Anywhere]](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/12zej1u6ihrdgn779sgx.png)

Choose a connection method: After setting the database user and network access, Close the pop-up. On the dashboard, click "Get connection string". Then click "Done"

Copy the connection string for use in the application:

It is in the following format:

mongodb+srv://username:<db_password>@todo-app-cluster.vj1ly.mongodb.net/?retryWrites=true&w=majority&appName=todo-app-cluster

Note that the UI may differ. But the process of configuration remains largely the same.

Creating an Environment variable file In our index.js we specified a process.env to access environment variables. Hence, we need to create the file named .env as follows in the todo directory and add the connection string:

vim .env

Add the connection string similar to the below:(Preferably get the connection string on the mongo Atlas dashboard)

DB = mongodb+srv://username:<db_password>@todo-app-cluster.vj1ly.mongodb.net/?retryWrites=true&w=majority&appName=todo-app-cluster

Next we will update the index.js to reflect the use of the .env so that Node.js can connect to the database. Simply delete the existing content of the file and add the following code:

const express = require('express');
const bodyParser = require('body-parser');
const mongoose = require('mongoose');
const routes = require('./routes/api');
const path = require('path');
require('dotenv').config();

const app = express();

const port = process.env.PORT || 5000;

// Connect to the Database
mongoose.connect(process.env.DB, { 
  useNewUrlParser: true,
  useUnifiedTopology: true 
})
.then(() => console.log(`Database connected successfully`))
.catch(err => console.log(err));

// Since mongoose promise is depreciated, we override it with node's promise
mongoose.Promise = global.Promise;

app.use((req, res, next) => {
  res.header("Access-Control-Allow-Origin", "*");
  res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
  next();
});

app.use(bodyParser.json());

app.use('/api', routes);

app.use((err, req, res, next) => {
  console.log(err);
  next();
});

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

This code sets up a RESTful API for a todo application, allowing clients to:

Retrieve all todos (GET /todos)
Create a new todo (POST /todos)
Delete a specific todo (DELETE /todos/:id)

It interacts with the MongoDB database using the Mongoose model we defined earlier. The routes now have actual implementations, unlike in the previous version where they were empty.
To use this router in your main application, you would typically do something like:

const todoRoutes = require('./routes/api');
app.use('/api', todoRoutes);

This would make the routes accessible under the /api path, so the full paths would be /api/todos for GET and POST, and /api/todos/:id for DELETE.

We will start the Node.js server:

node index.js

Step 2 Test the Backend code with Postman

We will test the backend code by using Restful API. We will use Postman to test our API.

Install postman on your machine or use the web version.

I will be using the web version as it provides equal functionality. Also, because I am using a cloud IDE.

Visit Postman website and sign up or sign in to your account.

In Postman Web, create a new request by clicking "Send an API Request".
Input the endpoint you want to test in the URL field. (A)
Set the method by clicking on the dropdown. (B)
set the body if necessary. (C)
To set the body, click on the body tab below the URL field >> Select raw >> add the task in json format. For example:

{
  "action": "take out trash"
}

Set the Headers: (D)

Key: value (Content-type: application/json)

(E) shows example result of the test on the POST request. The 200 OK status code means that the request was successful.

It is important to test all the endpoints:

Testing the HTTP GET request to display a list of tasks, we get the following:

Testing the HTTP DELETE request to test how to deleting an existing task:

if we check the GET endpoint again , we see that we are left with two tasks instead of three.

Step 3 Create the frontend

We will create a user interface for a web client to interact with the to-do app via API.

First scaffold the app with the create-react-app command. Run the following command in the Todo directory:

npx create-react-app client

This command creates a new folder in the Todo directory called client where we will add all the react code.

We will install the dependencies needed

install concurrently

npm install concurrently --save-dev

The command runs multiple commands simultaneously

install nodemon

npm install nodemon --save-dev

This is used to run and monitor the server. nodemon restarts and load new changes automatically if there is any change in the code.

Next, we will edit the package.json folder within our Todo directory to contain the following code:
Replace the "scripts" section with the following:

"scripts": {
"start": "node index.js",
"start-watch": "nodemon index.js",
"dev": "concurrently \"npm run start-watch\" \"cd client && npm start\""
},

Configure proxy in package.json

We will configure proxy in package.json

change directory to the client folder:

cd client

Open the package.json file:

vim package.json

Add the key value pair:

"proxy": "http://localhost:5000"

Including the proxy configuration makes it possible to access the application directly from the browser by calling the server url http://localhost:5000 rather than always including the entire path http://localhost:5000/api/todos

Navigate to the Todo directory and run:

npm run dev

The app will run on localhost:3000, the default port for react app.

Hint: Open security group port to allow TCP port 3000.

Step 4 Create the React Components

React makes use of reusable components which makes code modular. We will have two stateful components and one stateless component for the todo app.

Change directory to the src folder from the todo directory:

cd client/src

Create a folder called components and cd into it

mkdir components && cd components

We will create three files namely: Input.js, ListTodo.js Todo.js

In the input.js, enter the following code:

import React, { Component } from 'react';
import axios from 'axios';

class Input extends Component {
  state = {
    action: ""
  };

  addTodo = () => {
    const task = { action: this.state.action };

    if (task.action && task.action.length > 0) {
      axios.post('/api/todos', task)
        .then(res => {
          if (res.data) {
            this.props.getTodos();
            this.setState({ action: "" });
          }
        })
        .catch(err => console.log(err));
    } else {
      console.log('input field required');
    }
  };

  handleChange = (e) => {
    this.setState({
      action: e.target.value
    });
  };

  render() {
    let { action } = this.state;
    return (
      <div>
        <input type="text" onChange={this.handleChange} value={action} />
        <button onClick={this.addTodo}>add todo</button>
      </div>
    );
  }
}

export default Input;

This React component, Input, enables users to add new Todo items. It features a text input field and an "Add Todo" button, validating non-empty text input. Upon submission, it sends a POST request to /api/todos using Axios, refreshing the Todo list by calling the getTodos prop function. The component's state manages the input value, resetting it after successful submission. It's designed to interact with a RESTful API and is exported for use in larger Todo applications.

To make use of Axios, a Promise based HTTP client for the browser and node.js, cd into the client and run npm install axios or yarn add axios

cd ../../ # or cd client Depending on your current location on the terminal 

npm install axios

Change to the components directory:

cd src/components

Open the ListTodo.js with a text editor:

vim ListTodo.js

Paste the code below into the file:

import React from 'react';

const ListTodo = ({ todos, deleteTodo }) => {
  return (
    <ul>
      {todos && todos.length > 0 ? (
        todos.map((todo) => (
          <li key={todo._id} onClick={() => deleteTodo(todo._id)}>
            {todo.action}
          </li>
        ))
      ) : (
        <li>No todo(s) left</li>
      )}
    </ul>
  );
};

export default ListTodo;

This component is designed to display and manage a list of Todo items, allowing users to delete individual items.

Then add the following code to the Todo.js file:

import React, { Component } from 'react';
import axios from 'axios';

import Input from './Input';
import ListTodo from './ListTodo';

class Todo extends Component {
  state = {
    todos: []
  };

  componentDidMount() {
    this.getTodos();
  }

  getTodos = () => {
    axios.get('/api/todos')
      .then(res => {
        if (res.data) {
          this.setState({
            todos: res.data
          });
        }
      })
      .catch(err => console.log(err));
  };

  deleteTodo = (id) => {
    axios.delete(`/api/todos/${id}`)
      .then(res => {
        if (res.data) {
          this.getTodos();
        }
      })
      .catch(err => console.log(err));
  };

  render() {
    const { todos } = this.state;

    return (
      <div>
        <h1>My Todo(s)</h1>
        <Input getTodos={this.getTodos} />
        <ListTodo todos={todos} deleteTodo={this.deleteTodo} />
      </div>
    );
  }
}

export default Todo;

This React code defines a Todo component that manages a Todo list application. It fetches Todo items from a RESTful API on mount, displays them using the ListTodo component, and allows users to add new items via the Input component. The component also handles deleting Todo items by sending a DELETE request to the API and updating the list. The getTodos function is used to refresh the list after adding or deleting items, ensuring the UI stays synchronized with the API data.

We will open the App.js file located in src. Change directory to the src directory and enter the following:

import React from 'react';

import Todo from './components/Todo';
import './App.css';

const App = () => {
    return (
        <div className="App">

            <Todo />
        </div>
    );

}

export default App;

This code defines the main App component, which serves as the entry point for the application. It imports the Todo component and renders it within a div element with the class "App". The Todo component is responsible for managing a Todo list, and this App component provides a container for it. The application's CSS styles are also imported from App.css. Overall, this code sets up the basic structure for the Todo list application.

Still in the src directory , we will open the App.css and paste the following csscode:

vi App.css

.App {
    text-align:center;
    font-size: calc(10px + 2vmin);
    width: 60%;
    margin-left: auto;
    margin-right: auto;
}

input {
    height: 40px;
    width: 50%;
    border: none;
    border-bottom: 2px #101113 solid;
    background: none;
    font-size: 1.5rem;
    color: #787a80;
}

input:focus {
    outline: none;
}

button {
    width: 25%;
    height: 45px;
    border: none;
    margin-left: 10px;
    font-size: 25px;
    background: #101113;
    border-radius: 5px;
    color: #787a80;
    cursor: pointer;

}

button:focus {
    outline:none;
}

ul {
    list-style: none;
    text-align: left;
    padding: 15px;
    background: #171a1f;
    border-radius: 5px;

}

li {
    padding: 15px;
    font-size: 1.5rem;
    matrgin-bottom: 15px;
    background: #282c34;
    border-radius: 5px;
    overflow-wrap: break-word;
    cursor: pointer;
}

@media only screen and (min-width: 300px) {
    .App {
        width: 80%;
    }
}

input {
    width:100%
}

button {
    width:100%;
    margin-top: 15px;
    margin-left: 0;
}

@media only screen and (min-width: 640px) {
    .App {
        width: 60%;
    }
}

input {
    width: 50%;
}

button {
    width: 30%;
    margin-left: 10px;
    margin-top: 0;
}

In the src directory, also open the index.css and enter the following css code:

body {
    margin: 0;
    padding: 0;
    font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", "Roboto", "Oxygen", "Ubuntu", "Cantarell", "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif;
    -webkit-font-smoothing: antialiased;
    -moz-osx-font-smoothing: grayscale;
    box-sizing: border-box;
    background-color: #282c34;
    color: #787a80;
  }

  code {
    font-family: source-code-pro, Menlo, Monaco, Consolas, "Courier New", monospace;
  }

Step 5 Run the app

Cd into Todo directory run:

cd ../..
npm run dev

We will be able to view the app running on port 3000 as shown below. we will observe that the app has listed already, what we used in the postman tests. The todo list app is functional!

Conclusion

This project demonstrates the complete deployment process of a MERN stack to-do application on AWS. From setting up the backend with Node.js and Express.js, to integrating MongoDB Atlas for database storage, and finally deploying the React-based frontend, this guide covers the essential steps for developing and deploying a dynamic single-page application (SPA). The final product allows users to add, view, and delete tasks, leveraging AWS infrastructure for scalability and security.

LEMP STACK ON AWS

Omolara Adeboye — Thu, 26 Sep 2024 00:28:06 +0000

Table of Content

Project Overview
Steps
- Step 0 Prepare Prerequisites
- Step 1 Install Nginx web server
- Step 2 Install MySQL
- Step 3 Install PHP
- Step 4 Configure Nginx to use PHP processor
- Step 5 Test server block with html
- Step 6 Test server block with php
- Step 7 Retrieve data from the MySQl database with PHP
Conclusion

Project Overview

The LEMP stack is a widely used software stack for serving web applications and dynamic web pages. LEMP stands for:

Linux: The foundational operating system, known for its robustness and security.
Engine-X (Nginx): The high-performance web server that handles incoming requests and serves static content, acting as a reverse proxy and load balancer.
MySQL: A reliable relational database management system that stores application data in structured tables.
PHP: A server-side scripting language used for web development, enabling dynamic content through database interactions.

How LEMP Works

Nginx receives HTTP requests from clients. For dynamic pages, it forwards requests to PHP, which processes them, interacts with MySQL for data retrieval or storage, and sends results back to Nginx, which then responds to the client.

In this project, i will provision the AWS EC2 instance from the commandline. The required prerequisite for this is configuration of the AWS CLI.

Prerequisites

AWS Account
Ubuntu Linux
AWS cli

Steps

Step 0 Prepare Prerequisites

Install the AWS CLI. Follow this guide to install the AWS CLI on your local machine.
Configure AWS CLI: Run the following to configure your AWS credentials:

aws configure

You will be required to enter your AWS Access Key ID, Secret Access Key, Region and Output format.

Run : aws configure list to verify the configuration.

Choose an AMI: You can visit the console to choose an appropriate image or run the following command:

aws ec2 describe-images --owners amazon

Create a key pair (if required)

Create a key pair with the following command:

aws ec2 create-key-pair --key-name lempKey --query 'KeyMaterial' --output text > lempKey.pem

This command shows no output. The file is created in the current working directory and can be viewed with the ls command

When you check the content, it should be similar to the image below:

Set the appropriate permission:

chmod 400 lempKey.pem

Security group: If not already created, create security group with SSH and http/https access.

Run the command:

aws ec2 create-security-group --group-name <MySecurityGroup> --description "Security group to allow ssh access"

This command will show the GroupId which be used in the next step.

Run the following command to authorize ssh Access

aws ec2 authorize-security-group-ingress --group-id <GroupId> --protocol tcp --port 22 --cidr <Your-Ip/32>

Run the following to authorize http and https access:

http

aws ec2 authorize-security-group-ingress --group-id <GroupId> --protocol tcp --port 80 --cidr 0.0.0.0/0

https

aws ec2 authorize-security-group-ingress --group-id <GroupId> --protocol tcp --port 443 --cidr 0.0.0.0/0

Verify that the rules were added correctly:

aws ec2 describe-security-groups --group-ids <GroupId>

Launch an ec2 instance: Run the command below to launch an instance

aws ec2 run-instances --image-id [ami-id] --count 1 --instance-type [instance-type] --key-name [key-pair] --security-group-ids [security-group-id]

Replace [ami-id], [instance-type], [key-pair], [security-group-id] as appropriate

On the console:

Note that to create the instance with a name, include a name tag as follows:

aws ec2 run-instances --image-id <ami-id> --count 1 --instance-type <instance-type> --key-name <key-pair> --security-group-ids <security-group-id> --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=lempserver}]'

ssh into the server. Ensure you are in the directory where your private key is and run the following command. Otherwise, use the full path of the key.

ssh -i lempKey.pem ubuntu@54.164.242.184

Step 1 Install Nginx web server

Update apt packages and Install Nginx on the server

sudo apt update -y
sudo apt install nginx -y

Verify the installation of Nginx and whether it is running as a service .

sudo systemctl status nginx

Verify via curl on the terminal or on the web browser

Curl

curl http://localhost:80

web browser
Retrieve the public Ip address with the following command:

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` && curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/public-ipv4

View on browser:

http://[Ubuntu-webserver-IP]

Step 2 Install MySQL

Mysql is a popular relational database that we can use in this project. We will install it with apt.

sudo apt install mysql-server -y

Verify with:

sudo systemctl status mysql

sudo mysql

We will set a password for the root user with the following command:

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password 
BY 'Passw0rd321@'

It is important to use a strong password. We used Passw0rd321@ for simplicity.

Exit the mysql shell with the command exit

We will run a security script that comes preinstalled with Mysql to remove some insecure default settings and lock down the database system. Start the script by running:

sudo mysql_secure_installation

Test login to mysql

sudo mysql -p

Exit with the command exit

We will be asked to configure the VALIDATE PASSWORD PLUGIN.

Step 3 Install PHP

While Apache embeds the PHP interpreter in each request, Nginx requires an external program to handle PHP processing. Two packages are needed:

sudo apt install php-fpm php-mysql

fpm stands form FastCGI process manager

Verify:

ls /var/run/php/

sudo systemctl status php8.3-fpm #Replace with the appropriate version

Step 4 Configure Nginx to use PHP processor

We will set up server blocks in nginx (which is similar to virtual host in Apache) to contain the configuration details. Our domain name will be project_lemp. The default server block in nginx is found at var/www/html. We will create another directory under var/www/ to serve our domain:

First create a system user with no access to login for improved security:

sudo useradd -rs /usr/sbin/nologin project_lemp

Then create the root directory

sudo mkdir /var/www/project_lemp

This command produces no output if successful.

Assign ownership of the directory The current owner is root as seen in the image

sudo chown -R project_lemp:project_lemp /var/www/project_lemp

Open a new configuration file in the nginx sites-available directory

sudo nano /etc/nginx/sites-available/project_lemp

Enter the following configuration:

server {
    listen 80;
    server_name project_lemp www.project_lemp;
    root /var/www/project_lemp;  # Path to your project's root directory
    index index.html index.htm index.php;

    # Access and error logs
    access_log /var/log/nginx/project_lemp.access.log;
    error_log /var/log/nginx/project_lemp.error.log;

    # Handling static files
    location / {
        try_files $uri $uri/ =404;
    }

    # PHP-FPM Configuration
    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/var/run/php/php8.3-fpm.sock;  # Update to the PHP version>

    }

    # Deny access to .htaccess files
    location ~ /\.ht {
        deny all;
    }
}

The breakdown of the server block is as follows:

1. Server Configuration
   - listen 80;: Listens on port 80 (default HTTP port).
   - server_name project_lemp www.project_lemp;: Specifies the server name (domain or subdomain).
   - root /var/www/project_lemp;: Sets the document root directory for the project.

2. Index Files
   - index index.html index.htm index.php;: Defines the index files to serve when a directory is requested.

3. Logging
   - access_log /var/log/nginx/project_lemp.access.log;: Logs access requests.
   - error_log /var/log/nginx/project_lemp.error.log;: Logs errors.

4. Static File Handling
   - location / { try_files $uri $uri/ =404; }: Attempts to serve files from the requested URI; if not found, returns a 404 error.

5. PHP Configuration
   - location ~ \.php$ { ... }: Handles PHP files using PHP-FPM (FastCGI Process Manager).

include snippets/fastcgi-php.conf;: Includes PHP configuration settings.
- fastcgi_pass unix:/var/run/php/php8.3-fpm.sock;: Forwards PHP requests to PHP-FPM via a Unix socket (update the PHP version as needed).

6. Security
  - location ~ /\.ht { deny all; }: Denies access to files starting with .ht (e.g., .htaccess).

Next activate the configuration by creating a link to the config file form nginx sites-enabled directory:

sudo ln -s /etc/nginx/sites-available/project_lemp /etc/nginx/sites-enabled/

Test the configuration by running:

sudo nginx -t

The configuration is ok if it shows an image similar to the one below:

Also, disable the default nginx host configured on port 80:

sudo unlink /etc/nginx/sites-enabled/default

Reload nginx to apply the changes:

sudo systemctl reload nginx

Step 5 Test server block with html

We will create a test index.html file using aws instance metadata to confirm that our server block is working as expected:

Create an index.html in /var/www/project_lemp with the following command and assign the correct permissions:

sudo touch /var/www/project_lemp/index.html;sudo chown -R project_lemp:project_lemp /var/www/project_lemp/index.html

Then write the following content:

sudo bash c 'echo 'HELLO LEMP from hostname' $(TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` && curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/public-hostname) 'with public IP' $(TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` && curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/public-ipv4) > /var/www/project_lemp/index.html'

Set the correct permissions for the file:

sudo chmod 644 /var/www/project_lemp/index.html

Here is the image of the metadata info of our server on browser:

Note: There is a difference in IP. I had to unavoidably pause the configurations. Being that I am using a cloud IDE-gitpod, the workspace and hence, the EC2 instance had to be recreated

Step 6 Test server block with php

Create a file named info.php in /var/www/project_lemp/.

sudo nano /var/www/project_lemp/info.php

paste the following script:

<?php
phpinfo();
?>

Assign the right ownership and permissions.

sudo chown project_lemp:project_lemp /var/www/project_lemp/info.php
sudo chmod 644 /var/www/project_lemp/info.php

Visit the browser with the IP of the server referencing the info.php file.

http://[Public-ip]/info.php

This shows sensitive details about the php environment and ubuntu server as shown below:

Remove the file or edit the script to contain:

<?php
echo "PHP is working";
?>

Step 7 Retrieve data from the MySQl database with PHP

We will:

Create a test database with a simple 'To-do list'
Configure Access to the database so that nginx will query data from the db and display it.

We will create a new user with caching_sha2_password authentication method which is the default authentication introduced in MySql 8.0. Modern versions of PHP e.g PHP 7.4 and above having the associated MySQL extensions support it.

To confirm this, we will visit the php info page (Use the php info script we used previously and view it on the browser) and look for the MySQL native driver (mysqlnd) as shown:

Alternatively, you can verify the availability of the necessary driver by checking the version of php you are using and the modules installed:

Check php version

php -v

Check php modules

php -m

Connect to the MySQL console with root:

sudo mysql -p

Create the database named example_database:

CREATE DATABASE `example_database`;

Create the user named example_user using caching_sha2_password as the default authentication method with the password asPassw0rd123@

CREATE USER 'example_user'@'%' IDENTIFIED WITH caching_sha2_password BY 'Passw0rd123@';

Using the wildcard '%' means that the user can connect form any host machine and he is not retricted to logging in from the host machine lie the root user.

We will give the user permission over the example_database:

GRANT ALL PRIVILEGES ON example_database. * TO 'example_user'@'%';

(Note the period in front of the database)

This command grants this user example_user full access to only the example_user database. He doesn't have access to any other databases on the server.

Exit the MySQL console:

exit

Test that the new user is properly authenticated by logging in again:

mysql -u example_user -p

Note:
In older legacy applications using old versions of php you cn use the mysql_native_password method of authentication.

View the database and create the todo_list table:

View database with the following command:

SHOW DATABASES;

Select the database:

USE example_database;

Create a table with the following command:

CREATE TABLE todo_list (
    item_id INT AUTO_INCREMENT PRIMARY KEY,
    task VARCHAR (255) NOT NULL,
    status ENUM('pending', 'completed') DEFAULT 'pending'    
);

Insert a few rows of content to the table

INSERT INTO todo_list (task, status) VALUES
('Buy groceries', 'pending'),
('Walk the dog', 'pending'),
('Finish the report', 'pending'),
('Read a book', 'pending'),
('Prepare dinner', 'pending');

Confirm the data you entered by running:

SELECT * FROM todo_list;

Exit the MySQL console.

Create a PHP script that will connect to MySQL and query the content:

First create th PHP file:

vi /var/www/project_lemp/todo_list.php

Enter the following script that connects to the MySQL database and queries the content of the todo_list. It also displays the results in a list. If there is a problem with the database connection , it will throw an exception.

<?php
// Database configuration
$host = 'localhost'; // Change if necessary
$database = 'example_database';
$user = 'example_user';
$password = 'Passw0rd123@'; // Replace with your actual password
$table = "todo_list";

try {
    // Establish database connection
    $db = new PDO("mysql:host=$host;dbname=$database", $user, $password);

    // Set PDO to throw exceptions on error
    $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

    // Prepare and execute the query
    $query = "SELECT item_id, task, status FROM $table";
    $stmt = $db->query($query);
    $todos = $stmt->fetchAll(PDO::FETCH_ASSOC);

    // Display results
    if ($todos) {
        echo "<h1>To-Do List</h1>";
        echo "<ul>";
        foreach ($todos as $todo) {
            echo "<li>ID: {$todo['item_id']}, Task: {$todo['task']}, Status: {$todo['status']}</li>";
        }
        echo "</ul>";
    } else {
        echo "No tasks found in the to-do list.";
    }
} catch (PDOException $e) {
    // Handle connection errors
    echo "Database connection failed: " . $e->getMessage();
}
?>

The functionality of the script

This PHP script connects to a MySQL database, retrieves data from a "todo_list" table, and displays the tasks in an unordered list.

Functionality Breakdown:

1. Database Configuration: The script defines database connection parameters (host, database name, username, password, and table name).

2. Database Connection: It establishes a connection to the database using PDO (PHP Data Objects) and sets the error mode to throw exceptions.

3. Query Execution: The script executes a SELECT query to retrieve all tasks from the "todo_list" table.

4. Data Retrieval: It fetches all query results as an associative array.

5. Displaying Results: If tasks exist, it displays them in an unordered list with task ID, description, and status. Otherwise, it shows a "No tasks found" message.

6. Error Handling: The script catches and displays any PDO exceptions that occur during database connection or query execution.

Visit the browser and you will see an image similar to the one below:

http://[Publi-ip]/todo_list.php

Conclusion

In this project, we successfully deployed a LEMP stack on AWS, providing a robust and efficient environment for serving web applications. By implementing a test database and creating a dynamic PHP script, we demonstrated how to interact with the database, showcasing the full capabilities of the LEMP stack. This hands-on experience solidified our understanding of server configurations and database management in a cloud environment.

Automating User Management with Advanced Bashscript

Omolara Adeboye — Tue, 02 Jul 2024 21:49:10 +0000

As part of the requirements of my DevOps internship with HNG, I am required to create a bash script to automate user management.
Managing users in a Linux environment can be a repetitive and error-prone task if done manually, especially when dealing with a large number of users. To simplify this process, you can use a bash script to automate user creation, group assignment, and password management. Let's walk through a comprehensive bash script designed to streamline user management. This script reads a list of usernames and groups from a text file, creates users and groups accordingly, sets up home directories, generates random passwords, and logs all actions for audit purposes. You can find the complete script on my GitHub repository.

This bash script performs several key tasks:

Ensures root privileges: The script checks if it is run as root.
Logs actions: Logs important actions and errors for audit and debugging.
Validates inputs: Ensures usernames and group names are valid.
Creates users and groups: Adds new users and groups as specified in an input file.
Sets passwords: Generates and sets random passwords for new users.
Configures home directories: Sets appropriate permissions for user home directories.

Detailed Explanation of script.

Script Header and Logging The script begins by defining a logging function to record actions and errors to a log file.

#!/usr/bin/bash

log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}

Password Generation A function to generate random passwords is defined, which uses /dev/urandom for secure random number generation.

generate_password() {
    local password_length=12
    tr -dc A-Za-z0-9 </dev/urandom | head -c $password_length
}

Ensuring Root Privileges The script checks if it is run with root privileges. If not, it logs an error and exits. Root users have user id 0.

if [ "$(id -u)" -ne 0 ]; then
    echo "This script must be run as root or with sudo privileges" >&2
    log "Script not run as root or with sudo privileges"
    exit 1
fi

File Paths and Initial Checks The script sets file paths for the user input file, log file, and password storage. It also verifies the existence of these files.

USER_FILE="$1"
LOG_FILE="/var/log/user_management.log"
PASSWORD_FILE="/var/secure/user_passwords.txt"

if [ -z "$USER_FILE" ]; then
    echo "Usage: $0 <name-of-text-file>"
    log "No user file provided. Usage: $0 <name-of-text-file>"
    exit 1
fi

if [ ! -f "$USER_FILE" ]; then
    echo "Input file not found"
    log "Input file '$USER_FILE' not found"
    exit 1
fi

if [ ! -f "$LOG_FILE" ]; then
    touch "$LOG_FILE"
    chmod 0600 "$LOG_FILE"
    log "Log file created: $LOG_FILE"
fi

if [ ! -f "$PASSWORD_FILE" ]; then
    mkdir -p /var/secure
    touch "$PASSWORD_FILE"
    chmod 0600 "$PASSWORD_FILE"
    log "Password file created: $PASSWORD_FILE"
fi

Input Validation The script validates the usernames and group names using regular expressions to ensure they contain only alphanumeric characters, hyphens, and underscores.

validate_username() {
    if [[ ! "$1" =~ ^[a-zA-Z0-9_-]+$ ]]; then
        return 1
    fi
    return 0
}

validate_groups() {
    IFS=',' read -ra group_list <<< "$1"
    for group in "${group_list[@]}"; do
        if [[ ! "$group" =~ ^[a-zA-Z0-9_-]+$ ]]; then
            return 1
        fi
    done
    return 0
}

User and Group Creation The script reads the user file line by line, creating users and their groups as necessary. It also ensures that each user has a personal group and sets up the home directory with appropriate permissions.

while IFS=';' read -r username groups; do
    username=$(echo "$username" | xargs)
    groups=$(echo "$groups" | xargs)

    if [ -z "$username" ] || [ -z "$groups" ]; then
        log "Invalid line format in user file: '$username;$groups'"
        user_creation_failed=true
        continue
    fi

    if ! validate_username "$username"; then
        log "Invalid username format: '$username'"
        user_creation_failed=true
        continue
    fi

    if ! validate_groups "$groups"; then
        log "Invalid group format: '$groups'"
        group_creation_failed=true
        continue
    fi

    if id "$username" &>/dev/null; then
        log "User $username already exists."
        continue
    fi

    all_users_exist=false

    if ! getent group "$username" > /dev/null; then
        if groupadd "$username"; then
            log "Group $username created."
        else
            log "Failed to create group $username."
            group_creation_failed=true
            continue
        fi
    fi

    IFS=',' read -ra group_list <<< "$groups"
    for group in "${group_list[@]}"; do
        if ! getent group "$group" > /dev/null; then
            if groupadd "$group"; then
                log "Group $group created."
            else
                log "Failed to create group $group."
                group_creation_failed=true
            fi
        fi
    done
    unset IFS

    if useradd -m -g "$username" -G "$groups" "$username"; then
        log "User $username created and added to groups $groups"
        users_created=true
        any_users_created=true
    else
        log "Failed to create user $username"
        user_creation_failed=true
        continue
    fi

    password=$(generate_password)
    log "Generated password for $username"

    if echo "$username:$password" | chpasswd; then
        echo "$username,$password" >> "$PASSWORD_FILE"
        log "Password set for $username and stored securely"
    else
        log "Failed to set password for $username"
        password_setting_failed=true
        continue
    fi

    if chown "$username:$username" "/home/$username" && chmod 700 "/home/$username"; then
        log "Home directory for $username set up with appropriate permissions."
    else
        log "Failed to set up home directory for $username"
        home_directory_setup_failed=true
    fi

done < "$USER_FILE"

Summary and Exit After processing all lines, the script logs a summary of the actions taken and exits.

log "User creation script run completed."

if [ "$any_users_created" = true ]; then
    echo "$(date '+%Y-%m-%d %H:%M:%S') - User creation script completed successfully."
elif [ "$all_users_exist" = true ]; then
    echo "Users already exist. Nothing left to do"
else
    echo "$(date '+%Y-%m-%d %H:%M:%S') - No users were created successfully. Check log file."
    log "No users were created successfully. Please check the input file format: username;group1,group2,group3."
fi

[ "$user_creation_failed" = true ] && echo "Users creation incomplete." && log "Some users were not created due to errors. Check file format"
[ "$password_setting_failed" = true ] && echo "Users' passwords creation incomplete." && log "Some users' passwords were not set due to errors. Check file format"
[ "$group_creation_failed" = true ] && echo "Groups creation incomplete." && log "Some groups were not created due to errors. Check file format"
[ "$home_directory_setup_failed" = true ] && echo "Home directories creation incomplete." && log "Some home directories were not set up due to errors."

exit 0

Conclusion
This script offers a reliable way to automate processes related to user management, lowering the possibility of mistakes and saving important administrative time. System administrators can create user accounts in a consistent and safe manner with this script, which also manages passwords and group assignments. Detailed logs are kept for accountability and debugging purposes. To get real-time internship experience, sign up with HNG

Understanding Placement Groups: Optimize Your EC2 Instance Deployment

Omolara Adeboye — Thu, 04 Jan 2024 22:49:02 +0000

Placement groups help you to organise your instances. They help to control where your instances are placed in a group of physical servers within an Availability zone in the AWS cloud. There are three types of placement groups:

Cluster
Partition
Spread

Cluster

In the cluster arrangement, instances are packed close together in an Availability zone. This has the advantage of reducing latency because the instances are placed on the same hardware rack in the same AZ. It is recommended for applications that require low latency and high network throughput. The disadvantage of this arrangement is that if an hardware failure occurs, all the instances may be affected as a result of reduced hardware fault tolerance for instances within the group. One common error with launching instances in a cluster is insufficient capacity error. The solution to this is to stop and start all of the instances in the placement group and retry the launch. It is recommended to use a single launch request to launch the number of instances that you need in the placement group. A single launch request means launching the instances in a single API call (launch at once). Also, using the same instance type for all the instances will help to prevent this error.

To create a cluster placement group from the AWS CLI use the following command:

aws ec2 create-placement-group --group-name  --strategy cluster

Partition

In partition placement group, groups of instances are placed in different logical isolations called partitions. Each partition has its own hardware. Instances within a partition share the same hardware but they do not share hardware with groups of instances in another partition. This allows for isolation and minimises hardware failure within your application. It is beneficial for launching large distributed workloads and replicated workloads e.g HDFS, HBase and Cassandra. Partitions can be created accross multiple AZs in the same region.

To create a partition placement group named Phoenix-APP with six partitions from the AWS CLI use the following command:

aws ec2 create-placement-group --group-name Phoenix-APP --strategy partition --partition-count 6

Spread

In spread placement group, EC2 instances are placed accross distinct underlying hardware partitions to reduce the risk of failure by providing isolation. It is especially beneficial for critical applications. Instances can be placed accross racks or hosts. Racks can span accross multiple AZs in the same region. Hosts are applicable to AWS Outpost. In racks, a maximum of 7 instances per availability zone is permitted.

To create a cluster placement group from the AWS CLI use the following command:

aws ec2 create-placement-group --group-name  --strategy spread

Reference:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placement-groups.html?icmpid=docs_ec2

Forem: Omolara Adeboye

Deploying A MERN Stack To-Do App on AWS: A Step-by-Step Guide

Table of Contents

Project Overview

Prerequisites

Steps

Step 0 Prepare Prerequisites

Step 1 Configure the backend

Set up application code

Step 2 Test the Backend code with Postman

Step 3 Create the frontend

Step 4 Create the React Components

Step 5 Run the app

Conclusion

LEMP STACK ON AWS

Table of Content

Project Overview

Steps

Step 0 Prepare Prerequisites

Step 1 Install Nginx web server

Step 2 Install MySQL

Step 3 Install PHP

Step 4 Configure Nginx to use PHP processor

Step 5 Test server block with html

Step 6 Test server block with php

Step 7 Retrieve data from the MySQl database with PHP

Conclusion

Automating User Management with Advanced Bashscript

Detailed Explanation of script.

Understanding Placement Groups: Optimize Your EC2 Instance Deployment

Cluster

Partition

Spread