Forem: Vijeth Simha

Serving Dynamic Images in Node.js with Typescript : A Simple Guide

Vijeth Simha — Thu, 01 Feb 2024 10:33:35 +0000

While working on a Node.js project with TypeScript, I encountered a fascinating challenge regarding serving static and dynamic image assets. Everything functioned flawlessly in the development environment, with static and dynamic assets residing in the public folder. However, upon deploying to production, an unexpected issue arose.

The Issue

The development setup seamlessly served static and dynamic images from the public folder. The complication emerged during the deployment process to the production environment. Here's what happened:

As part of the deployment, the TypeScript codebase was transpiled to JavaScript. This step also involved copying all static assets from the public folder to a designated production folder.
The static images, being part of the pre-defined assets, were successfully transferred to the production environment. They continued to load without any issues, just as they did in the development environment.
The real problem surfaced with dynamic assets. Despite the initial setup working in development, the dynamic images failed to display in production. This was perplexing, as the deployment process didn't inherently account for assets created post-build — those dynamic images generated on the fly and intended to be served from the same public folder.

Root Cause

The crux of the issue lies in the deployment strategy for the TypeScript application. The build process, designed to prepare the production application, inadvertently overlooked the nature of dynamically generated assets. These assets, unlike their static counterparts, weren't part of the initial asset copying to the production folder. Consequently, when new dynamic images were created, they were not accessible in the structured production environment, leading to the observed issue of missing images.

The above problem could have been easily addressed by saving the images in external storage like S3 or Firebase storage but I wanted to solve it in the existing architecture itself.

So this is how is solved.

Solution

To address the challenge of serving dynamic assets in production, I implemented a strategy that ensures both static and dynamic images are accessible, regardless of the environment. Here's how I tackled the issue:
The first step was recognizing that the deployment process did not accommodate dynamically generated images. While static assets were copied to the production directory during the build process, dynamic assets created at runtime were not being served from the expected directory.
To ensure dynamic assets were accessible, I decided to serve them from a directory outside the build output (dist). This approach prevents dynamic assets from being overwritten or missed during the build process.
I configured the Express server to serve static files from two directories: one for the pre-existing static assets and another for dynamic assets. Here's an example of how I set it up in Express:

import express from 'express';
import path from 'path';

const app = express();

var env = process.env.NODE_ENV || 'development';

if(env === "development") {
  // In development, serve static files from 'public'
  app.use('/static', express.static(path.join(__dirname, "public")));
} else {
  // In production, serve dynamic files from 'dynamic', located outside of 'dist'
  app.use('/dynamic', express.static(path.join(__dirname, '..', "dynamic")));
}

Key Aspects of the Solution:

Environment-Aware Configuration: By leveraging the NODE_ENV environment variable, the server dynamically adjusts its behavior based on the current environment—development or production. This ensures that the correct set of assets is served in each context.

In the development environment, the server serves static assets from the public directory. This setup is straightforward and aligns with the typical Express.js configuration for handling static files.
I needed a way to access the dynamic directory, which resides outside dist. By employing path.join(__dirname, '..', 'dynamic'), I effectively directed the server to step out of the dist directory and into the dynamic directory to serve the runtime-generated assets.
Directory Navigation with '..': The '..' in the path plays a crucial role, instructing the server to navigate up one directory level from its current location (dist) and then into the dynamic directory. This ensures that dynamic assets generated post-deployment are accessible, without being constrained by the build output directory structure.

Finally

As you embark on this journey, feel free to use my project as a practical example to follow along with the steps outlined above.

Should you encounter any challenges or have questions during the process, please don't hesitate to leave a comment below.

Feel free to reach out to me on

The Essential Guide to Domain Setup and SSL Security

Vijeth Simha — Fri, 19 Jan 2024 08:59:42 +0000

Thank you for taking the time to read this post.

In my last post, I discussed automating the deployment process using GitHub CI/CD. Building on that topic, this post will delve into how to connect your domain and attach SSL.

Before proceeding, I highly recommend checking the previous articles in the deployment series, as this post is a continuation of the previous two.

With that said, let's dive in.

Table of contents

Attach Domain
Attach SSL

Attach Domain

The first step is to edit the existing Nginx configuration file. You can do this by opening the file in a text editor, such as nano, with the following command

sudo nano /etc/nginx/sites-available/<<YOUR_PROJECT_NAME>>

Note: For the following step to work, you must point your domain to the DigitalOcean IP address, or else the domain connection won't work.

Add your domain to the configuration as shown below.

server {
    listen 80;
    server_name <<your_domain_name>>;

    location / {
        proxy_pass http://localhost:<<YOUR_NODE_JS_PORT>>;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Replace <<your_domain_name>> with your domain name and <<YOUR_NODE_JS_PORT>> with the port number where your Node.js app is running.

You can test the configuration by running below command.

sudo nginx -t

If you encounter an error, reopen the server block file and carefully review it for any typos or missing characters. After ensuring that your configuration file's syntax is correct, you should reload Nginx to apply the new configuration by running below command

sudo systemctl reload nginx

Attach SSL

The first step in using Let’s Encrypt to obtain an SSL certificate is to install the Certbot software on your server. Execute the following command to do so.

sudo apt install certbot python3-certbot-nginx

If you have a firewall enabled, you'll need to adjust its settings to allow HTTPS traffic. Fortunately, Nginx registers several profiles with UFW (Uncomplicated Firewall) upon installation. You can check the current firewall status with

sudo ufw status

To allow HTTPS traffic, enable the 'Nginx Full' profile, which covers both HTTP and HTTPS. This step makes the separate 'Nginx HTTP' profile redundant, so you can remove it

sudo ufw allow 'Nginx Full'
sudo ufw delete allow 'Nginx HTTP'

After these changes, verify the updated firewall status by running

sudo ufw status

The status should now show as 'active', similar to the example in the image below.

Certbot offers several methods to obtain SSL certificates, facilitated by its plugins. The Nginx plugin, in particular, is designed to automatically reconfigure Nginx and reload its configuration as needed. To utilize this plugin, enter the following command

sudo certbot --nginx -d <<your_domain_name>>

Ensure you replace <<your_domain_name>> with your actual domain name.

When you run this command, Certbot will prompt you for several pieces of information:

Email Address: For important notifications related to your SSL certificate.
Terms and Conditions Acceptance: You'll need to agree to the Let's Encrypt terms of service.
Redirection Details: Certbot will ask whether you want to redirect HTTP traffic to HTTPS, effectively enforcing a secure connection.

Let’s Encrypt's certificates have a validity period of ninety days. This relatively short duration is intentional, aimed at encouraging users to automate the process of certificate renewal.

Fortunately, the certbot package, which we've already installed, simplifies this process. It adds a systemd timer to your system, configured to run twice daily. This timer automatically renews any certificate that's within thirty days of expiration.

You can query the status of the timer with systemctl

sudo systemctl status certbot.timer

You will get output as the image below.

To test the renewal process without making any actual changes, you can perform a dry run with certbot using the following command

sudo certbot renew --dry-run

If the dry run completes without any errors, you're all set. Certbot will handle the renewal of your certificates and will ensure that Nginx is reloaded to apply the changes.

In the event that the automated renewal process encounters an issue, Let’s Encrypt will send a notification to the email address you provided. This notification will serve as a warning, alerting you when your certificate is nearing its expiration date.

That's it for this guide.

Thank you so much for reading through to the end!.

Finally

As you embark on this journey, feel free to use my project as a practical example to follow along with the steps outlined above.

Should you encounter any challenges or have questions during the process, please don't hesitate to leave a comment below.

Feel free to reach out to me on

Node.js with TypeScript: The Ultimate Guide to Automated Deployment on DigitalOcean Using GitHub Actions

Vijeth Simha — Fri, 12 Jan 2024 20:12:28 +0000

Thank you for taking the time to read this post.

In my last post, I discussed deploying code to DigitalOcean. Building on that, this post delves into automating the deployment process using GitHub Actions.

We'll explore the steps and benefits of this method, ensuring a seamless and efficient deployment experience.

Table of contents

Create Github secret keys
Automate the deployment

Create Github secret keys

To begin, we need to generate secret keys that will securely store our DigitalOcean credentials.

First, log in to your GitHub account. Once logged in, navigate to your repository and look for the settings icon. This is typically found in the upper right corner, represented by a gear symbol. Click on this icon to access your repository settings.

Then, navigate to the 'Security' section. Here, you'll find the 'Secrets and Variables' option. Click on this to proceed. Next, select 'Actions' to access the area where you can manage your secret keys. This is illustrated in the image below.

It's essential to create secrets for SSH_PASSWORD, SSH_USER, and DROPLET_IP. These represent the SSH login password, username, and IP address of your DigitalOcean droplet, respectively.

To add a new secret, click on the 'New Repository Secret' button. This can be found at the top right of the 'Secrets and Variables' section, as shown in the first image below.

Next, you'll need to enter the necessary credentials. In the fields provided, add the respective values for each secret key. This step is illustrated in the second image below.

Automate the deployment

To begin automating deployment, start by creating a .github/workflows folder in your project's root directory.

Next, create a YAML file within this folder. You can name it as you prefer, such as deployment.yml.

Follow the below steps.

The first step in configuring the pipeline is to name it, as shown below

name: CI/CD

The next step is to trigger the CI/CD Pipeline by:

on:
  push:
    branches:
      - main

on: push specifies that the instructions will execute whenever code is pushed to the repository.
branches: - main indicates that these actions will trigger only for pushes to the 'main' branch.

The next step is to define the Job Definition

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

build-and-deploy: This is the name of the job. It's a set of tasks that will be executed.
runs-on: ubuntu-latest: This line ensures the job runs on the latest Ubuntu version, a widely-used operating system for server environments.

Following this, we define the individual steps within the job.

 steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Install dependencies
        run: npm install

      - name: Build
        run: npm run build

      - name: Deploy
        run: |
          sshpass -p ${{ secrets.SSH_PASSWORD }} ssh -o StrictHostKeyChecking=no ${{ secrets.SSH_USER }}@${{ secrets.DROPLET_IP }} << EOF
            cd <<YOUR_CODE_REPOSITORY_NAME>>/
            git pull origin main
            npm install
            npm run build
            pm2 reload all
          EOF

The above section involves four steps.

Checkout Code
- uses: actions/checkout@v2: This is a GitHub Action to check out the code from the repository, making it available to subsequent steps in the job.
Install Dependencies
- run: npm install: This command installs all the dependencies required for the project, as defined in a file usually called package.json.
Build
- run: npm run build: This command builds the project. Depending on how it's set up, this might involve converting TypeScript code to JavaScript code, minifying files, etc.
Deploy
- This is a multi-line script that automates the deployment of the code to a remote server.
- It uses sshpass to log in to the remote server using SSH without needing to enter a password manually. The password and other details are stored in GitHub secrets for security.
- The script then changes the directory to <<YOUR_CODE_REPOSITORY_NAME>>, pulls the latest changes from the master branch, installs any new dependencies, builds the project on the server, and finally, reloads all processes managed by pm2, which is a process manager for Node.js applications.

If you're already familiar with the process or prefer to skip the detailed explanations, you can simply copy the code snippet provided below and paste it into your deployment.yml file.

name: CI/CD

on:
  push:
    branches:
      - main

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Install dependencies
        run: npm install

      - name: Build
        run: npm run build

      - name: Deploy
        run: |
          sshpass -p ${{ secrets.SSH_PASSWORD }} ssh -o StrictHostKeyChecking=no ${{ secrets.SSH_USER }}@${{ secrets.DROPLET_IP }} << EOF
            cd <<YOUR_CODE_REPOSITORY_NAME>>/
            git pull origin main
            npm install
            npm run build
            pm2 reload all
          EOF

This is it now push the code to your main branch.

This completes the setup! Now, go ahead and push your code to the main branch. If everything is configured correctly, you should see a green checkmark indicating a successful deployment, as illustrated in the image below.

This signifies that your automated deployment pipeline is functioning as expected.

Finally

As you embark on this journey to automate your deployment to DigitalOcean feel free to use my project as a practical example to follow along with the steps outlined above.

Should you encounter any challenges or have questions during the process, please don't hesitate to leave a comment below.

Feel free to reach out to me on

From Beginner to Pro: Deploying TypeScript Applications with Node.js & PostgreSQL on DigitalOcean

Vijeth Simha — Sun, 31 Dec 2023 19:33:00 +0000

When I first started with NodeJS there were tons of resources to deploy NodeJs with on Digital Ocean droplet with managed database solutions.

However, I found a gap in resources when it came to hosting both the database and the Node.Js application on the same droplet.

I could have moved ahead with a managed database solution but I didn't want to pay just for the POC:)

Although there is Digital Ocean documentation for hosting PostgreSQL, it didn't work for my needs, possibly due to outdated information.

Let's dive straight into the topic.

This guide could serve as a reference therefore a table of contents is necessary.

Table of contents

Creating a DigitalOcean Droplet
Deploying Node.js
Deploying PostgreSQL
Verifying the Database Connection from Ubuntu Root Folder(optional)
Verifying PostgreSQL Connection with Node.js
NGINX

Creating a DigitalOcean Droplet

The first step is to create your account on DigitalOcean.

After signing up or logging in, you will be directed to the droplets page on DigitalOcean.

Here, you can create a new droplet by clicking the 'Create' button, as illustrated in the image below.

Next, select your preferred region and data center as shown in the image below.

Choose the OS image as 'Ubuntu' as we are using the Ubuntu server throughout this article as depicted in the image below.

Choose 'Password' as the authentication method as it helps you start quickly as depicted in the image below.

Select the Droplet type and CPU options as regular as depicted in the image below.

In the final step, set the hostname for your droplet as depicted in the image below.

Finally. click on the 'Create Droplet' button. Voila!, your droplet is ready.

Deploying Node.js

ssh root@your_droplet_ip

The above command initiates a secure connection to your Ubuntu server.

Once you have logged in, it's good practice to update your system's package index. Enter the command below

sudo apt update && sudo apt upgrade

The sudo apt update command refreshes the list of available packages and their versions, but it doesn't install or upgrade any packages.

The sudo apt upgrade command is used to install the newest versions of all packages currently installed on the system.

The next step is to install Node.js and npm(Node Package Manager) on your Ubuntu server

sudo apt install nodejs && sudo apt install npm

the above command installs Node.js, the JavaScript runtime, and npm, which is used to manage Node.js packages.

Next, clone your GitHub repository by using the command below.

git clone <<YOUR_GITHUB_REPO_LINK>>

Replace <<YOUR_GITHUB_REPO_LINK>> with the actual link to your GitHub repository.

After cloning, navigate to your project repository by

cd into <<YOUR_PROJECT_NAME>>

Replace <<YOUR_PROJECT_NAME>> with the name of your project directory.

Now, install your project dependencies.

npm install

Next, install the production process manager for Node.js applications, using the command below

npm install pm2 -g

PM2 helps in managing and keeping your application online. I'll explain more about the importance of the PM2 package shortly.

As we are talking about the TypeScript. We need to compile the TypeScript code into JavaScript. Hence we need to run the command below.

npm run build

This process sets up the Node.js server to the DigitalOcean droplet.

Deploying PostgreSQL

Now, let's deploy PostgreSQL on the same droplet.

First, install postgresql and postgresql-contrib by the command below.

sudo apt install postgresql postgresql-contrib

postgresql-contrib package includes additional utilities and functionality that are not part of the standard PostgreSQL distribution. You can check here for more info.

Next, switch from the current user context to the postgres context by running the command below.

sudo -i -u postgres

Note: -i flag is very important without it certain environment variables, including those set in the login shell's startup scripts will not be reset or executed. In short, we can't create new users or a new database.

P.S. I invested a lot of time at the start without adding -i flag.

Once you execute the above command you will see like below.

postgres@<<YOUR_SSHNAME>>:~$

Now, connect to the PostgreSQL command line interface(CLI) by running the command below.

psql

Upon successful connection, you'll be presented with the PostgreSQL CLI, as shown in the accompanying image.

Caution: When setting up your PostgreSQL database, it's crucial to ensure consistency between your local environment and production settings. Specifically, the database name and password you use while creating your PostgreSQL database on the DigitalOcean droplet should match those specified in your Node.js application's configuration files. This alignment is essential for successful database connection and operation, both in development and production environments.

If there's a mismatch in these credentials, your Node.js application might encounter connection issues, leading to potential downtime or functionality problems. Always double-check these details to ensure a smooth deployment process.

Next, we need to create the user role in the database. Execute the following command, replacing <<USERNAME_YOU_LIKE>> with your desired username and <<PASSWORD_YOU_LIKE>> with a secure password

CREATE USER <<USERNAME_YOU_LIKE>> WITH PASSWORD '<<PASSWORD_YOU_LIKE>>';

Note: It's crucial to enclose the password within single quotes ' '

After setting up the user, the next step is to create a database. Use this command, replacing <> with your chosen database name and <> with the username you created in the previous step.

CREATE DATABASE <<DATABASE_NAME>> OWNER '<<USERNAME>>';

Note: Ensure that <<USERNAME>> matches the username you've just created.

To verify your database has been created successfully, run the command below.

\l

You should be able to see your new database on the list.

To quit the PostgreSQL command line interface. run the command below.

\q

Then, to leave the Postgres user context and return to the Ubuntu root folder, type

exit

And there you have it! We've successfully set up a user and a database in PostgreSQL on our Ubuntu server.

Uff.. It's been quite a journey, but we're making great progress.

Verifying the Database Connection from Ubuntu Root Folder(optional)

Now let's verify if we can connect to the newly created database from the Ubuntu server.

sudo adduser <<USERNAME>>

Note: This command will prompt you to create a new password for the system user and fill in some additional details. You can opt to keep these details at their default values.

To open the PostgreSQL shell as the <<USERNAME>>, run the below command.

sudo -u <<USERNAME>> psql -d <<DATABASE_NAME>>

Once logged in, check the connection details with

\conninfo

If everything is set up correctly, you will see a message similar to:
You are connected to database <<DATABASE_NAME>> as user <<USERNAME>> via socket in "/var/run/postgresql" at port "5432".

This confirmation indicates a successful connection to your PostgreSQL database from the Ubuntu server.

Verifying PostgreSQL Connection with Node.js ###

Now we'll verify the database that we set up is successfully connected to the Node.js server.

Navigate to your project repository by

cd into <<YOUR_PROJECT_NAME>>

Once in your project directory, start your Node.js application to check the database connection

npm start

If the database connection is properly configured in your application, you should not encounter any error messages related to the database. Successful execution without errors indicates that your Node.js server is correctly connected to your PostgreSQL database.

Now, exit the project by pressing ctrl+c and navigate to Ubuntu root server by running the command below.

cd

NGINX

Now we have come to the last bit, exposing the project to the internet.

We will use the internet's favorite(also my personal favorite) NGINX

You could ask me why NGINX.

Here are my top 3 reasons.

Nginx efficiently handles static files and a large number of simultaneous connections, offloading this work from Node.js.
Nginx acts as a security layer, shielding your Node.js application from direct exposure to the internet.
Nginx can manage SSL/TLS encryption, simplifying the setup of HTTPS for your Node.js app.

With that said let's start.

Install NGINX by running the command below.

sudo apt install nginx

Next, we'll create a configuration file for your project in the /etc/nginx/sites-available/ directory. This file is crucial for directing internet traffic to your Node.js application

sudo nano /etc/nginx/sites-available/<<YOUR_PROJECT_NAME>>

You will see the empty editor like the image below.

Now let us add the below code.

server {
    listen 80;
    server_name <<your_domain_or_IP>>;

    location / {
        proxy_pass http://localhost:YOUR_NODE_JS_PORT;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Replace <> with your domain name or IP address and YOUR_NODE_JS_PORT with the port number where your Node.js app is running.

This configuration does the following

Listens on port 80, the default port for HTTP.
server_name specifies the domain name or IP address NGINX responds to.
proxy_pass directs requests to your Node.js app.
Additional proxy_set_header settings ensure proper handling of connections and upgrades (important for WebSocket support).

After adding the code save the file by pressing Ctrl+O, then press Enter, and exit the editor with Ctrl+X.

Next test the NGINX configuration.

sudo nginx -t

If the test is successful, create a symbolic link to enable your site configuration

sudo ln -s /etc/nginx/sites-available/<<YOUR_PROJECT_NAME>>  
 /etc/nginx/sites-enabled/

Restart the NGINX server to apply the changes

sudo systemctl restart nginx

Navigate to the project root by running the command below.

cd <<PROJECT_NAME>>

Finally, start your Node.js server permanently with PM2:

pm2 start dist/index.js

PM2 not only helps keep your Node.js application running efficiently but also provides a suite of features beneficial for application monitoring, logging, and performance optimization, especially in a production environment.

If you're interested in further streamlining your workflow, I have good news for you! In my next post, I delve into the intricacies of fully automating this deployment process. Check out my upcoming post

Finally

As you embark on this journey of deploying your Node.js application with PostgreSQL on DigitalOcean and configuring NGINX, feel free to use my project as a practical example to follow along with the steps outlined above.

Should you encounter any challenges or have questions during the process, please don't hesitate to leave a comment below.

Feel free to reach out to me on