Forem: Mike Siegiel

Server side code highlighting in Node

Mike Siegiel — Fri, 27 Dec 2019 20:56:04 +0000

What we are building

A simple server side Node / Express code to transform Markdown content into a fully formatted HTML with highlighted code blocks.

Technologies we use

Markdown is a lightweight markup language with plain text formatting syntax. Its design allows it to be converted to many output formats.

Highlight.js is a syntax highlighter written in JavaScript. It works in the browser as well as on the server. It works with pretty much any markup, doesn’t depend on any framework, and has automatic language detection.

Unified is a friendly interface backed by an ecosystem of plugins built for creating and manipulating content.

Unified plugins: remark-parse, remark-rehype, rehype-stringify, rehype-highlight

Introduction - why use server side code highlighting

At Regbrain we decided to implement server side code highlighting to boost loading time of our main website. We constantly benchmark our website with Lighthouse and aim for top performance scores.

Loading JavaScript to highlight code in the browser was taking too much time. First, the JavaScript files had to be fetched and then the browser was repainting the content resulting in a slower website. To improve speed, we decided to implement code highlighting on a server and now we send fully formatted text to the browser.

At this point, you may be wondering, how is highlighting code server side performant? We will explore that in more details later on, but first, let's walk through our technical solution.

Server side code highlighting

Our articles are written in markdown so our workflow needs to take raw markdown as input and serve a fully formatted html. We do it in the following steps:

Fetch markdown content
Transform markdown into a markdown syntax tree using remark-parse
Transform markdown syntax tree to html syntax tree using remark-rehype
Traverse html syntax tree to apply code highlighting to content within <code> tags using rehype-highlight
Transform html syntax tree to string to send to the client using rehype-stringify

We achieve all the above with unified framework and plugins as follows:

Import required libraries

We grab the unified framework and the required plugins

let unified = require('unified')
let markdown = require('remark-parse')
let remark2rehype = require('remark-rehype')
let highlight = require('rehype-highlight')
let html = require('rehype-stringify')

Create a unified processor

We create a processor which pipes together all the plugins above to achieve our chain of transformations from markdown to fully highlighted html:

  let processor = unified()
    // Transform markdown into a markdown syntax tree
    .use(markdown)
    // Transform markdown syntax tree to html syntax tree
    .use(remark2rehype)
    // Traverse html syntax tree to apply code highlighting to content within code tags
    .use(highlight)
    // Transform html syntax tree to string to send to the client
    .use(html)

Transform!

We now have the processor which can parse any markdown input as follows:

let input = some markdown content
let output = await processor.process(input)

Express js router implementation example

We implement the above steps in our Express app as follows:

let express = require('express')
let router = express.Router()

let unified = require('unified')
let markdown = require('remark-parse')
let remark2rehype = require('remark-rehype')
let html = require('rehype-stringify')
let highlight = require('rehype-highlight')

router.get('/:slug', async function (req, res, next) {
  let input = await article.from.database.in.markdown()

  let processor = unified()
    .use(markdown)
    .use(remark2rehype)
    .use(highlight)
    .use(html)

  let output = await processor.process(input)

  res.render('article', output)
})

module.exports = router

Don't forget the CSS

The last thing we need to do is to include highlight css styles on our pages. The easiest way would be to simply link them as external styles, but that would impair our website loading speed as fetching external styles blocks page rendering. To avoid the performance penalty, we include all css as an internal style on a page.

<!doctype html>
<html>
  <head>
    <style>
      {{all page's style including highlightjs css}}
    </style>
  </head>
  <body>
  </body>
</html>

The performance question revisited

How do we make server side rendering performant? Even though the above code highlighting slows down our server a little bit compared to sending 'clean' html, we implement a number of additional layers below which allow us achieve excellent page loading speed:

AMP - we serve our main content as AMP pages by default. That means that Google and Bing can cache our pages and serve it really fast on mobile devices.

No external styles or JavaScript (other than async AMP) - we do not use any blocking external resources such as styles, images or JavaScript files. This is already enforced by following the AMP specification, but even if we did not implement AMP, this would a good approach to take to improve page load speed. All our css is internal. We prepare css server side and make it specific to the type of content that we serve to avoid including unused styles (...within reason...).

Minification - we use css and html minification to further reduce the size of our pages.

CDN - we use a global content distribution network and configure our HTTP headers to get benefits of CDN caching, we also configure asset compression for our CDN.

With the set up above we can serve even ten Express apps on the smallest AWS EC2 instance, which works out to be really cost attractive compared to various options of hosting individual apps separately as a service.

Node app with Nginx on Amazon EC2

Mike Siegiel — Sun, 15 Dec 2019 15:58:31 +0000

How to launch a Node js app behind an Nginx proxy on Amazon EC2 instance in a step by step walkthrough.

Two things before we start

You need a few things before we start the launch:

1) Node js app in a git repo - make sure your Node app is ready and available to clone from a git repo

2) AWS account - if you don't have one yet, you can sign up here

That's it, we are ready to roll!

Launch an EC2 instance

1) Sign in to your AWS management console

2) Go to EC2 service

3) Go to Launch instance

4) Select the first one from the list: Amazon Linux 2 AMI for the 64-bit (x86) - basically the first default option

5) Select instance type. Assuming you are running a simple app then go for the cheapest instance: t3a.nano. Click Review and launch as we are blasting through the complexity here and will take all the default setting and only change the few we need later. On the next screen confirm again by clicking Launch

6) You will be asked to select an existing key pair or create a new key pair - it doesn't matter, we will be using a web based SSH client from AWS console to select any option. Click Launch Instances

7) Click View Instances and go make yourself a cup of coffee while AWS is launching your new instance. It will take a few minutes.

Set up security groups

As your instance is being launched, you need to configure AWS security groups so that you and your website visitors will be able to access your app (and you will be able to connect to it via SSH).

1) On EC2 Dashboard, find your new instance and scroll the view to the right to see what Security Group your instance is in. Remember the name of the security group

2) In the left hand side menu find and go to Security Groups

3) Select the security group and from 'Actions' select Edit inbound rules

4) In inbound rules you want to add the following two rules:

Type: SSH, Source: Anywhere
Type: HTTP Source: Anywhere Click Save

5) Go back to Instances

Connect to your instance via SSH

Ok, so you got your coffee and you can see your new instance status is 'running' and the status checks are green on the AWS EC2 Dashboard? Let's go then and connect to the instance via SSH:

1) From the AWS EC2 Instances dashboard, select your instance, press Connect

2) From connection options choose EC2 Instance Connect (browser-based SHH connection) - this is the easiest and fastest option with no config required!

Install Node, git, yarn and PM2 to run your app

Now the fun begins, we will blast through the installation of all the software you need. You will now be working in your SSH terminal

1) Update the system with the latest packages and basic environment

sudo yum update

2) Install nvm in order to install Node in the next step:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.1/install.sh | bash

3) Close and re-open your SSH console for the change to take effect

4) Install Node

nvm install node

5) Install Git

sudo yum install git

6) Install yarn. You are wondering why not an npm? Here is an article why you should use yarn. Bottom line, yarn is quicker and more resilient.

curl -o- -L https://yarnpkg.com/install.sh | bash

7) Re-open the terminal window for changes to take effect

8) Clone your Node app from your git repo

git clone https://github.com/<username>/<repository-name>.git <folder-name>
-- or --
git clone https://<username>@bitbucket.org/<username>/<repository-name>.git

9) Check if it's there

ls

10) Do you see a new folder with your app? Great, go to your app folder

cd <your-app-folder>

11) Install your app packages

yarn install

12) Install Node process manager to run your node app as a service.

You need a Node process manager so that it takes care of automatically restarting and reloading your app when something goes wrong. We will use PM2 - a very popular and production ready process manager.

yarn global add pm2

13) Launch your node app with the PM2 process manager. The example below assumes your app starts via index.js. Replace index.js with another file, e.g. app.js or server.js depending on how you would normally start your app in Node.

pm2 start index.js --name my-app

14) Make PM2 automatically restart your app if something goes wrong

pm2 startup

Retype the command the console asks you for in order to create the start up configuration. You need to re-type it as copy-paste does not really work in the console!

It is a bit painful but make sure you get this one right as otherwise PM2 will not re-start.

Now save the PM2 set up:

pm2 save

Set up Nginx to run your app behind the proxy

1) Install Nginx

sudo yum install nginx

You will get a warning command to install AWS curated Nginx package - great, that's what we need!

sudo amazon-linux-extras install nginx1.12

2) Edit Nginx config to redirect HTTP traffic from port :80 to the port your app is running on the local host.

In the example below I assumed your app is running on port 3000. If your app is running on a different port make sure to reflect that in the line of code proxy_pass http://127.0.0.1:3000; in the configuration below:

Open the editor:

sudo nano /etc/nginx/nginx.conf

and edit the config file to contain the following server block (leave everything else as is):

server {
   listen         80 default_server;
   listen         [::]:80 default_server;
   server_name    localhost;
   root           /usr/share/nginx/html;
   location / {
       proxy_pass http://127.0.0.1:3000;
   }
}

Just in case you are new to the nano editor - you can press Ctr + X to finish editing and you will be prompt whether to save the file.

3) Restart Nginx

sudo service nginx restart

4) Set up Nginx to restart automatically if something goes wrong

sudo chkconfig nginx on

You are done! Up and running.

Go to your AWS dashboard with EC2 instances and find the Public DNS (IPv4) for your new instance. If you copy the url into the browser you should be able to see the output from your Node app.

Next steps

1) You still need to think how to set up a proper domain for your app

I like to use AWS Route 53 and Cloudfront CDN to forward the traffic into the EC2 instance. That gives me resilience, free SSLs certificates and HTTPs redirects without having to worry about further Nginx configurations and management of certificate on the EC2 instance.

2) You need to configure your Nginx proxy headers and learn more about the server and location blocks configurations.

3) You probably want to set up a Git repo with your Nginx configuration files so that you can edit them in a proper code editor and than just pull their latest versions into your EC2 instance

4) Use a proper SSH client like PuTTY from your local machine. It will be a bit faster and smoother experience than the web client via AWS Console

5) This about the caching strategy for your app