Express+React Monorepo Setup with Lerna

Benoît Hubert — Sat, 31 Aug 2019 20:44:58 +0000

Initially published on my blog

Changelog

2019-08-31: added a 5th step (backend-frontend connection, serve React build from Express)

Motivation

Setting up a basic full-stack JavaScript application is not that hard by itself, but becomes complicated and tedious as you throw in more requirements, such as performing linting and testing before allowing commits.

I've been investigating ways to do it properly, out of personal interest, and with the aim of teaching good practices to my students. Enforcing strict coding conventions tends to annoy them at first, but since we do it at an early stage of their training, it quickly becomes natural for them to follow good practices.

In this post, we'll describe how to set up an Express + React application repository. First, let's describe our requirements.

Requirements

We'll setup a monorepo, using Lerna. As the name implies, in a monorepo, you keep all your app's "components" in a single repository. Lerna refers to these components as "packages". Among other things, it allows you to run npm scripts in all the packages with a single command, for tasks such as:

starting your app (npm start),
running tests (npm test),
or any custom script

In order to improve code quality, and prevent anyone from pushing broken code to GitHub, we'll setup Git pre-commit hooks: Git hooks allow you to automatically perform tasks on specific Git events (pre-commit, pre-push, etc.). We'll set them up using Husky, in order to perform these tasks on pre-commit events:

Linting with ESLint (Airbnb coding style)
Testing with Jest

Additionally, we'll set up the backend package in order to use ES6 modules, and use Yarn for dependency management.

Steps

We'll break down the following into 5 major steps:

Repo initialization and Lerna setup
Frontend app setup, with ESLint/Airbnb config
Backend app setup, with ESLint/Airbnb config
Git pre-commit hooks setup with Husky
Connect frontend and backend apps

Repository initialization

This part is quite straightforward.

Install Yarn globally if it's not already done: npm i -g yarn
Create an empty directory and cd into it
Initialize a Git repo: git init
Initialize root-level package.json: yarn init --yes (modify version to 0.0.1 afterwards)
Install Lerna and Husky as a dev dependency, at repo root level: yarn add --dev lerna
Create Lerna config: npx lerna init, modify the version, and add "npmClient": "yarn" to the generated lerna.json
Create a global .gitignore
Write a minimal README.md

Here's the content of the initial .gitignore:

node_modules
.DS_Store

And the lerna.json file:

{
  "npmClient": "yarn",
  "packages": [
    "packages/*"
  ],
  "version": "0.0.1"
}

Let's commit that before moving on! You can review this first commit here.

Frontend app setup with CRA

We're gonna use Create React App to bootstrap the frontend app. You need to install it first: npm i -g create-react-app.

Before getting any further, let's create a branch. We're doing this in order to break down the steps into digestible bits, but will squash-merge intermediate branches at the end of each major step.

git checkout -b setup-frontend

Then let's generate the frontend app:

cd packages
create-react-app front

Then remove useless some files from front/src that we won't use:

cd front
rm README.md src/index.css src/App.css src/logo.svg

We have to remove the imports from index.js and App.js accordingly, and we'll replace the JSX returned by App with a simple "Hello World".

Let's check that the app works, git add everything and commit after that! Not of much interest since it's mostly auto-generated stuff, but you can review this commit here.

Custom ESLint setup

CRA provides a default ESLint setup. It's under the eslintConfig key of package.json:

  ...
  "eslintConfig": {
    "extends": "react-app"
  },
  ...

We're gonna change this config, in order to use Airbnb's coding style.

We first initialize a stand-alone ESLint config file:

npx eslint --init

Then we setup ESLint with Airbnb coding style, with the following choices:

How would you like to use ESLint? To check syntax, find problems, and enforce code style
What type of modules does your project use? JavaScript modules (import/export)
Which framework does your project use? React
Where does your code run? Browser
How would you like to define a style for your project? Use a popular style guide
Which style guide do you want to follow? Airbnb (https://github.com/airbnb/javascript)
What format do you want your config file to be in? JSON
Would you like to install them now with npm? (Y/n) N (we'll install them with Yarn)

After that we can:

copy-paste generated .eslintrc.json's content to under the eslintConfig section of package.json (that's why we chose JSON),
delete .eslintrc.json to avoid redundancy,
install the deps with Yarn: yarn add --dev eslint@^6.2.2 typescript@latest eslint-plugin-react@^7.14.3 eslint-config-airbnb@latest eslint-plugin-import@^2.18.2 eslint-plugin-jsx-a11y@^6.2.3 eslint-plugin-react-hooks@^1.7.0,
test the config with npx eslint src/, which reports many errors - most of them due to the src/serviceWorker.js file,
create a .eslintignore file to ignore the src/serviceWorker.js file (which we won't modify anyway),
re-run npx eslint src/, which complains about JSX in .js files, and it being not defined (in App.test.js),
rename the .js files to give them the .jsx extension:
- cd src
- git mv App.js App.jsx
- git mv App.test.js App.test.jsx
- git mv index.js index.jsx
run the linter again - getting a weird All files matched by 'src' are ignored. message, which we can fix by running ESLint with npx eslint src/**/*.js*,
fix the 'it' is not defined error by adding "jest": true to env section in eslintConfig,
add "lint": "npx eslint --fix src/**/*.js*", under the scripts key

After that, we can lint our frontend app by simply running yarn lint.

Let's stage and commit that! Find this commit here.

After that, let's squash-merge the front-setup branch into master - done via this PR.

Backend app setup

This step is gonna be a bit more complicated, so again, we're gonna create an intermediate branch, in order to break it down (after having pulled our master branch).

git checkout -b setup-backend

Simple server creation

Get back to the ~/packages folder, then:

mkdir -p back/src
cd back
npm init --yes
yarn add express body-parser

Let's edit package.json and set version to 0.0.1, and main to build/index.js, before we move on.

Let's also create a .gitignore files to ignore node_modules. That's redundant with the root .gitignore file, but could be useful if we take out the back package out of this repo, for stand-alone use. Besides, we'll have specific stuff to ignore on the backend side.

We're gonna create a simple server in src/index.js, using ES6 import/export syntax:

// src/index.js
import express from 'express';
import bodyParser from 'body-parser';

const port = process.env.PORT || 5000;
const app = express();

app.listen(port, (err) => {
  if (err) {
    console.error(`ERROR: ${err.message}`);
  } else {
    console.log(`Listening on port ${port}`);
  }
});

Of course, unless we use Node 12 with --experimental-modules flag, running node src/index fails with:

import express from 'express';
       ^^^^^^^

SyntaxError: Unexpected identifier
    at Module._compile (internal/modules/cjs/loader.js:723:23)
    ...

I'm not comfortable with using experimental stuff in production, so Babel still seems a more robust option. We'll set it up before committing anything.

Babel setup

Sources:

Let's install all we need: Babel, and also nodemon to restart our server on every change.

yarn add --dev @babel/cli @babel/core @babel/preset-env @babel/node nodemon

@babel/node will allow us to run ES6 code containing import and export statements. The doc explicitly advises not to use it in production, but the other Babel tools will allow us to generate a build suitable for production use.

Then create a .babelrc file containing this:

{
  "presets": ["@babel/preset-env"]
}

Then add a start script to package.json:

  ...
  "scripts": {
    "start": "nodemon --exec ./node_modules/@babel/node/bin/babel-node.js src/index",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  ...

Now we can start our server using yarn start. Hurray! Let's stage and commit our whole back folder (find the commit here).

Build setup

We'll store the production build in the build folder inside packages/back. We could name it dist instead, but I like being consistent with what the CRA build system does.

Let's create a build (and create the build folder) with this command:

npx babel src -d build

It works! We can reference this command as a build script in package.json for convenience (yarn build). The build can be run via node build/index.

  ...
  "scripts": {
    "build": "npx babel src -d build",
    "start": "nodemon --exec ./node_modules/@babel/node/bin/babel-node.js src/index"
    "test": "echo \"Error: no test specified\" && exit 1",
  },
  ...

While we're at it, let's add the build folder to .gitignore.

Tests setup

We'll use these:

Jest,
supertest which will allow testing the Express routes (integration testing)

yarn add --dev jest supertest

Then specify jest as the test script in package.json.

Let's also create a test folder where we'll put our tests. We'll see later on how to organize our test files inside that folder.

Let's write a first test, app.integration.test.js, inside that folder.

// test/app.integration.test.js
import request from 'supertest';
import app from '../src/app';

describe('app', () => {
  it('GETs / and should obtain { foo: "bar" }', async () => {
    expect.assertions(1);
    const res = await request(app)
      .get('/')
      .expect(200);
    expect(res.body).toMatchInlineSnapshot();
  });
});

There are two important things to note here.

we import app from ../src/app, which doesn't exists. We indeed have to split src/index.js into two distinct files.
see the toMatchInlineSnapshot() call at the end of the test? Jest will automatically fill in the parentheses with the expected return values.

Let's address the first.

The new app.js file will export the Express app, so that it can be imported from both the test file and the index file:

// src/app.js
import express from 'express';
import bodyParser from 'body-parser';

const app = express();

module.exports = app;

The modified index.js file will import it and start the server:

// src/index.js
import app from './app';

const port = process.env.PORT || 5000;

app.listen(port, (err) => {
  if (err) {
    console.error(`ERROR: ${err.message}`);
  } else {
    console.log(`Listening on port ${port}`);
  }
});

We check that yarn start and yarn build still function, then try yarn test.

For some reason, we get a ReferenceError: regeneratorRuntime is not defined if we don't configure Babel properly.

We actually have to rename .babelrc to babel.config.js, and modify its content to (see Using Babel in Jest docs):

module.exports = {
  presets: [['@babel/preset-env', {targets: {node: 'current'}}]],
};

This solves the problem. Now the test runs but, of course, fails: no routes are defined in the Express app, so we need to add a '/' route in app.js:

// ...
const app = express();

app.get('/', (req, res) => res.json({ foo: 'bar' }));
// ...

We still get an error:

Cannot find module 'prettier' from 'setup_jest_globals.js'

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:259:17)

Which brings us back to the second point. In order to automatically modify code in the test, Jest uses Prettier, which ensures consistent formatting. Obviously prettier is missing here, so let's install it:

yarn add --dev prettier

Let's run yarn test again: it passes. But if we have a look at test/app.integration.test.js, we see that Prettier applied formatting that isn't consistent with the Airbnb coding style we chose to follow. Fixing that is as easy as creating a Prettier config file, .prettierrc.js:

// .prettierrc.js
module.exports = {
  trailingComma: 'es5',
  tabWidth: 2,
  semi: true,
  singleQuote: true
};

We remove the code that was added by the previous test inside toMatchInlineSnapshot call's parentheses, and run the test again. This time the formatting is consistent with our coding style.

We're done with this, let's stage and commit (see here).

ESLint setup

We'll setup ESLint for Node.js with Airbnb style.

yarn add --dev eslint
npx eslint --init

Let's answer the questions:

How would you like to use ESLint? To check syntax, find problems, and enforce code style
What type of modules does your project use? JavaScript modules (import/export)
Which framework does your project use? None of these
Does your project use TypeScript? N
Where does your code run? Node
How would you like to define a style for your project? Use a popular style guide
Which style guide do you want to follow? Airbnb (https://github.com/airbnb/javascript)
What format do you want your config file to be in? JavaScript
Would you like to install them now with npm? (Y/n) N (again, we'll install them with Yarn)

Then install the deps:

yarn add --dev eslint-config-airbnb-base@latest eslint@6.2.2 eslint-plugin-import@^2.18.2

Then add a "lint": "npx eslint --fix *.js src test *.js", under scripts in package.json.

Running yarn lint for the first time, we get a few errors. We need to:

use the bodyParser import in app.js,
add jest: true under env in .eslintrc.js

As a result, we only have the no-console left, which will be good enough for now (we could setup a proper logger later). Let's save that (commit).

We're done (for now)

That step was long! Don't worry, we're almost done!

Let's squash-merge the setup-backend branch into master via a PR, then pull master.

Pre-commit hooks setup

Husky install

We're gonna setup pre-commit hooks with Husky, so that linting and tests are carried out on every pre-commit event.

git checkout -b setup-husky

Let's get back to the repo root and install Husky:

yarn add --dev husky

Let's commit at this point (here).

`lint-staged` setup

In each of front and back packages, we're gonna install lint-staged, which as the name implies, lints the staged files before committing.

cd packages/front
yarn add --dev lint-staged
cd ../back
yarn add --dev lint-staged

In the package.json of each package, we add a lint-staged section. back and front differ slightly, by the paths to check.

What it does is:

run yarn lint, which fixes auto-fixable errors, but prevents for going further if a more serious error occurs.
stage files again

Here is the front version:

...
"lint-staged": {
  "src/**/*.js*": [
    "yarn lint",
    "git add"
  ]
}
...

Here is the back version:

...
"lint-staged": {
  "**/*.js": [
    "yarn lint",
    "git add"
  ]
}
...

Still in package.json, add a precommit script (same for back and front) to run lint-staged:

  ...
  "scripts": {
    ...
    "precommit": "lint-staged",
    ...
  }
  ...

In front and back packages, we can test this setup by adding errors to App.jsx and app.js, respectively (like declaring an unused variable).

Then we can git add these files to stage them, then run yarn precommit, which should trigger an error. After that, we can revert these files to their previous states, and git add them again.

At this point, pre-commit scripts are set up, but we need to actually run them on pre-commit events. Let's commit before getting there (commit).

Husky setup

Back at the repo root, let's add a husky section to package.json:

  ...
  "husky": {
    "hooks": {
      "pre-commit": "npx lerna run --concurrency 1 --stream precommit"
    }
  }
  ...

It's worth explaining what this does. On each pre-commit event, the npx lerna run --concurrency 1 --stream precommit is run.

npx lerna run <script> will run <script> in each of the packages. We add these flags:

--stream in order to get console output from the scripts as it's emitted
--concurrency 1 to run the scripts from each package sequentially.

Now the pre-commit hooks are configured, and if there are linting errors, we won't be able to commit before fixing them.

Let's git add and commit everything (here).

Hold on, we're not done yet, we also want the tests to be run on pre-commit hooks!

Trigger tests on pre-commit hooks

We have to update the precommit script in each packages's package.json, to run both lint-staged and test:

  ...
  "precommit": "lint-staged && yarn test"
  ...

Additionnally, we want to prevent tests to running in watch mode in React app (which is the default set by CRA).
This requires amending the test script, in frontend app's package.json. See this comment by Dan Abramov.

We install cross-env to have a working cross-platform setup:

yarn add --dev cross-env

And update package.json accordingly, replacing react-scripts test with cross-env CI=true react-scripts test --env=jsdom for the test script.

We make both the back-end and front-end tests fail by making dummy changes to the apps.

For example, in the React app (App.jsx), let's amend the <h1>'s content:

<h1>Hello World { { foo: 'bar' } }</h1>

In the Express app (app.js), let's change what's returned by the '/' route:

app.get('/', (req, res) => res.json({ foo: 'buzz' }));

Then we stage everything and try to commit. We end up with an error, which is great!

lerna ERR! yarn run precommit exited 1 in 'back'
lerna WARN complete Waiting for 1 child process to exit. CTRL-C to exit immediately.
husky > pre-commit hook failed (add --no-verify to bypass)

After reverting the apps to their working state, we're all set! Let's commit this (here).

We can conclude this step by squash-merging the setup-husky branch into master (PR and resulting commit on master).

Connect backend and frontend apps

In this final step, we're gonna setup two additional things:

Fetch data from the backend in the React app
Setup the backend app in order to expose the React build

First let's create a branch to work on this.

git checkout -b setup-back-front-connection

Fetch data from the backend

Let's start with amending the integration test. We'll fetch data from the /api/foo endpoint instead of /. We then have to update app.js accordingly.

Then let's head to the front package.

First we'll add "proxy": "http://localhost:5000" to package.json. Then we'll fetch the /api/foo endpoint from the App component.

Here's the updated App.jsx file:

import React, { useState, useEffect } from 'react';

function App() {
  const [foo, setFoo] = useState('N/A');
  useEffect(
    () => {
      fetch('/api/foo')
        .then((res) => res.json())
        .then((data) => setFoo(data.foo))
        .catch((err) => setFoo(err.message));
    },
  );
  return (
    <div>
      <h1>Hello World</h1>
      <p>
Server responded with foo:
        {foo}
      </p>
    </div>
  );
}

export default App;

Last, in the root-level package.json, we add a scripts section:

...
  "scripts": {
    "lint": "lerna run lint --stream",
    "start": "lerna run start --stream"
  },
...

Now when we run yarn start, Lerna will run the start script in both back and front packages, which means we can launch our full-stack app in a single command-line (and a single terminal window!). Same for yarn lint!

Let's commit this and move on.

Serve the React production build

We're gonna have to amend the app.js file in the back package, in order to do the following:

Compute the absolute path of the build folder, which is right under the front package.
Check whether we are in a production environment or not. If it's the case:
- Setup the build folder as a static assets directory
- Create a wildcard route to serve build/index.html for all unmatched paths

Here's the updated app.js:

// src/app.js
import express from 'express';
import bodyParser from 'body-parser';
import path from 'path';

// Check whether we are in production env
const isProd = process.env.NODE_ENV === 'production';

const app = express();
app.use(bodyParser.json());

app.get('/api/foo', (req, res) => res.json({ foo: 'bar' }));

if (isProd) {
  // Compute the build path and index.html path
  const buildPath = path.resolve(__dirname, '../../front/build');
  const indexHtml = path.join(buildPath, 'index.html');

  // Setup build path as a static assets path
  app.use(express.static(buildPath));
  // Serve index.html on unmatched routes
  app.get('*', (req, res) => res.sendFile(indexHtml));
}

module.exports = app;

We'll now build the backend app by running yarn build, then move to the front folder and run the same command.

Then, going back to our back folder, let's start the app in production mode:

NODE_ENV=production node build/index

Visiting http://localhost:5000, we should see our React app, up and running.

Let's commit this.

That's it!

A last PR (resulting commit on master), and we're done!
Let's tag that commit:

git tag initial-setup
git push --tags

Final thoughts

Setting all this up is a bit tedious and took me quite some time, even though I'd already done something similar before!

So if you don't want to spend precious time, feel free to re-use this setup. I suggest you download an archive of the initial-setup release, instead of forking this repo. This can be used as a starting point for your new project.

I didn't cover every aspect of a project setup, since my focus was more on the ESLint/Jest part. Among the things that we could do to go further:

Set up Prettier
Set up a database, with or without an ORM
Set up dotenv

Let me know if that might be of some interest to you guys!

Also, I'd like to hear your thoughts and suggestions on this setup: I'm eager to know about anything you're doing differently, and why!

Thanks for reading!

Forem: Benoît Hubert