Forem: Ahmed Elsakaan

🎓 Monorepo College Lecture 2: Build Me Up Buttercup

Ahmed Elsakaan — Sat, 08 Apr 2023 22:59:41 +0000

Hello everyone and welcome to the second part of Monorepo College, if you haven't already done so, make sure to read the first part of the series firstly and come back for this one after.

In this part, we will be initializing the project, getting all of the initial files out of the way and then configure Prettier as well as create the first package of our monorepo which will be a tsconfig package responsible for sharing TypeScript configuration files to the other packages we will create in the future.

Getting things off the ground

First things first, I always make sure to get the git side of things all setup and ready as the first step into setting up any project, and Acme is no different.

Let's create a new directory for our project, obviously we will call it Acme.

# Create an "Acme" directory
mkdir Acme

# Change terminal directory to it
cd Acme

Now that we have created the directory for the project and changed directories into it inside our terminal, let's initialise git and create a .gitignore file.

# Initialize git
git init

# If your starting branch is not main, you can change it like so:
git branch -m main

For the .gitignore file, I usually pick a template from github/gitignore which is a collection of great .gitignore file templates. For this project I will be using the Node template.

Next steps

Documentation & community files

In any open source project, you should have a minimum of a README.md, LICENSE, CODE_OF_CONDUCT.md, CONTRIBUTING.md and SECURITY.md, these are highly dependent on your project so I won't be providing a template for them however here are some resources that might help you out.

README.md - https://www.makeareadme.com/
LICENSE - https://opensource.org/licenses/
CODE_OF_CONDUCT.md - https://www.contributor-covenant.org/version/2/1/code_of_conduct/
CONTRIBUTING.md - https://contributing.md/how-to-build-contributing-md/
SECURITY.md - Just explain how to report security vulnerabilities in your project.

Initialising pnpm

For this project, we will be using pnpm as our package manager of choice, but obviously you can use any one you prefer.

`package.json`

Let's firstly create our root level package.json file for the workspace:

pnpm init

This will create a default package.json file at the root of our project, the default package.json is fine but we will change some stuff in ours, here's mine after some changes:

{
  "name": "acme",
  "version": "0.1.0",
  "private": true,
  "description": "The next billion dollar startup",
  "license": "MIT",
  "author": "Acme",
  "packageManager": "pnpm@8.1.1",
  "engines": {
    "node": ">=18.0.0",
    "pnpm": ">=8.1.1"
  }
}

I will go briefly over some of the stuff I have in the package.json file but you can always read the documentation on all of them here

packageManager - This field is read by corepack to automatically pull the package manager with the exact version ensuring that your entire team uses the same package manager and the same version.
engines - This is used to specify ranges of Node and package manager versions that our project is compatible with, by default this option will only warn you if you use an incompatible version but we will modify that behaviour to error out. This is again one measure to ensure your whole team are using the same tools.

`pnpm-workspace.yaml`

The pnpm-workspace.yaml file is used to tell pnpm three things:

This project is a monorepo.
This is the root of the monorepo.
Here are the directories that you should expect packages to be located at.

So let's create our file, we will have it so that pnpm knows our packages will be located in apps/, packages/ or packages/config/:

packages:
  - "packages/*"
  - "packages/config/*"
  - "apps/*"

With this in place, we have officially entered monorepo land 🎉

`.npmrc`

The .npmrc file is a special file that modifies the default behaviour of your package manager, in our case pnpm, I'm going to show you the .npmrc file that we will have and then explain what's going on in there, as always I recommend reading the docs for this as that there are a lot of options that you can specify.

Read the docs

engine-strict=true
prefer-workspace-packages=true
public-hoist-patterns[]=*prisma*
public-hoist-patterns[]=*eslint*
public-hoist-patterns[]=*prettier*

engine-strict=true will make it so that it will error out if you use an incompatible node version with the project.
prefer-workspace-packages - this setting tells pnpm to prioritize the installation of packages locally from our monorepo EVEN if that package has a newer version on the npm registry. This setting is useful for the following case scenarios:

someone merges a new PR with changes to a package, this bumps the version up and publishes it to npm automatically.
someone else is still working on another change in this package, but now when they commit again the CI operation will pull the npm registry version because that version is newer than the one locally. Suddenly checks in CI are messed up.
This can obviously be mitigated by pulling from main immediately after the other PR is merged but we will 100% forget to do that sometimes.

public-hoist-patterns - By default, pnpm won't hoist any packages other than ESLint and Prettier packages to the root node_modules folder, This is mostly great but because we will be using Prisma as our database ORM, it does not play nicely in monorepo settings and hence why we need to add it to our public-hoist-patterns to make it work. This overrides the default value of Prettier and ESLint packages though hence why we need to add them again.

Misc files

Now that we have pnpm all setup in our workspace, we just need to add a couple more files before we get to the fun stuff.

`.editorconfig`

EditorConfig is a tool that defines coding styles for multiple editors and IDEs, this will be somewhat of a fallback for users who don't have prettier formatting in their editor and prettier does support .editorconfig by default so there is no reason to not have it.

root = true

[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = false
insert_final_newline = false

`.nvmrc`

NVM is a Node version manager commonly used to install multiple versions of Nodejs on one system, by having a .nvmrc file we tell nvm which nodejs version to use with this project.

18.15

Setting up TypeScript

In our workspace, we will be using one TypeScript version for the entire monorepo, this means that we will need to install it in our root package.json file.

pnpm add -D typescript @types/node -w

The -w flag is needed to install dependencies to the root package.json file.

We will be setting up the @acme/tsconfig package later on after prettier to share tsconfig presets to our future packages.

If you are using VSCode, your editor is probably using TypeScript v4.9 or something along those lines which doesn't align with our version and will cause errors in particularly using multiple extends. To fix this we need to create a new file at .vscode/settings.json and include this code which will tell VSCode to use our workspace version of TypeScript:

{
  "typescript.tsdk": "node_modules/typescript/lib"
}

Setting up Prettier

Prettier is a code formatter, it supports many languages and editors so we will be using it to ensure a consistent coding style throughout our project and provide format on save capabilities. We will also be using some prettier plugins to give us more functionality.

Install prettier and plugins

pnpm add -D prettier @types/prettier @ianvs/prettier-plugin-sort-imports prettier-plugin-packagejson prettier-plugin-jsdoc prettier-plugin-tailwindcss -w

Configuring prettier

To configure prettier, let's create a prettier.config.cjs file that will host our configuration options.

/** @typedef {import('@ianvs/prettier-plugin-sort-imports').PluginConfig} SortImportsConfig */
/** @typedef {import('prettier').Config} PrettierConfig */

/** @type {PrettierConfig | SortImportsConfig} */
const config = {
  semi: true,
  singleQuote: true,
  trailingComma: "all",
  plugins: [
    require.resolve("@ianvs/prettier-plugin-sort-imports"),
    require.resolve("prettier-plugin-packagejson"),
    require.resolve("prettier-plugin-jsdoc"),
    require.resolve("prettier-plugin-tailwindcss"),
  ],
  pluginSearchDirs: false,
  importOrder: [
    "^react",
    "<TYPES>",
    "<TYPES>^[./]",
    "<THIRD_PARTY_MODULES>",
    "",
    "^@acme/(.*)$",
    "",
    "^@/(.*)$",
    "^[./]",
  ],
  importOrderSeparation: false,
  importOrderSortSpecifiers: true,
  importOrderMergeDuplicateImports: true,
};

module.exports = config;

The first 3 options are self explanatory, then we list the plugins and the important bit here after is the pluginSearchDirs: false which without it, prettier-plugin-tailwindcss acts weirdly and won't work as expected.

Next up we configure the import statements order that we want to use, whenever we use "" it's to add separations between our import statements and we set the default separation behaviour to false instead.

The importOrderSortSpecifiers option will format different specifiers in an import statement such as type imports and normal imports.

Lastly, the importOrderMergeDuplicateImports will merge import statements from the same source into one import statement.

`.prettierignore`

The .prettierignore file is used to exclude prettier from formatting certain files, we will use this to tell prettier to not format any dist and .next folders as well as the pnpm-lock.yaml file.

.next
pnpm-lock.yaml
dist

Adding format scripts to our package.json

In our root package.json file, let's add some scripts that we can run to format the entire workspace with prettier. Add this to our already existing package.json file:

"scripts": {
  "format:check": "prettier --check .",
  "format:write": "prettier --write ."
}

We can now run pnpm format:write to format all of the files in our project!

Creating our first package

Now that we have pnpm and prettier all setup, we are ready to introduce the first package to our monorepo, exciting times!

Since we have already installed typescript in our monorepo, the next step is to setup TypeScript in our workspace in a way that will make it easy to use it in our future packages.

We will be creating a tsconfig package that shares TS configuration presets to our packages.

Initializing the package

Let's create the directory that will host our package at packages/config:

mkdir packages
mkdir packages/config
mkdir packages/config/tsconfig

First things first, let's create a package.json file for our package:

{
  "name": "@acme/tsconfig",
  "version": "0.1.0",
  "private": true,
  "description": "Presets for common tsconfig.json configurations",
  "license": "MIT",
  "author": "Acme"
}

Don't forget to include "private": true, since we won't be publishing it to NPM.

Creating our tsconfigs

Now that we have the package.json for our @acme/tsconfig package setup, we then need to install some dependencies for it.

I personally am a big advocate for writing the least amount of tsconfig possible, and the tsconfig/bases package serves as a great source for getting tsconfig templates.

Let's install some of them:

pnpm --filter tsconfig add -D @tsconfig/node18 @tsconfig/strictest @tsconfig/next @tsconfig/vite-react

Using the --filter option, these dependencies will be installed in packages/config/tsconfig/package.json file and NOT the root package.json file.

Base tsconfig

Now that we have our dependencies installed, let's create our first preset which will be named base.json in packages/config/tsconfig, this file will be a base configuration that all the other ones will extend from:

{
  "$schema": "https://json.schemastore.org/tsconfig",
  "display": "Base",
  "extends": ["@tsconfig/strictest/tsconfig", "@tsconfig/node18/tsconfig"],
  "compilerOptions": {
    "allowJs": true,
    "checkJs": true,
    "moduleResolution": "bundler",
    "module": "esnext",
    "resolvePackageJsonExports": true,
    "verbatimModuleSyntax": false
  }
}

Since we are using TypeScript version 5, we can extend multiple tsconfigs rather than only one. In our base configuration we extend the strictest and the node18 presets and then we modify them slightly to better fit our needs.

Some of the worthy modifications include:

moduleResolution we set the base module resolution to TypeScript 5's new bundler, since we will be using tsup throughout this monorepo for bundling, this serves as a great resolution and allows us to enable resolvePackageJsonExports which will enable easy multi exports support for out packages.
resolvePackageJsonExports is a field that forces TypeScript to read and resolve files from a package's exports field in it's package.json file, this allows us to easily create multiple exports for our packages and avoids us using hacks such as typesVersions.

React tsconfig

The next tsconfig file on our list is a react specific one, this will serve as a tsconfig file for all of our react packages such as the UI components library package coming in the future, create a react.json file in our package with the following content:

{
  "$schema": "https://json.schemastore.org/tsconfig",
  "display": "Base",
  "extends": ["./base.json", "@tsconfig/vite-react/tsconfig"],
  "compilerOptions": {
    "moduleResolution": "bundler",
    "allowJs": true
  }
}

Here we simply extend the base configuration and @tsconfig/vite-react/tsconfig, enable the bundler moduleResolution since the vite-react preset overrides the one in our base.json and lastly allow javascript files so that we can also type check them.

Next.js tsconfig

The last file that we will create for our package will be a Next.js specific tsconfig preset called nextjs.json:

{
  "$schema": "https://json.schemastore.org/tsconfig",
  "display": "Nextjs",
  "extends": ["./base.json", "@tsconfig/next/tsconfig"],
  "compilerOptions": {
    "moduleResolution": "nodenext"
  }
}

We extend the base.json preset again and also this time @tsconfig/next/tsconfig. We also modify the moduleResolution to nodenext because Next.js does not work with the bundler moduleResolution and will replace it with node upon running the dev command which we do not want.

Plugging everything up

Now that we have base.json, react.json and nextjs.json in our package, we are almost there to have a fully functional package that we can use throughout our monorepo. The last part is to specify in our package.json file that we expect these files to be accessible. This is done through the files property. And so after all of this work our final package.json of @acme/tsconfig should look like this:

{
  "name": "@acme/tsconfig",
  "version": "0.1.0",
  "private": true,
  "description": "Presets for common tsconfig.json configurations",
  "license": "MIT",
  "author": "Acme",
  "files": ["./base.json", "./nextjs.json", "./react.json"],
  "devDependencies": {
    "@tsconfig/next": "^1.0.5",
    "@tsconfig/node18": "^1.0.1",
    "@tsconfig/strictest": "^2.0.0",
    "@tsconfig/vite-react": "^1.0.1"
  }
}

We now have a fully functional monorepo package and to start using it, let's create a tsconfig.json file at the root of our monorepo that will handle providing a TypeScript configuration for the entire monorepo (unless another tsconfig.json file is in a package, that will be used instead for the package). We do this to mainly provide ESLint with a tsconfig.json file for our root files such as the prettier.config.cjs file.

We firstly need to install @acme/tsconfig in our monorepo's root package.json, to do this we simply:

pnpm add -D @acme/tsconfig -w

We can now create a tsconfig.json file at the root of our repository with the following content:

{
  "extends": "@acme/tsconfig/base",
  "compilerOptions": {
    "noEmit": true
  },
  "include": [
    ".eslintrc.cjs",
    "**/*.ts",
    "**/*.tsx",
    "**/*.js",
    "**/*.jsx",
    "**/*.cjs",
    "**/*.mjs",
    "**/*.cts",
    "**/*.mts"
  ]
}

You can see how we are now extending from @acme/tsconfig/base this points to packages/config/tsconfig/base.json.

To test that this is working, we can define a new property in our prettier.config.cjs file that does not exist on the prettier options type and we can expect typescript to display an error. This happens as that we allow JavaScript files in our base tsconfig and we also set the checkJs property to true, so we don't need to explicitly define // @ts-check for every JS file.

Conclusion

And with all of that, we now have a fully setup prettier configuration as well as we created our first package, @acme/tsconfig. We also setup TypeScript configuration presets that we will be able to use once we start adding a Next.js application, a React UI components library...etc

We will use the latest and greatest features of TypeScript v5 in this monorepo to give us the best DX possible and let us move quicker creating and developing apps and packages.

I know this might have been a very dense post, this is why I don't expect everyone to completely understand everything that we have gone through here. For that, feel free to DM me on Twitter with all of your questions and I will help you out.

One last thing

As the previous part, this post title is completely unrelated to the topic. This part's title is inspired by The Foundations' record, "Build Me Up Buttercup" release in 1968. Give it a listen!

🎓 Monorepo college lecture 1: a beautiful morning

Ahmed Elsakaan — Fri, 07 Apr 2023 07:06:21 +0000

Photo by Josh Felise on Unsplash

Embark on a thrilling journey to Monorepo Land, where code sharing and team collaboration are the order of the day! 🚄🌅

Greetings, fellow monorepo enthusiasts! You might recall that tweet from a while ago, but the wait is over. Monorepo College has arrived, and it's time to dive in!

The Monorepo Mission

Monorepo configurations can be challenging to set up, but once you nail them down, they should fade into the background, allowing you to focus on what truly matters. We've all been there: a monorepo is running smoothly, only for a tiny change to wreak havoc and send us back to configuration purgatory.

In reality, monorepos should streamline your workflow through shared code, so you can concentrate on delivering quality products rather than wrestling with broken configs.

It's no secret that monorepo conventions are hard to find, and everyone seems to draw inspiration from different sources. So, how can we be confident that our current setup will scale effectively down the road?

Introducing Monorepo College, a series of blog posts and YouTube videos designed to guide you through building a monorepo starter kit from scratch. The primary goal is to teach you how to architect a monorepo setup, but if you prefer to skip the process, the starter kit will be available on GitHub after the series.

In this first installment, we'll introduce the world of monorepos, outline the series' scope, and discuss some conventions to consider during setup.

Monorepos: The What and the Why

A monorepo, short for "monolithic repository," is a single repository housing multiple projects, packages, or codebases. This setup enables you to store all your projects in one location, facilitating code sharing between them.

To learn more about the concept of monorepos, check out Monorepo tools.

The Monorepo Marvel: Advantages

Monorepos boast several benefits over traditional polyrepos (multiple repositories) setups, including:

Effortless code sharing: With a monorepo, you can easily share components, such as a UI component, across multiple applications.
Consistent tooling: Monorepos enable you to standardize tool configurations across packages, resulting in a unified toolset. For example, an ESLint config package can be used by all packages in the monorepo, ensuring consistent lint rules.
Simplified dependency management: Monorepos offer a clearer view of package dependencies, and with tools like pnpm, you can update dependencies across packages with a single command. This helps avoid different projects using different versions of the same dependency, such as React.

While this list is by no means exhaustive, it highlights how monorepos can enhance the development process for more complex codebases.

The Monorepo Matrix: When It Works and When It Doesn't

Monorepos can be a perfect fit for some projects, but not for others. Generally:

If your code is highly complex and needs to be broken down into smaller, independently managed components, a monorepo is the way to go. For a simple portfolio website, a monorepo is overkill.
If your team works on distinct areas of the product, a monorepo is a must.
(Possibly a hot take) If you're building a startup or a project with an existing or planned team, a monorepo is likely your best bet.
If you're developing an open-source library with multiple parts, a monorepo is essential. For a simple utility library, there's no need for one.

As you can see, the more complex the codebase, the more a monorepo makes sense.

Building Our Monorepo Masterpiece

Before diving into the objectives of this series, let's introduce Acme, our imaginary startup. Throughout this series, we'll create a monorepo for Acme, starting from scratch.

Envisioning the Monorepo Structure

Here's a potential layout for our monorepo, which may change as we progress:

.
├── apps
│   ├── docs
│   └── web
└── packages
    ├── api
    ├── config
    │   ├── eslint
    │   ├── playwright
    │   ├── storybook
    │   ├── tailwind
    │   ├── tsconfig
    │   ├── tsup
    │   └── vitest
    ├── database
    ├── features
    ├── storybook
    └── ui

Apps

The apps folder will house all of Acme's applications, such as mobile and web apps, marketing sites, and product-specific applications.

packages/config

Before discussing the packages folder, let's examine the packages/config directory. This folder will contain configurations for tools used by other packages in the monorepo, such as shared tsconfig presets, tailwind, tsup, vitest, and more.

Centralizing tool configurations helps streamline maintenance and troubleshooting.

packages

The packages directory will host various packages used by the apps. For example, a ui package will contain shared UI components used by the applications.

We'll also create a packages/api for our tRPC API, which will communicate with and fetch data from our database stored in packages/database.

Rationale Behind the Decisions

Our layout prioritizes flexibility and modularity. Everything is separated in little packages throughout the monorepo and they all come together to form a fully functional application.

If we wanted to replace tsup, for example, we'd only need to modify packages/config/tsup. You could opt for a more modular name like packages/config/bundler, but we're confident tsup will serve us well for now.

Most packages (excluding config) won't be used by many apps, and usage should not determine whether code is split into a package. In essence, if a product area can be isolated in its own package, it probably should. Isolating UI components and the API allows us to replace applications while retaining easy access to these components. This approach is also ideal for teamwork, as individuals can focus solely on their assigned packages, improving their development experience.

Tools and Technologies to Tackle

Throughout this series, we'll explore various technologies and learn how to configure them in a monorepo context, including:

pnpm
TypeScript
Storybook
Next.js
Clerk
Prisma
Tsup
ESLint
Prettier
Changesets
Tailwindcss
Vitest
Playwright
NX
Turborepo
Github actions

The primary focus will be on general configuration principles to help you better understand and apply these techniques to any technology in your monorepo.

Homework

Each installment will include optional homework or challenges to deepen your understanding of monorepos.

For this part, try this simple task:

Create a monorepo with packages/is-odd and packages/is-even. The packages/is-even should return the inverse of packages/is-odd. This exercise will familiarize you with sharing code between packages.

Feel free to send me your creation via Twitter DM, and I'll provide feedback!

Wrapping Up

Getting monorepos right can be challenging, but when set up correctly, they should streamline your development process. Monorepo College aims to provide insights into monorepo structures and configurations, helping you create efficient and effective monorepos that boost, rather than hinder, your development.

Join me in the next installment, where we'll begin building Acme. Long live Acme!

One last thing

Each post will feature an unrelated title inspired by a song. This episode's title comes from The Rascals' "A Beautiful Morning," released in 1968. Give it a listen!