Forem: Jackie McDoniel

Automating your package deployment in an Nx Monorepo with Changeset

Jackie McDoniel — Wed, 06 Oct 2021 02:25:41 +0000

All right, all right. I know, another monorepo package deployment post. But this was a pattern I didn't have luck finding with my Google Fu, and I figure that others may be interested in how to make an Nx workspace work with the changesets package.

If you haven't heard of Nx, it's a monorepo build framework that helps keep your build, test, and run scripts ini the same format while having different actions, along with being smart enough to determine what has and hasn't been affected since your last changes.

Changesets on the other hand is a semver version management tool that integrates well with pnpm and yarn workspaces by looking at the workspace file and walking you through a wizard for setting up which packages have changed (and has a great GitHub Actions integration and bot for previewing which packages will be updated and how).

Now that these are defined, onto the meat of those post.

So what's the problem?

I migrated from using a pnpm workspace to using Nx with pnpm workspaces so that I could take advantage of running all of my commands through Nx and get some sweet caching of my commands and familiarity between all of the commands. So now instead of having to try to remember "Is it pnpm -r build?" and the "How do I filter to only some of the projects again?", I only have to remember pnpm nx build <project> or pnpm nx --run-many build --all. I promise, these commands start rolling off your fingers as you keep using them.

Now, with my pnpm workspace, every package was built in a dist directory right next to the src so I'd have packages/<package>/src and packages/<package>/dist and I could easily keep a package.json per package and have it deploy from right there. With Nx, that's not really the case anymore. There's a root package.json that holds all of the dependencies and most of the time, the outDir is set to be <workspaceRoot>/dist/<package>/src. Kind of makes deploying from a central location just a bit harder.

For the build command with Nx, I'm actually using the @nrwl/node:package executor, which runs a build command via tsc and copies over the package.json (if you have one), and sets it up with the proper path for publishing, and copies the package's README and CHANGELOG if they exist. If you don't have a package.json for the library, Nx will create one for you and populate it with the dependencies it finds in the compiled code. You can set the outputPath to anything you want, but I've found that dist/<package> works really well for this automation setup. You could actually do packages/<package>, but then your ts and your js will get mixed with each other, which I find very messy and confusing.

So now why is having these packages in the dist/ directory a problem? Well, because changesets works by reading the workspace's configuration, and as we want to have our source code version tracked, but not the compiled code, we don't normally include the dist in our git repository. Because of this, when we do things like pnpm changeset to create a new changeset, we'll eventually be modifying the package.json in the packages/<package>/ directory.

The changeset action that I mentioned earlier works like this during your CI workflow:

check if there's one or more changeset file(s)
1. if yes, open a PR with the updates to the appropriate package.jsons (in the packages/<package> directories)
2. if no, check that the packages in the workspace configuration are all up to date
  1. if no, run the publish action to update npm with the most recent package version

Something to note is that when the package.json for a specific package, that package will then be picked up by nx affected meaning that you can only build the packages that will be published in CI if you feel up to going that route.

So a typical workflow would look something like

git pull
git checkout -b
make changes
commit changes as often as you normally would
pnpm changeset
follow the changeset wizard and set up the changes to be made
git add .
git commit
git push origin
merge PR
let the changeset action make a new version PR
review and merge
let changeset publish

The final problem occurs in this "let changeset publish" step, and the solution is coming up.

Modifying the workspace file only in CI

So after playing with ideas of using different tools, manual workflows, I finally came to a solution that works well using some bash scripting in CI. For me, my pnpm-workspace.yaml file usually looks something like

packages:
  - 'packages/**

and I realized that with the sed tool I would be able to change that packages/** to dist/** only during CI so that the changeset publish command would look in the correct directory for the packages I want to publish. So now the actions file looks something like this:

name: Release

on:
  push:
    branches:
      - main

env:
  NX_BRANCH: ${{ github.event.pull_request.head.ref }}
  NX_RUN_GROUP: ${{ github.run_id }}
  NX_CLOUD_DISTRIBUTED_EXECUTION: true
  NX_CLOUD_AUTH_TOKEN: ${{ secrets.NX_CLOUD_TOKEN }}

jobs:
  release:
    name: Release
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repo
        uses: actions/checkout@master
        with:
          # This makes Actions fetch all Git history so that Changesets can generate changelogs with the correct commits
          fetch-depth: 0
      - name: Derive appropriate SHAs for base and head for `nx affected` commands
        uses: nrwl/nx-set-shas@v1

      - name: Setup Node.js 14.x
        uses: actions/setup-node@master
        with:
          node-version: 14.x

      - name: Install pnpm
        run: npm i -g pnpm

      - name: Install Dependencies
        run: pnpm i --frozen-lockfile=false

      - name: Build Projects
        run: pnpm build

      - name: Modify Workspace File
        run: sed -e "s|'packages\/|'dist/|" pnpm-workspace.yaml > pnpm-new.yaml && mv pnpm-new.yaml pnpm-workspace.yaml

      - name: Create Release Pull Request or Publish to npm
        id: changesets
        uses: changesets/action@master
        with:
          # This expects you to have a script called release which does a build for your packages and calls changeset publish
          publish: pnpm release
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

      - name: Stop Nx Cloud Agents
        run: pnpx -y nx-cloud stop-all-agents

  nx_agent:
    runs-on: ubuntu-latest
    name: Nx Agent
    timeout-minutes: 60
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
        with:
          node-version: 14.x
      - name: Install pnpm
        run: npm i -g pnpm

      - name: Install Dependencies
        run: pnpm i --frozen-lockfile=false
      - run: pnpx nx-cloud start-agent

Normally, I'd use the -i option for sed, to write in place, but that's not an option on GitHub Action Runners, so writing to a temp file and overwriting the original file works instead.

So now, what's happening is that when a PR is made with a changeset (or multiple changesets) a second PR will be automatically opened by the changeset action to update bump the package versions and generate the CHANGELOG files based on the changesets being merged in. Then, when that PR gets merged, Nx will see what packages have been modified, build them to the dist/<package> directory, copying over the package.json, README, and CHANGELOG. Now our custom sed script will rewrite the pnpm-workspace.yml so that we point changesets to the correct directory for the to-be-published packages. Changesets will then see what packages are there, what is public, and what has mismatched versions compared to what's on the npm registry, and publish whatever is missing so that it's all up to date. Also, a GitHub release will be created during this too, so you have a tag and release pointing to the repo at that point in time.

Overall, I'm pretty excited to have this flow automated and working through three of my favorite package management tools. Everything will also work if you're using a yarn workspace instead, just change the sed script to modify the workspace file for yarn instead of the one for pnpm. I'm currently using this for my ogma and nest-commander repositories, feel free to have a look if you need some inspiration and/or real life examples. If you're developing packages and using an Nx workspace and need automated package deployment, give this a shot.

A side note that I just ran into: if you have workspace in your package.json because you changed from yarn to pnpm or some similar reason, make sure to remove it, or the above workflow will no longer work as expected.

Setting Up Sessions with NestJS, Passport, and Redis

Jackie McDoniel — Tue, 24 Aug 2021 23:51:24 +0000

Jay is a member of the NestJS core team, primarily helping out the community on Discord and Github and contributing to various parts of the framework.

If you're here, you're either one of the avid readers of my, just stumbling about dev.to looking for something interesting to read, or you're searching for how to implement sessions with Passport and NestJS. Nest's own docs do a pretty good job of showing how to set up the use of JWTs with Passport, but are lacking when it comes to how to use sessions. Maybe you want to use a session store because of supporting some legacy software. Maybe it's because JWTs bring too much complexity with scope. Maybe it's because you're looking for an easier way to set up refresh tokens. Whatever the case, this article is going to be for you.

Pre-requisites

I'm going to be using NestJS (it's in the title, so I hope that's obvious) and I'm going to be making use of Guards so if you don't know what those are, I highly suggest reading up on them first. Don't worry, I'll wait.

I'm also going to be not using an HTTP client like Postman or Insomnia, but using cURL instead. I lke living in the terminal as much as I can, as it gives me immediate feedback between my terminals. Feel free to use whichever you prefer, but the code snippets will be curls.

And speaking of immediate feedback, I'm also going to be using tmux, which is a terminal multiplexer, to allow me to run multiple terminals at a time within the same window and logical grouping. This allows me to keep a single terminal window up and view my server logs, docker-compose instance and/or logs, and make curls without having to alt-tab to change views. Very handy, and very customizable.

Lastly, I'll be using docker and a docker-compose file to run a Redis instance for the session storage and to allow for running a redis-cli to be able to query the redis instance ran by Docker.

All of the code will be available to follow along with and run here. Just note that to run it after you clone and run the install for the repo, you'll need to cd blog-posts/nestjs-passport-sessions and then run nest start --watch yourself. Just a side effect of how the repo is set up for my dev.to blogs.

Following along from scratch

If you're following along with the code that's pre-built, feel free to skip over this.

To set up a similar project from scratch, you'll need to first set up a Nest project, which is easiest through the Nest CLI

nest new session-authentication

Choose your package manager of choice, and then install the follow dependencies

pnpm i @nestjs/passport passport passport-local express-session redis connect-redis bcrypt

And the following peer dependencies

pnpm i -D @types/passport-local @types/express-session @types/connect-redis @types/bcrypt @types/redis

npm and yarn work fine as well, I just like pnpm as a package manager

Now you should be okay to follow along with the rest of the code, building as we go.

NestJS and Passport

The AuthGuard()

Like most @nestjs/ packages, the @nestjs/passport package is mostly a thin wrapper around passport, but Nest does do some cool things with the passport package that I think are worth mentioning. First, the AuthGuard mixin. At first glance, this mixin may look a little intimidating, but let's take it chunk by chunk.

export const AuthGuard: (type?: string | string[]) => Type<IAuthGuard> = memoize(createAuthGuard);

Ignoring the memoize call, this createAuthGuard is where the magic of class creation happens. We end up passing the type, if applicable, to the createAuthGuard method and will eventually pass that back to the @UseGuards(). Everything from here on, unless mentioned otherwise, will be a part of the createAuthGuard method.

class MixinAuthGuard<TUser = any> implements CanActivate {
  constructor(@Optional() protected readonly options?: AuthModuleOptions) {
    this.options = this.options || {};
    if (!type && !this.options.defaultStrategy) {
      new Logger('AuthGuard').error(NO_STRATEGY_ERROR);
    }
  }
...

The constructor allows for an optional injection of AuthModuleOptions. This is what is passed to PassportModule.register(). This just allows Nest to figure out if the defaultStrategy is used or the named one passed to AuthGuard.

  async canActivate(context: ExecutionContext): Promise<boolean> {
    const options = {
      ...defaultOptions,
      ...this.options,
      ...await this.getAuthenticateOptions(context)
    };
    const [request, response] = [
      this.getRequest(context),
      this.getResponse(context)
    ];
    const passportFn = createPassportContext(request, response);
    const user = await passportFn(
      type || this.options.defaultStrategy,
      options,
      (err, user, info, status) =>
        this.handleRequest(err, user, info, context, status)
    );
    request[options.property || defaultOptions.property] = user;
    return true;
  }

This here reads through decently well, we have custom methods for getting the authentication options (defaults to returning undefined), getting the request and response objects (defaults to context.switchToHttp().getRequest()/getResponse()), and then this createPassportContext method that is called and then it's return is immediately called with the strategy name and options. Then, we set req.user to the return of passportFn and return true to let the request continue. The next code block is not a part of the mixin or MixinAuthGuard class.

const createPassportContext = (request, response) => (type, options, callback: Function) =>
  new Promise<void>((resolve, reject) =>
    passport.authenticate(type, options, (err, user, info, status) => {
      try {
        request.authInfo = info;
        return resolve(callback(err, user, info, status));
      } catch (err) {
        reject(err);
      }
    })(request, response, err => (err ? reject(err) : resolve())),
  );

Here's where some magic may be seen to happen: Nest ends up calling passport.authenticate for us, so that we don't have to call it ourselves. In doing so, it wraps passport in a promise, so that we can manage the callback properly, and provides it's own handler to the authenticate function. This entire method is actually creating a different callback function so that we can end up calling this.handleRequest with the err, user, info, and status returned by passport. This can take a bit of time to understand, and isn't necessarily needed, but it's usually good to know what some of the code under the hood is doing.

  handleRequest(err, user, info, context, status): TUser {
    if (err || !user) {
      throw err || new UnauthorizedException();
    }
    return user;
  }

This is pretty straightforward, but it's useful to know this method is here. As mentioned in Nest's docs if you need to do any debugging about why the request is failing, here is a good place to do it. Generally just adding the line console.log({ err, user, info, context, status }) is enough, and will help you figure out pretty much anything going wrong within the passport part of the request.

There's two other classes I want to talk about before getting to the implementation, but I promise it'll be worth it!

The PassportStrategy()

So the next mixin we have to look at is the PassportStrategy mixin. This is how we end up actually registering our strategy class's validate method to passport's verify callback. This mixin does a little bit more in terms of some advance JS techniques, so again, lets take this chunk by chunk.

export function PassportStrategy<T extends Type<any> = any>(
  Strategy: T,
  name?: string | undefined
): {
  new (...args): InstanceType<T>;
} {
  abstract class MixinStrategy extends Strategy {

This part is pretty straightforward, we're just passing the passport strategy class and an optional renaming of the strategy to the mixin.

constructor(...args: any[]) {
  const callback = async (...params: any[]) => {
    const done = params[params.length - 1];
    try {
      const validateResult = await this.validate(...params);
      if (Array.isArray(validateResult)) {
        done(null, ...validateResult);
      } else {
        done(null, validateResult);
      }
    } catch (err) {
      done(err, null);
    }
  };

This is the first half of the constructor. You'll probably notice right of the bat that we don;'t call super, at least not yet. This is because we're setting up the callback to be passed to passport later. So what's happening here is we're setting up a function that's going to be calling this.validate and getting the result from it. If that result happens to be an array, we spread the array (passport will use the first value), otherwise we'll end up calling the done callback with just the result. If there happens to be an error, in good ole callback style, it'll be passed as the first value to the done method.

  super(...args, callback);
  const passportInstance = this.getPassportInstance();
  if (name) {
    passportInstance.use(name, this as any);
  } else {
    passportInstance.use(this as any);
  }
}

Now we end up calling super, and in doing so, we overwrite the original verify with the new callback we just created. This sets up the entire passport Strategy class that we're going to use for the strategy's name. Now all that's left to do is tell passport about it, by calling passportInstance.use(this) (or passing the custom name as the first argument).

If any of that went a little too deep, don't worry. It's something you can come back to if you really want, but isn't necessary for the rest of ths article.

PassportSerializer

Finally, an actual class! This is the most straightforward and the last bit of passport I'll talk about before getting into the implementation of sessions. This class usually won't be used in Nest applications _unless you are using sessions, and we're about to see why.

So passport has the notion of serializing and deserializing a user. Serializing a user is just taking the user's information and compressing it/ making it as minimal as possible. In many cases, this is just using the ID of the user. Deserializing a user is the opposite, taking an ID and hydrating an entire user out of it. This usually means a call to a database, but it's not necessary if you don't want to worry about it. Now, Nest has a PassportSerializer class like so:

export abstract class PassportSerializer {
  abstract serializeUser(user: any, done: Function);
  abstract deserializeUser(payload: any, done: Function);

  constructor() {
    const passportInstance = this.getPassportInstance();
    passportInstance.serializeUser((user, done) => this.serializeUser(user, done));
    passportInstance.deserializeUser((payload, done) => this.deserializeUser(payload, done));
  }

  getPassportInstance() {
    return passport;
  }
}

You should only ever have one class extending the PassportSerializer, and what it should do is set up the general serialization and deserialization of the user for the session storage. The user passed to serializeUser is usually the same value as req.user, and the payload passed to deserializeUser is the value passed as the second parameter to the done of serializeUser. This will make a bit more sens when it is seen in code.

Break Time

Okay, that was a lot of information about NestJS and Passport all at once, and some pretty complex code to go through. Take a break here if you need to. Get some coffee, stretch your legs, go play that mobile game you've been wanting to. Whatever you want to do, or continue on with the article if you want.

Running Redis Locally

You can either install and run redis locally on your machine, or you can use a docker-compose.yml file to run redis inside a container. The following compose fle is what I used while working on this article

# docker-compose.yml

version: '3'
services:
  redis:
    image: redis:latest
    ports:
      - '6379:6379'
  rcli:
    image: redis:latest
    links:
      - redis
    command: redis-cli -h redis

And then to run redis, I just used docker compose up redis -d. When I needed to run the redis CLI, I used docker compose run rcli to connect to the redis instance via the docker network.

Setting Up the Middleware

Now on to the middleware we're going to be using: for setting up sessions and a way to store them, I'm going to be using express-session, and connect-redis for the session and session store, and redis as the redis client for connect-redis. I'm also going to be setting up our middleware via a Nest middleware instead of using app.use in the bootstrap so that when we do e2e testing, the middleware is already set up (that's out of the scope of this article). I've also got redis set up as a custom provider using the following code

// src/redis/redis.module.ts

import { Module } from '@nestjs/common';
import * as Redis from 'redis';

import { REDIS } from './redis.constants';

@Module({
  providers: [
    {
      provide: REDIS,
      useValue: Redis.createClient({ port: 6379, host: 'localhost' }),
    },
  ],
  exports: [REDIS],
})
export class RedisModule {}

// src/redis/redis.constants.ts

export const REDIS = Symbol('AUTH:REDIS');

which allows for us to use @Inject(REDIS) to inject the redis client. Now we can configure our middleware like so:

// src/app.module.ts

import { Inject, Logger, MiddlewareConsumer, Module, NestModule } from '@nestjs/common';
import * as RedisStore from 'connect-redis';
import * as session from 'express-session';
import * as passport from 'passport';
import { RedisClient } from 'redis';

import { AppController } from './app.controller';
import { AppService } from './app.service';
import { AuthModule } from './auth';
import { REDIS, RedisModule } from './redis';

@Module({
  imports: [AuthModule, RedisModule],
  providers: [AppService, Logger],
  controllers: [AppController],
})
export class AppModule implements NestModule {
  constructor(@Inject(REDIS) private readonly redis: RedisClient) {}
  configure(consumer: MiddlewareConsumer) {
    consumer
      .apply(
        session({
          store: new (RedisStore(session))({ client: this.redis, logErrors: true }),
          saveUninitialized: false,
          secret: 'sup3rs3cr3t',
          resave: false,
          cookie: {
            sameSite: true,
            httpOnly: false,
            maxAge: 60000,
          },
        }),
        passport.initialize(),
        passport.session(),
      )
      .forRoutes('*');
  }
}

and have passport ready to use sessions. There's two important things to note here:

passport.initialize() must be called before passport.session().
session() must be called before passport.initialize()

With this now out of the way, let's move on to our auth module.

The AuthModule

To start off, let's define our User as the following

// src/auth/models/user.interface.ts

export interface User {
  id: number;
  firstName: string;
  lastName: string;
  email: string;
  password: string;
  role: string;
}

And then have RegisterUserDto and LoginUserDto as

// src/auth/models/register-user.dto.ts

export class RegisterUserDto {
  firstName: string;
  lastName: string;
  email: string;
  password: string;
  confirmationPassword: string;
  role = 'user';
}

and

// src/auth/models/login-user.dto.ts

export class LoginUserDto {
  email: string;
  password: string;
}

Now we'll set up our LocalStrategy as

// src/auth/local.strategy.ts

import { Injectable } from '@nestjs/common';
import { PassportStrategy } from '@nestjs/passport';
import { Strategy } from 'passport-local';
import { AuthService } from './auth.service';

@Injectable()
export class LocalStrategy extends PassportStrategy(Strategy) {
  constructor(private readonly authService: AuthService) {
    super({
      usernameField: 'email',
    });
  }

  async validate(email: string, password: string) {
    return this.authService.validateUser({ email, password });
  }
}

Notice here we're passing usernameField: 'email' to super. This is because in our RegisterUserDto and LoginUserDto we're using the email field and not username which is passport's default. You can change the passwordField too, but I had no reason to do that for this article. Now we'll make our AuthService,

// src/auth/auth.service.ts

import { BadRequestException, Injectable, UnauthorizedException } from '@nestjs/common';
import { compare, hash } from 'bcrypt';

import { LoginUserDto, RegisterUserDto } from './models';
import { User } from './models/user.interface';

@Injectable()
export class AuthService {
  private users: User[] = [
    {
      id: 1,
      firstName: 'Joe',
      lastName: 'Foo',
      email: 'joefoo@test.com',
      // Passw0rd!
      password: '$2b$12$s50omJrK/N3yCM6ynZYmNeen9WERDIVTncywePc75.Ul8.9PUk0LK',
      role: 'admin',
    },
    {
      id: 2,
      firstName: 'Jen',
      lastName: 'Bar',
      email: 'jenbar@test.com',
      // P4ssword!
      password: '$2b$12$FHUV7sHexgNoBbP8HsD4Su/CeiWbuX/JCo8l2nlY1yCo2LcR3SjmC',
      role: 'user',
    },
  ];

  async validateUser(user: LoginUserDto) {
    const foundUser = this.users.find(u => u.email === user.email);
    if (!user || !(await compare(user.password, foundUser.password))) {
      throw new UnauthorizedException('Incorrect username or password');
    }
    const { password: _password, ...retUser } = foundUser;
    return retUser;
  }

  async registerUser(user: RegisterUserDto): Promise<Omit<User, 'password'>> {
    const existingUser = this.users.find(u => u.email === user.email);
    if (existingUser) {
      throw new BadRequestException('User remail must be unique');
    }
    if (user.password !== user.confirmationPassword) {
      throw new BadRequestException('Password and Confirmation Password must match');
    }
    const { confirmationPassword: _, ...newUser } = user;
    this.users.push({
      ...newUser,
      password: await hash(user.password, 12),
      id: this.users.length + 1,
    });
    return {
      id: this.users.length,
      firstName: user.firstName,
      lastName: user.lastName,
      email: user.email,
      role: user.role,
    };
  }

  findById(id: number): Omit<User, 'password'> {
    const { password: _, ...user } = this.users.find(u => u.id === id);
    if (!user) {
      throw new BadRequestException(`No user found with id ${id}`);
    }
    return user;
  }
}

our controller

// src/auth/auth.controller.ts

import { Body, Controller, Post, Req, UseGuards } from '@nestjs/common';

import { LocalGuard } from '../local.guard';
import { AuthService } from './auth.service';
import { LoginUserDto, RegisterUserDto } from './models';

@Controller('auth')
export class AuthController {
  constructor(private readonly authService: AuthService) {}

  @Post('register')
  registerUser(@Body() user: RegisterUserDto) {
    return this.authService.registerUser(user);
  }

  @UseGuards(LocalGuard)
  @Post('login')
  loginUser(@Req() req, @Body() user: LoginUserDto) {
    return req.session;
  }
}

and our serializer

// src/auth/serialization.provider.ts

import { Injectable } from '@nestjs/common';
import { PassportSerializer } from '@nestjs/passport';

import { AuthService } from './auth.service';
import { User } from './models/user.interface';

@Injectable()
export class AuthSerializer extends PassportSerializer {
  constructor(private readonly authService: AuthService) {
    super();
  }
  serializeUser(user: User, done: (err: Error, user: { id: number; role: string }) => void) {
    done(null, { id: user.id, role: user.role });
  }

  deserializeUser(payload: { id: number; role: string }, done: (err: Error, user: Omit<User, 'password'>) => void) {
    const user = this.authService.findById(payload.id);
    done(null, user);
  }
}

along with our module

// src/auth/auth.module.ts

import { Module } from '@nestjs/common';
import { PassportModule } from '@nestjs/passport';

import { AuthController } from './auth.controller';
import { AuthService } from './auth.service';
import { LocalStrategy } from './local.strategy';
import { AuthSerializer } from './serialization.provider';

@Module({
  imports: [
    PassportModule.register({
      session: true,
    }),
  ],
  providers: [AuthService, LocalStrategy, AuthSerializer],
  controllers: [AuthController],
})
export class AuthModule {}

All we need to do for the AuthSerializer is to add it to the providers array. Nest will instantiate it, which will end up calling passport.serializeUser and passport.deserializeUser for us (told you going over that would be useful).

The Guards

So now let's get to our guards, as you'll notice up in the AuthController we're not using AuthGuard('local'), but LocalGuard. The reason for this is because we need to end up calling super.logIn(request), which the AuthGuard has, but doesn't make use of by default. This just ends up calling request.login(user, (err) => done(err ? err : null, null)) for us, which is how the user serialization happens. This is what kicks off the session. I'll repeat that because it's super important. super.logIn(request) is how the user gets a session. To make use of this method, we can set up the LocalGuard as below

// src/local.guard.ts

import { ExecutionContext, Injectable } from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';

@Injectable()
export class LocalGuard extends AuthGuard('local') {
  async canActivate(context: ExecutionContext): Promise<boolean> {
    const result = (await super.canActivate(context)) as boolean;
    await super.logIn(context.switchToHttp().getRequest());
    return result;
  }
}

We have another guard as well, the LoggedInGuard. This guards ends up just calling request.isAuthenticated() which is a method that passport ends up adding to the request object when sessions are in use. We can use this instead of having to have the user pass us the username and password every request, because there will be a cookie with the user's session id on it.

// src/logged-in.guard.ts

import { CanActivate, ExecutionContext, Injectable } from '@nestjs/common';


@Injectable()
export class LoggedInGuard implements CanActivate {
  canActivate(context: ExecutionContext) {
    return context.switchToHttp().getRequest().isAuthenticated();
  }
}

And now we have one other guard for checking if a user is an admin or not.

// src/admin.guard.ts

import { ExecutionContext, Injectable } from '@nestjs/common';

import { LoggedInGuard } from './logged-in.guard';

@Injectable()
export class AdminGuard extends LoggedInGuard {
  canActivate(context: ExecutionContext): boolean {
    const req = context.switchToHttp().getRequest();
    return super.canActivate(context) && req.session.passport.user.role === 'admin';
  }
}

This guard extends our usual LoggedInGuard and checks for the user's role, which is saved in the redis session, via the AuthSerializer we created earlier.

A couple of extra classes

There's a few other classes that I'm making use of. It'll be easiest to view them in the GitHub repo, but I'll add them here if you just want to copy paste:

// src/app.controller.ts

import { Controller, Get, UseGuards } from '@nestjs/common';
import { AdminGuard } from './admin.guard';

import { AppService } from './app.service';
import { LoggedInGuard } from './logged-in.guard';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Get()
  publicRoute() {
    return this.appService.getPublicMessage();
  }

  @UseGuards(LoggedInGuard)
  @Get('protected')
  guardedRoute() {
    return this.appService.getPrivateMessage();
  }

  @UseGuards(AdminGuard)
  @Get('admin')
  getAdminMessage() {
    return this.appService.getAdminMessage();
  }
}

// src/app.service.ts

import { Injectable } from '@nestjs/common';

@Injectable()
export class AppService {
  getPublicMessage(): string {
    return 'This message is public to all!';
  }

  getPrivateMessage(): string {
    return 'You can only see this if you are authenticated';
  }

  getAdminMessage(): string {
    return 'You can only see this if you are an admin';
  }
}

// src/main.ts

import { Logger } from '@nestjs/common';
import { NestFactory } from '@nestjs/core';

import { AppModule } from './app.module';

const bootstrap = async () => {
  const app = await NestFactory.create(AppModule);
  const logger = app.get(Logger);
  await app.listen(3000);
  logger.log(`Application listening at ${await app.getUrl()}`);
};

bootstrap();

Testing out the flow

So now, we can run everything all together and test out the flow. First things first, make sure the Redis instance is running. Without that, the server won't start. Once it's running, run nest start --watch to start the server in dev mode which will recompile and restart on file change. Now it's time to send some curls.

Testing Existing Users

So let's start off with some existing user test. We'll try to log in as Joe Foo.

curl http://localhost:3000/auth/login -d 'email=joefoo@test.com&password=Passw0rd!' -c cookie.joe.txt

If you aren't familiar with curl, the -d make the request a POST, and sends the data as application/x-www-form-urlencoded which Nest accepts by default. The -c tells curl that it should start the cookie engine and save the cookies to a file. If all goes well, you should get a response like

{"cookie":{"originalMaxAge":60000,"expires":"2021-08-16T05:30:51.621Z","httpOnly":false,"path":"/","sameSite":true},"passport":{"user":1}}

Now we can send a request to /protected and get our protected response back

curl http://localhost:3000/protected -b cookie.joe.txt

With -b we are telling curl to use the cookies found in this file.

Now let's check the registration:

curl http://localhost:3000/auth/register -c cookie.new.txt -d 'email=new.email@test.com&password=password&confirmationPassword=password&firstName=New&lastName=Test'

You'll notice that no session was created for the new user, which means they still need to log in. Now let's send that login request

curl http://localhost:3000/auth/login -c cookie.new.txt -d 'email=new.email@test.com&password=password'

And check that we did indeed create a session

curl http://localhost:3000/protected -b cookie.new.txt`

And just like that, we've implemented a session login with NestJS, Redis, and Passport.

To view the session IDs in redis, you can connect the redis-cli to the running instance and run KEYS * to get all of the set keys. By default connect-redis uses sess: as a session key prefix.

Conclusion

Phew, okay, that was definitely a longer article than I had anticipated with a much deeper focus on Nest's integration with Passport, but hopefully it helps paint a picture of how everything ties together. With the above, it should be possible to integrate sessions with any kind of login, basic, local, OAuth2.0, so long as the user object remains the same.

One last thing to note, when using sessions, cookies are a must. The client must be able to work with cookies, otherwise the session will essentially be lost on each request.

If you have any questions, feel free to leave a comment or find me on the NestJS Discord Server

Providing Providers to Dynamic NestJS Modules

Jackie McDoniel — Tue, 27 Jul 2021 17:18:14 +0000

Jay is a member of the NestJS core team, primarily helping out the community on Discord and Github and contributing to various parts of the framework.

If you've been working with NestJS for a while, you've probably heard of dynamic modules. If you're rather new to them, the docs do a pretty good job explaining them, and there's an awesome article by John about them as well that I would highly encourage reading.

I'm going to skip over the basics of dynamic modules, as the above links do a great job of explaining the concepts around them, and I'm going to be jumping into an advanced concept of providing a provider to a dynamic module. Let's unpack that sentence for a moment: we are wanting to call a dynamic module's registration method and provide a service to use instead of the default service the dynamic module already has.

The Use Case

Let's go with the general use case of having a general AuthModule with an AuthService that injects USER_SERVICE, to allow for swapping between database types (mongo, typeorm, neo4j, raw sql, etc). In doing this, we'd be able to publish the authentication module and let anyone make use of the package, while providing their own USER_SERVICE so long as it adheres to the interface we've defined.

The Setup

For this, I'm going to be using a package called @golevelup/nestjs-modules to help with the creation of the dynamic module. Instead of having to set up the entire forRoot and forRootAsync methods, we can extend a mixin and let the package take care of the setup for us. everything in this article will work without the package, I just like using it for the sake of simplicity. So, lets dive into setting up our AuthModule to be a dynamic module. First we need to create our injection token for the options

// src/auth.constants.ts

export const AUTH_OPTIONS = Symbol('AUTH_OPTIONS');
export const AUTH_SECRET = Symbol('AUTH_SECRET');
export const USER_SERVICE = Symbol('USER_SERVICE');

For now, you can ignore the AUTH_SECRET and USER_SERVICE symbols, but we'll need it here in a moment. The next is to set up the AuthModule's options interface

// src/auth.interface.ts

import { UserService } from './user-service.interface';

export interface AuthModuleOptions {
  secret: string;
  userService: UserService;
}

And the UserService interface defined as such:

// src/user-service.interface.ts

interface User {
  id: string;
  name: string;
  email: string;
}

export interface UserService {
  find: (id: string) => User;
  insert: (user: Exclude<User, 'id'>) => User;
}

Now, to make our AuthModule we can simply use the createConfigurableDynamicModule method like so:

// src/auth.module.ts

import { createConfigurableDynamicRootModule } from '@golevelup/nestjs-modules';
import { Module } from '@nestjs/common';

import { AUTH_OPTIONS } from './auth.constants';
import { AuthModuleOptions } from './auth.interface';
import { AuthService } from './auth.service';

@Module({
  providers: [AuthService],
})
export class AuthModule extends createConfigurableDynamicRootModule<AuthModule, AuthModuleOptions>(AUTH_OPTIONS) {}

And just like that, the module now has a forRoot, a forRootAsync, and a externallyConfigured static method that can all be taken advantage of (for more on the externallyConfigured method, take a look at the package's docs).

The Solution

So now, how do we ensure that users can pass in a UserService of their own, and how does our AuthService make use of it? Well, let's say that we have the following AuthService

// src/auth.service.ts

import { Injectable, Inject } from '@nestjs/common';
import { sign, verify } from 'jsonwebtoken';

import { AUTH_SECRET, USER_SERVICE } from './auth.constants';
import { UserService } from './user-service.interface';

@Injectable()
export class AuthService {
  constructor(
    @Inject(AUTH_SECRET) private readonly secret: string,
    @Inject(USER_SERVICE) private readonly userService: UserService,
  ) {}

  findUser(id: string) {
    return this.userService.find(id);
  }

  signToken(payload: Record<string, any>) {
    return sign(payload, this.secret);
  }

  verifyToken(token: string) {
    return verify(token, this.secret);
  }
}

We have it set up to inject two providers, AUTH_SECRET and USER_SERVICE (told you they'd be needed). So now all we need to do is provide these injection tokens. But how do we do that with a dynamic module? Well, taking the module from above, we can pass in a second parameter to the createConfigurableDynamicModule method and set up providers that should exist inside the module like so

// src/auth.module.with-providers.ts

import { createConfigurableDynamicRootModule } from '@golevelup/nestjs-modules';
import { Module } from '@nestjs/common';

import { AUTH_OPTIONS, AUTH_SECRET, USER_SERVICE } from './auth.constants';
import { AuthModuleOptions } from './auth.interface';
import { AuthService } from './auth.service';

@Module({
  providers: [AuthService],
})
export class AuthModule extends createConfigurableDynamicRootModule<AuthModule, AuthModuleOptions>(AUTH_OPTIONS, {
  providers: [
    {
      provide: AUTH_SECRET,
      inject: [AUTH_OPTIONS],
      useFactory: (options: AuthModuleOptions) => options.secret,
    },
    {
      provide: USER_SERVICE,
      inject: [AUTH_OPTIONS],
      useFactory: (options: AuthModuleOptions) => options.userService,
    },
  ],
}) {}

Using this approach, we are able to make use of the options passed in at the forRoot/forRootAsync level, while still allowing for injection of separate providers in the AuthService. Making use of this AuthModule would look something like the below:

// src/app.module.ts

import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';

import { AuthModule } from './auth';
import { UserModule, UserService } from './user';

@Module({
  imports: [
    AuthModule.forRootAsync({
      imports: [ConfigModule, UserModule],
      inject: [ConfigService, UserService],
      useFactory: (config: ConfigService, userService: UserService) => {
        return {
          secret: config.get('AUTH_SECRET_VALUE'),
          userService,
        };
      },
    }),
  ],
})
export class AppModule {}

This approach can work in many different contexts, and allows for a nice separation of all the options into smaller, more injectable sub-sets of options, which is what I do in my OgmaModule.

Introducing nest-commander

Jackie McDoniel — Thu, 08 Apr 2021 20:23:02 +0000

Jay is a member of the NestJS core team, primarily helping out the community on Discord and Github and contributing to various parts of the framework.

What's up Nestlings! So you've been developing web servers with NestJS for a while now, and you're in love with the DI context that it brings. You're enthralled by the class first approach, and you have "nest-itis" where you just want everything to work with Nest because, well, it just feels good. But now, after building some web servers and application, you realize you need to be able to run some one-off methods, or maybe you want to provide a full CLI package for one of your projects. You could write these commands using shell scripts (and I wish you the best if so), or you could use a CLI package like yargs or commander. Those are all fine options, but none of them are using Nest, which is sad for those of you with "nest-itis". So what do you do? You go to Google, type in "nest commander package" and search through the results, finding nest-commander. And that's where you see the light.

What sets nest-commander apart from other Nest CLI packages

`nestjs-command`

uses yargs as the underlying CLI engine
uses parameter decorators for command
has a CommandModuleTest built into the package

`nestjs-console`

uses commander for the underlying CLI engine
has a ConsoleService to create commands or can use decorator
no immediate testing integration

`nest-commander`

uses commander for the underlying CLI engine
uses a decorator on a class for a command and a decorator on class methods for command options
has a separate testing package to not bloat the final CLI package size
has an InquirerService to integrate inquirer into your CLI application
has a CommandFactory to run similar to NestFactory for a familiar DX

How to use it

Okay, so we've talked about what's different, but let's see how we can actually write a command using nest-commander.

Let's say we want to create a CLI that takes in a name, and an age and outputs a greeting. Our CLI will have the inputs of -n personName and -a age. In commander itself, this would look something like

const program = new Command();
program
  .option('-n <personName>')
  .option('-a <age>');
program.parse(process.argv);
const options = program.options();
options.age = Number.parseInt(options.age, 10);
if (options.age < 13) {
  console.log(`Hello ${options.personName}, you're still rather young!`);
} else if (12 < options.age && options.age < 50) {
  console.log(`Hello ${options.personName}, you're in the prime of your life!`);
} else {
  console.log(`Hello ${options.personName}, getting up there in age, huh? Well, you're only as young as you feel!`);
}

This works out well, and it pretty easy to run, but as your program grows it may be difficult to keep all of the logic clean and separated. Plus, in some cases you may need to re-instantiate services that Nest already manages for you. So enter, the @Command() decorator and the CommandRunner interface.

All nest-commander commands implement the CommandRunner interface, which says that every @Command() will have an async run(inputs: string[], options?: Record<string, any>): Promise<void> method. inputs are values that are passed directly to the command, as defined by the arguments property of the @Command() decorator. options are the options passed for the command that correlate back to each @Option() decorator. The above command could be written with nest-commander like so

@Command({ name: 'sayHello', options: { isDefault: true }})
export class SayHelloCommand implements CommandRunner {
  async run(inputs: string[], options: { name: string, age: number }): Promise<void> {
    if (options.age < 13) {
      console.log(`Hello ${options.personName}, you're still rather young!`);
    } else if (12 < options.age && options.age < 50) {
      console.log(`Hello ${options.personName}, you're in the prime of your life!`);
    } else {
      console.log(`Hello ${options.personName}, getting up there in age, huh? Well, you're only as young as you feel!`);
    }
  }

  @Option({ flags: '-n <personName>' })
  parseName(val: string) {
    return val;
  }

  @Option({ flags: '-a <age>' })
  parseAge(val: string) {
    return Number.parseInt(val, 10);
  }
}

Now all we need to do is add the SayHelloCommand to the Nest application and make use of CommandFactory in our main.ts.

// src/say-hello.module.ts
@Module({
  providers: [SayHelloCommand],
})
export class SayHelloModule {}

// src/main.ts
import { CommandFactory } from 'nest-commander';
import { SayHelloModule } from './say-hello.module';

async function bootstrap() {
  await CommandFactory.run(SayHelloModule);
}
bootstrap();

And there you have it, the command is fully operational. If you end up forgetting to pass in an option, commander will inform you the call is invalid. To run the command from here, compile your code as you normally would (either using nest build or tsc) and then run node dist/main.

Now, this is all fine and dandy, but the real magic, as mentioned before, is that all of Nest's DI context still works! So long as you are using singleton or transient providers, there's no limitation to what the CommandFactory can manage.

InquirerService

So now what? You've got this fancy CLI application and it runs awesome, but what about when you want to get user input during runtime, not just when starting the application. Well, that's where the InquirerService comes in. The first thing that needs to happen is a class with @QuestionSet() needs to be created. This will be the class that holds the questions for the named set. The name is important as it will be used in the InquirerService later. Say that we want to get the name and age at runtime or at start time, first we need to change the options to optional by changing from chevrons to brackets (i.e. <personName> to [personName]). Next, we need to create our question set

@QuestionSet({ name: 'personInfo' })
export class PersonInfoQuestions {
  @Question({
    type: 'input',
    name: 'personName',
    message: 'What is your name?'
  })
  parseName(val: string) {
    return val;
  }

  @Question({
    type: 'input',
    name: 'age',
    message: 'How old are you?'
  })
  parseAge(val: string) {
    return Number.parseInt(val, 10);
  }
}

Now in the SayHelloCommand we need to add in the InquirerService and ask for the information.

@Command({ name: 'sayHello', options: { isDefault: true })
export class SayHelloCommand implements CommandRunner {
  constructor(private readonly inquirerService: InquirerService) {}

  async run(inputs: string[], options: { personName?: string, age?: number }): Promise<void> {
    options = await this.inquirerService.ask('personInfo', options);
    if (options.age < 13) {
      console.log(`Hello ${options.personName}, you're still rather young!`);
    } else if (12 < options.age && options.age < 50) {
      console.log(`Hello ${options.personName}, you're in the prime of your life!`);
    } else {
      console.log(`Hello ${options.personName}, getting up there in age, huh? Well, you're only as young as you feel!`);
    }
  }
...

The rest of the class follows as above. Now we can pass in the options commander already found, and inquirer will skip over asking for them again, allowing for the a great UX by not having to duplicate their information (now if only resume services were so nice). Now in SayHelloModule we add in the PersonInfoQuestions to the providers and everything else just works ™️

@Module({
  providers: [
    SayHelloCommand,
    PersonInfoQuestions,
  ]
})
export class SayHelloModule {}

And just like that, we've now created a command line application using nest-commander, allowing for users to pass the info in via flags or using prompts and asking for it at runtime.

For more information on the project you can check the repo here. There's also a testing package to help with testing both the commander input and the inquirer input. Feel free to raise any issues or use the #nest-commander channel on the official NestJS Discord

Using the NestJS CLI GraphQL Plugin for E2E Testing

Jackie McDoniel — Wed, 09 Sep 2020 19:28:24 +0000

Jay is a member of the NestJS core team, primarily helping out the community on Discord and Github and contributing to various parts of the framework.

If you've been using NestJS for GraphQL, you probably know about the sweet @nestjs/cli plugin to auto-annotate all of your DTOs with GraphQL metadata, meaning you can cut down on code and make your development cycle faster (and let's be honest, who doesn't want that?). So you've written your code, using that awesome plugin, your unit tests are passing, you fire up your e2e tests and everything fails. GraphQL errors left and right about missing metadata, jest complaining about failures, and everything right in the world is now all so very wrong. Why did this happen, and how can we fix it?

Why did this happen?

To understand the why of this, we need to figure out what all Nest's build command does and how it works from a very high level. Below is a short list of what the build command is responsible for

compile code from ts to js
map paths properly for us (if using path aliases)
annotate DTOs with swagger decorators (if applied)
annotates DTOs with GraphQL decorators (if applied)
set up an automatic watcher to recompile on change (if using start --watch)

Now all of this looks really sweet compared to the standard Typescript compiler. In fact, Nest uses the Typescript compiler under the hood by default (though this can be swapped out for Webpack if you choose to do so). What's important to note here though, is that this is more than what tsc does by default. And because of all this, when we try to run our e2e tests through ts-jest, the default tsc compiler just isn't enough.

Okay, so how do I fix it?

The way I see it, there's two main options:

1) Create a ts-jest compiler plugin that can be swapped out with typescript
2) Create a process for building your test files specifically for e2e testing

While the first option sounds great, there's a couple of problems with it. It's specific to ts-jest so if anyone wanted to use something like mocha a whole new plugin would need to be written, and it's difficult to write, because Nest's build command is a wrap around of tsc, not an entirely new compiler in the first place.

So we go with option two and write a build process for e2e tests.

The Build Process

The first thing that we will need now is a new tsconfig, so that we can get the build to include both our source code and our test code. So, taking the standard setup from a nest new project, we can make a tsconfig.test.json that looks like this

{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "outDir": "dist-test"
  },
  "include": ["src", "test"]
}

This will test nest build to include both the src directory and the test dir, and use a different output directory than our normal build process.

hint: you should probably add the dist-test dir to your .gitignore so it isn't checked into version control

Now we need to update our jest-e2e.json that Nest provides for us. We need to remove the transform property, and change the regex used for finding the tests. The new jest-e2e.json should look something like this:

{
  "moduleFileExtensions": ["js", "json", "ts"],
  "rootDir": ".",
  "testEnvironment": "node",
  "testRegex": ".e2e-spec.js$"
}

Now the last thing to do is to create a build script and a test script to build the entire source code with tests and move them all to a single directory for tests. We can make use of pre scripts here and make two new scripts in the package.json that look like this

"pretest:e2e": "nest build -p tsconfig.test.json && cp ./test/jest-e2e.json ./dist-test/test/",
"test:e2e": "jest --config ./dist-test/test/jest-e2e.json"

With these commands, when we run npm run test:e2e or yarn test:e2e, we are telling nest to build the project using the new tsconfig.test.json we created, move the jest-e2e.json to the ./dist-test/test/ directory, to act similarly to where it already lives in the ./test/ dir (Typescript won't move non ts files by default), and then for jest (not ts-jest) to run the tests based on the config file.

To see a running version of this you can view my sample repository here.

If you have questions or comments let me know. And if you want to discuss more about NestJS in general feel free to check out our Discord.