Forem: ismaeil-shajar

Create a multi-tenancy application in Nest.js Part 4 (Authentication and Authorization setup)

ismaeil-shajar — Wed, 02 Mar 2022 17:05:53 +0000

Introduction

Authentication & Authorization are essential parts of most applications. There are many different approaches and strategies to handle authentication. This article presents the same like production that can be adapted.

we will use Nest.js to build the following :

user need login using /login endpoint
after login will get token.
any user has roles and privileges.
users can just access endpoints allowed their roles and privileges

we will follow these steps :

we need multiple databases one of them is the default database what we will do all authentication in it and then thw database for every tenant and will get Authorization privileges and policies from it
in the default database we need to create the Users table and Organization table that present tenants.
in tenant databases will add privileg_group table containing all tenant users' policies.
will add an endpoint in the user-service app to manage users and privileges.
create Auth module and export it in an external library
add guards for JWT, Roles and policies

Authentication

to add we will start from models and edit user entity and create needed service methods and controllers

Create User & Login

In user-service app

here in the default database we need a table to save tenants and their Organizations information so create an Organizations entity

@Table({tableName:'Organizations'})
export class Organizations extends Model<Organizations> {

  @Column({
    type: DataType.STRING,
    allowNull: false,
    unique: true,
  })
  name: string
  @Column({
    type: DataType.STRING,
    allowNull: false,
    unique: true,
  })
  domain: string

  @HasMany(()=>Users,'organizationId')
  users: Users[]

}

and create a relation in users table by edit user entity


@Table({tableName:'Users'})
export class Users extends Model<Users> {
    /*....*/
    @ForeignKey(() => Organizations)
    @Column
    organizationId: number;
}

now we can create usermodule users.module.ts and work in it to export related providers and imports from main user app microservice.

when I try to do that by simply removing SequelizeModule.forFeature from main module and adding it in UsersModule I get an error in dependency injection so I need to remove a name from SequelizeModuleAsyncOptions name: 'development' from SequelizeModule in the main module.

for testing in local will edit database connection in sequelize-config-service.ts but in production will implement another story

/**/
if(this.request.data ){
        const host=this.request.data['host']
        update domain=host?host.split('.'):"127.0.0.1".split('.');
        //...
}
/*...*/

now users.module.ts will be

@Module({
  imports: [
    SequelizeModule.forFeature([Users,Organizations]),
  ],
  controllers: [],
  providers: [],
  exports: [],
})
export class UsersModule {}

and edit user-service module

@Module({
   imports: [
  UsersModule,
  SequelizeModule.forRootAsync({
    imports:[UserConfigModule],
    useExisting: SequelizeConfigService,
  })],
  controllers: [UserServiceController],
  providers: [UserServiceService],
})

export class UserServiceModule {}

and now create UserService to add all service methods we will use in endpoints.
UserService contains:

dependency inject User and Organizations repository
method findOne to return username in our case we can use just email
methods to create and update Users and Organizations.
use bcrypt to save passwords

@Injectable()
export class UsersService {
    constructor(
        @InjectModel(Users)    private readonly userModel: typeof Users,
        @InjectModel(Organizations) private readonly organizationsModel:typeof Organizations) {
    }

    async findOne(username: string): Promise<Users> {

      return this.userModel.findOne({where:{email:username},nest: true}) ;
      }    

    async create( createUserDto:CreateUserDto):Promise<Users> {
      const hashpassword= await bcrypt.hash(createUserDto.password,10)
      createUserDto.password=hashpassword;
      return this.userModel.create(<Users>createUserDto)
    }

      async findId(id): Promise<Users> {
        return this.userModel.findOne({where:{id:id},raw: true,nest: true}) ;
      }

      async createOrg(createAccountDto: CreateAccountDto){
        let {owner,...orgDto} = createAccountDto
        return this.organizationsModel.create(<Organizations>orgDto);
      }
      async getOrg(name: string){
        return this.organizationsModel.findOne({where:{name:name}})
      }  
      async getOrgById(id: any){
        return this.organizationsModel.findByPk(id)
      }
      async findUserById(id: any): Promise<Users> {
        return this.userModel.findOne({
          attributes: {exclude: ['password']},
          where: { id: id },
          nest: true,
        });
      }
      async findUserAll(query:any): Promise<Users[]> {
        if(query.type && query.type=='admin'){
          query.type={[Op.in]:['tenant','admin']}
        }
        console.log('users where',query)
        return this.userModel.findAll({ attributes: {exclude: ['password']}, where:query, nest: true });
      }

      async update( id:any,userDto:any) {
        return this.userModel.update(userDto,{where:{id:id}})
      }
}

create user controller and will use normal REST controller and also nest microservice pattern @MessagePattern for controllers needed by other microservice like @MessagePattern('findOne')

@Controller()
export class UsersController {
    constructor(private readonly usersService:UsersService) {}
  @Get('user/:id')
  async getUserById(@Param('id')id:any){
    return this.usersService.findUserById(id);
  }

  @Get('user/name/:name')
  async eventGetUserById(@Param('name') username:any):Promise<any>{
    console.log("Logger we arrive",username)
    return this.usersService.findOne(username);
  }

  @MessagePattern('findOne')
  async getUserByName(data:any){
    console.log("Logger we arrive",data.username)
    return this.usersService.findOne(data.username);
  }
  @Get('user')
  async getUsers(@Query() query:any){
    return this.usersService.findUserAll(query);
  }

  @Put('user/:id')
  async updateUser(@Param('id') id:any,@Body() dto:any){
    return this.usersService.update(id,dto);
  }

  @Post('user')
  async createUser(@Body() userDto:CreateUserDto) {
    return  this.usersService.create(userDto)
   }
}

New auth-service app

this new service (app) will contain a login endpoint and also will test all authentication & authorization in its controller endpoint

now we create auth-service

nest g app auth-service

in the main service will add two methods login will just return request body and getHello

getHello will use it in testing

@Injectable()
export class AuthServiceService {
  getHello(): string {
    return 'Hello World!';
  }

  login(user:UserDto):UserDto {
    return user;
  }
}

create an endpoint for login in the controller and will let Post request in path "/login"


@Controller()
export class AuthServiceController {
  constructor(private readonly authService: AuthServiceService) {}

  @Post('login')
  async login(@Body() user) {
    return this.authService.login(user);
  }
}

now edit port in main.ts to available port in my case 3001 and test login
npm run start:dev auth-service

curl -X post http://127.0.0.1:3003/user -d '{"firstName":"mhd","lastName":"shajar", "email":"mhd@shajar.com","password":"123456","type":"admin"}'

curl -X post http://127.0.0.1:3001/login -d '{ "username":"mhd@shajar.com", "password":"123456"}'

Guard, Password and Local Strategy

as we said we will use password library to implement authentication so need install some devendency to use them

$ npm i --save @nestjs/passport passport-jwt passport-local
$ npm i --save-dev @types/passport-local
$ npm i --save bcrypt

and to implement we create a validation service that has single method called validateUser that will return true if the login credential is valid and false else.
to implement, this function we need to call controllers from user-service and will use ClientProxy to send a request to user-service findOne endpoint, after return user data will check password and return true if a valid password for user.

@Injectable()
export class AuthValidateService {
    constructor(
        @Inject('User_SERVICE') private readonly userServiceClient: ClientProxy
      ) {}

    validateUser(username: string, pass: string): Observable<UserDto>|null {

        return this.userServiceClient.send('findOne',
        {username:username})
        .pipe(map((user:UserDto)=>{ 

          if (user && bcrypt.compareSync(pass, user.password)) {
            const { password, ...result } = user;
            return result;
          }
          return null;}),
          catchError(err => {
            console.log('caught  error and return null ', err);
            return  of(null);
        })
          )
      } 
}

to use ClientProxy will edit auth-service module and add ClientsModule to access other microservices also will add AuthValidateService as providers

@Module({
  imports: [PassportModule,
    ClientsModule.register([
      {
        name: 'User_SERVICE',
        transport: Transport.REDIS,
        options: {
          url: 'redis://localhost:6379',
        },
      },
    ])
  ],
  controllers: [AuthServiceController],
  providers: [AuthServiceService,
    {provide:'AUTH_SERVICE',
    useClass:AuthValidateService
  },
    LocalStrategy, JwtStrategy],
})
export class AuthServiceModule {}

now add guards localguards which is just AuthGuard('local') from nest.js password library

@Injectable()
export class LocalAuthGuard extends AuthGuard('local') {
}

and about adding strategy will use PassportStrategy in our LocalStrategy and use validateUser from AuthValidateService to validate.

@Injectable()
export class LocalStrategy extends PassportStrategy(Strategy) {

  constructor(@Inject('AUTH_SERVICE') private authService ) {
    super({
      passReqToCallback: true,
    });
  }

  async validate(  request: Request,username: string, password: string): Promise<any> {
    const user = await firstValueFrom(this.authService.validateUser(username, password),{defaultValue:null});
    console.log('username',user);
    if (!user) {
      throw new UnauthorizedException();
    }
    return user;
  }
}

last edit in "/login" endpoint will add Guards LocalAuthGuard and nest will inject LocalStrategy using names.

  @UseGuards(LocalAuthGuard)
  @Post('login')
  async login(@Request() req) {
    return this.authService.login(req.user);
  }

done and now can test

curl -X post http://127.0.0.1:3003/user -d '{"firstName":"ismaeil","lastName":"shajar", "email":"ismaeil@shajar.com","password":"123456","type":"admin"}'

curl -X post http://127.0.0.1:3001/login -d '{ "username":"ismaeil@shajar.com", "password":"123456"}'

JWT token

and to create and decode token will install ready nest.js library @nestjs/jwt

$ npm i --save @nestjs/jwt

add jwt in main auth-service module

    JwtModule.register({
      secret: jwtConstants.secret,
      signOptions: { expiresIn: '6000s' },
    })

in auth service update login to return access token

@Injectable()
export class AuthServiceService {
  constructor(
    private readonly jwtService: JwtService,
  ) {}
  login(user:UserDto):AccessToken {
    const payload = { username: user.firstName,orgId:user.organizationId,sub: user.id};
    return {
      access_token: this.jwtService.sign(payload),
    }
  }
}

same as local guard will create JwtAuthGuard which just extends AuthGuard('jwt') from password library

@Injectable()
export class JwtAuthGuard extends AuthGuard('jwt') {
}

and also create JwtStrategy extends from PassportStrategy to implement validate function from it.
for more details can check nest Doc

@Injectable()
export class JwtStrategy extends PassportStrategy(Strategy) {
  constructor() {

    super({
      jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),
      ignoreExpiration: false,
      secretOrKey: jwtConstants.secret,
      passReqToCallback: true
    });
  }


  async validate( request: Request,payload: any) {
    const user={ userId: payload.sub, username: payload.username,roles:['admin'] }
    return user;
  }
}

finally, add jwtGuard in "/hello" endpoint

  @Get('hello')
  @UseGuards(JwtAuthGuard)
  getHello(@Request() req){
    console.log("hello",req.user);
    return "Hello";
  }

now we can test get http://127.0.0.1:3001/hello and use header Authorization with "Bearer ${token}"

Microservice Auth

I try to secure connections between microservices we can add a new function where we call Redis.

now AuthValidateService will be

@Injectable()
export class AuthValidateService {
    constructor(
        @Inject('User_SERVICE') private readonly userServiceClient: ClientProxy
      ) {}

    validateUser(username: string, pass: string): Observable<UserDto>|null {

        return this.userServiceClient.send('findOne',
       this.addMicroserviceToken({username:username}))
        .pipe(map((user:UserDto)=>{ 

          if (user && bcrypt.compareSync(pass, user.password)) {
            const { password, ...result } = user;
            return result;
          }
          return null;}),
          catchError(err => {
            console.log('caught  error and return null ', err);
            return  of(null);
        })
          )

      } 

    private addMicroserviceToken(data:any): any{
      data.token=bcrypt.hashSync("secret",10) ;
      return data;
  }
}

Export to the custom library

to use JWT Guard in all our app and not need to rewrite it we can create Library that manages authentication implementation

Collect to AuthModule

we start by creating AuthModule and every module or provider that is just used for authentication will remove it from the main module to the new AuthModule.
now create AuthModule and will use the dynamic module to make it configurable to receive Validation Service , redis url , jwt security key and ClientsModule name.
so will import all PassportModule, ClientsModule, JwtModule
and export LocalStrategy, JwtStrategy,JwtModule to use them outside module

@Module({})
export class AuthModule {
  static register(validationService,options?:{jwtsecret:string,
    userServiceClientProvider:string,
    redisUrl:string

  }): DynamicModule {
    return {
      module: AuthModule,
      imports:[ 
        PassportModule,
        ClientsModule.register([
          {
            name: options.userServiceClientProvider,
            transport: Transport.REDIS,
            options: {
              url: options.redisUrl,
            },
          },
          ]),        
        JwtModule.register({
          secret: options?options.jwtsecret:"secretKey",
          signOptions: { expiresIn: '6000s' },
          })
      ],
      providers: [
        {
        provide:'JWTSECRET',
        useValue:options?options.jwtsecret:"secretKey"
        }
        ,validationService ,LocalStrategy, JwtStrategy],
      exports: [LocalStrategy, JwtStrategy,JwtModule],
    };
  }
}

no will edit jwtStratigy to pass security key from outside module

  constructor(@Inject('JWTSECRET') private jwtsecrit) {

    super({
      jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),
      ignoreExpiration: false,
      secretOrKey: jwtsecrit,
      passReqToCallback: true
    });
  }

Clean App Module

remove all not needed modules and export them from AuthModule

module will be

remove all from auth-service module and will be

@Module({
  imports: [
  AuthModule.register(
    {
    provide: 'AUTH_SERVICE',
    useClass:AuthValidateService
    }
  ,
    {
    userServiceClientProvider:'User_SERVICE',
    redisUrl:'redis://localhost:6379',
    jwtsecret:jwtConstants.secret
    }
  )],
  controllers: [AuthServiceController],
  providers: [AuthServiceService],
})
export class AuthServiceModule {}

and test again to check every function works.

Create Library

create library
nest g library auth-lib

and now move all AuthModule to lib and we just edit auth-lib.module.ts and will add will be same like authModule

@Module({})
export class AuthLibModule {
  static register(validationService,options?:{jwtsecret:string,
    userServiceClientProvider:string,
    redisUrl:string

  }): DynamicModule {
    return {
      module: AuthLibModule,
      imports:[ 
        PassportModule,
        ClientsModule.register([
          {
            name: options.userServiceClientProvider,
            transport: Transport.REDIS,
            options: {
              url: options.redisUrl,
            },
          },
          ]),        
        JwtModule.register({
          secret: options?options.jwtsecret:"secretKey",
          signOptions: { expiresIn: '6000s' },
          })
      ],
      providers: [
        {
        provide:'JWTSECRET',
        useValue:options?options.jwtsecret:"secretKey"
        }
        ,validationService ,LocalStrategy, JwtStrategy],
      exports: [LocalStrategy, JwtStrategy,JwtModule],
    };
  }
}

last steps export guards in index.ts

export * from './auth-lib.module';
export {JwtAuthGuard} from './auth/guards/jwt-auth.guard';
export {LocalAuthGuard} from './auth/guards/local-auth.guard';

Clean all modules

I remove all authModule, strategies, and guards
and replace them by the custom library and test everything work

now everything works
the library will be

Authentication

I will write nest.js definition "Authorization refers to the process that determines what a user can do. For example, an administrative user is allowed to create, edit, and delete posts. A non-administrative user is only authorized to read the posts."

RolesGuard / user type

now will start create check for Roles or user types .
I will start directly write needed in lib
start from basic roles and will use Enum to create .

export enum Role {
    User = 'user',
    Admin = 'admin',
  }

and to pass roles in method will use anotation by create Roles decorator

import { SetMetadata } from '@nestjs/common';
import { Role } from './role.enum';

export const ROLES_KEY = 'roles';
export const Roles = (...roles: Role[]) => SetMetadata(ROLES_KEY, roles);

and finally, create RouleGuard with basic verification by implementing CanActivate and using refactor to get @Role annotation set value from method to check user data and required method role

@Injectable()
export class RolesGuard implements CanActivate {
  constructor(private reflector: Reflector) {}

  canActivate(context: ExecutionContext): boolean {
    const roles = this.reflector.get<string[]>(ROLES_KEY, context.getHandler());
    if (!roles) {
         return true;
    }
    const {user} = context.switchToHttp().getRequest();
    return this.matchRoles(roles,user['roles'])
  }
  matchRoles(roles:string[],userRoles:Role[]){
      return roles.filter(role=>!userRoles.includes).length==0;
  }
}

Policies and Privilege groups

Here we have multiple levels of users and we need more advanced RBAC implementation, to do that we need to edit user-service by adding entity save every user policy
First, create privileg-group table in a database

 @Table({tableName:'Privilege-group'})
export class PrivilegeGroup extends Model<PrivilegeGroup> {

    @Column( {
      unique: true,
        allowNull: false
      })
    groupName: string;

    @Column(DataType.STRING)
    roles: string;//Policy[]

    @BelongsToMany(() => Users, () => GroupUser)
    users: Users[] 

}

and will use many to many relation by connect two table using groupUser table

 @Table
export class GroupUser extends Model {
  @ForeignKey(() => Users)
  @Column
  userId: number

  @ForeignKey(() => PrivilegeGroup)
  @Column
  privilegeGroupId: number
}

will create type Policy to recieve authorities json

import { ActionEnum } from "./action-enum";

export class Policy{
    action: ActionEnum[];
    resources: string;
}

and we have Action enum contain all actions like read, write, and delete

export enum ActionEnum {
    Manage = 'manage',
    Create = 'create',
    Read = 'read',
    Update = 'update',
    Delete = 'delete',
  }

will add new decorator PrivilegService to access privileg-group repository to create and get privilegs

@Injectable()
export class PrivilegeService {
    constructor(
        @InjectModel(PrivilegeGroup)    private readonly privilegeModel: typeof PrivilegeGroup,
        @InjectModel(GroupUser) private readonly groupUserModel: typeof GroupUser
    ){}

    getGroupList() { return this.privilegeModel.findAll(); }

    getGroup(id: any) { return this.privilegeModel.findOne({where:{id:id}});}

    async createGroup(createGroup:CreateGroup):Promise<PrivilegeGroup>{
        const createGroupDto=new CreateGroupDto();
        createGroupDto.groupName=createGroup.groupName;
        createGroupDto.roles=JSON.stringify(createGroup.roles);
        return this.privilegeModel.create(<PrivilegeGroup>createGroupDto)
    }

    async addUserGroup(createGroupUserDto: CreateGroupUserDto[]):Promise<GroupUser[]>{
        return this.groupUserModel.bulkCreate(createGroupUserDto);
    }

    async deleteUserGroup(userId: any,privilegeGroupId:any) {
        return this.groupUserModel.destroy({where:{userId:userId,privilegeGroupId:privilegeGroupId}})
      }
    getUserGroup(id: any) {
        return this.groupUserModel.findAll({where:{userId:id}})
      }

}

and create endpoints in privileg controller

@Controller("privilege")
export class PrivilegeController {
    constructor(private readonly privilegeService:PrivilegeService){}

    @Get('/')
    getGroupList() { return this.privilegeService.getGroupList()}
    @Get('/:id')
    getGroup(@Param('id')id: number) { return this.privilegeService.getGroup(id);}

    @Post('/add')
    async createGroup(@Body() createGroup:CreateGroup){
        return this.privilegeService.createGroup(createGroup)
    }
    @Post('/group-users/save')
    async addUserGroup(@Body() createGroupUserDtoList: CreateGroupUserDto[]){
        return this.privilegeService.addUserGroup(createGroupUserDtoList);
    }
    @Delete("/group-users/delete")
    async deleteUserGroup(@Query('userId') userId: number,@Query('privilegeGroupId') privilegeGroupId:number) {
        return this.privilegeService.deleteUserGroup(userId,privilegeGroupId);
      }


    @Get('/group-users/user/:userId')
    getUserGroup(@Param('userId')userId: number) {
        return this.privilegeService.getUserGroup(userId);
      }
}

last will edit user-service to get user will privileges

@Injectable()
export class UsersService {
/// the rest..
    async findOne(username: string): Promise<Users> {
      return this.userModel.findOne({include:[{model:PrivilegeGroup , attributes:["roles"],through: {attributes: []}}],where:{email:username},nest: true}) ;
    }    
  /// the rest..
  }

Edit JWT Token

and roles in token add authorities in token

use decorator and guard

custom decorator

import { SetMetadata } from "@nestjs/common";
import { Policy } from "./policy";

export const CHECK_POLICIES_KEY = 'check_policy';
export const CheckPolicies = (...policy:Policy[]) =>
  SetMetadata(CHECK_POLICIES_KEY, policy);

policies Guards

@Injectable()
export class PoliciesGuard implements CanActivate {
  constructor(
    private reflector: Reflector
  ) {}
  canActivate(context: ExecutionContext): boolean {

    const policies =this.reflector.getAllAndOverride<Policy[]>(
      CHECK_POLICIES_KEY,
      [context.getHandler(),context.getClass()]) || [];

  const {user} = context.switchToHttp().getRequest();

  try{
      const isAuthUser = this.checkPolicies(user['policies'].map(policy => JSON.parse(policy)), policies);
      return isAuthUser;
    }catch(err){
      return false;
    }
  }


  private checkPolicies(userPolicies: Policy[], policies: Policy[]) {
    const isAuthUser = policies.filter(resourcePolicy => {
      return userPolicies.filter(
        policy => this.isAutherize(resourcePolicy, policy)
      ).length >= 1;
    }).length;
    return isAuthUser>=1;
  }

  private isAutherize(resourcePolicy: Policy,  policy: Policy): unknown {
    return this.hasResource(resourcePolicy, policy.resources) && this.hasAction(resourcePolicy, policy);
  }

  private hasResource(resourcePolicy: Policy, resource: string) {
    return resource=='*' || resourcePolicy.resources.indexOf(resource) == 0;
  }

  private hasAction(resourcePolicy: Policy, policy: Policy): unknown {
    return resourcePolicy.action.filter(action => !policy.action.includes(action)).length == 0;
  }
}

Test

simple test in auth-service controller will add endpoint called hello

  @Get('hello')
  @Roles(Role.Admin)
  @UseGuards(JwtAuthGuard,RolesGuard,PoliciesGuard)
  @CheckPolicies({action:[ActionEnum.Read],resources:'hello'})
  getHello(@Request() req){
    return "Hello";
  }

in UseGuards we can add multiple guards to execute them in orders so we can pass variables between guards using context request
not to test we run /login then call get request /hello

Conclusion

final version is Github roles-policies branch

Create a multi-tenant application in Nest.js Part 3 (multi-database setup)

ismaeil-shajar — Sat, 13 Nov 2021 14:11:35 +0000

In part 1, we set up the nestjs framework and configured and tested microservices architecture application using nest.js. In part 2, we used Sequelize and mongoose to access the database and tested for both MySQL database and MongoDB.

Async Connection

In this part; we will see how to let the application connect to multiple databases depending on the request. Since it is a multi-tenancy application, each tenant has their own database containing their data accessing the same application, thus the application needs to connect to different databases.
We will change the pass repository option method and use forRootAsync() instead of forRoot(), we need to use a custom class for configuration.

For both sequelize and mongoose, add this:

MongooseModule.forRootAsync({
    useClass:MongooseConfigService
  }),
SequelizeModule.forRootAsync({
      useClass:SequelizeConfigService
})

We will create a config file and two classes: MongooseConfigService and SequelizeConfigService

The plan here is to add injection for each incoming request and use a domain to switch between connections.

So we need to use @Injectable({scope:Scope.REQUEST}), and on the class constructor we use @Inject(REQUEST) private read-only request so we can get host information from request data.

For example let say the domain is example.com, and we have a tenant called company1, then the domain will be company1.example.com. Same thing for company2 tenant, the domain will be company2.example.com and so on.

config/MongooseConfigService.ts

import { Inject, Injectable, Scope } from "@nestjs/common";
import { REQUEST } from "@nestjs/core";
import { MongooseModuleOptions, MongooseOptionsFactory } from "@nestjs/mongoose";

@Injectable({scope:Scope.REQUEST})
export class MongooseConfigService implements MongooseOptionsFactory {
    constructor(@Inject(REQUEST) private readonly request,){}

  createMongooseOptions(): MongooseModuleOptions {
    let domain:string[]
    let database='database_development'
    if(this.request.data ){
      domain=this.request.data['host'].split('.')
      console.log(this.request)
    }
    else{
      domain=this.request['headers']['host'].split('.')
    }

    console.log(domain)
    if(domain[0]!='127' && domain[0]!='www' && domain.length >2){
      database='tenant_'+domain[0]
      console.log('current DB',database)
    }
    return {
      uri: 'mongodb://localhost:27017/'+database,
    };
  }
}

config/SequelizeConfigService.ts

import { Inject, Injectable, Scope } from "@nestjs/common";
import { REQUEST } from "@nestjs/core";
import { CONTEXT, RedisContext, RequestContext } from "@nestjs/microservices";
import { SequelizeModuleOptions, SequelizeOptionsFactory} from "@nestjs/sequelize";

@Injectable({scope:Scope.REQUEST})
export class SequelizeConfigService implements SequelizeOptionsFactory {
    constructor(@Inject(REQUEST) private readonly request:RequestContext){}

    createSequelizeOptions(): SequelizeModuleOptions {

      let domain:string[]
      let database='database_development'
      if(this.request.data ){
        domain=this.request.data['host'].split('.')
        console.log(this.request)
      }
      else{
        domain=this.request['headers']['host'].split('.')
      }

      console.log(domain)
      if(domain[0]!='127' && domain[0]!='www' && domain.length >2){
        database='tenant_'+domain[0]
        console.log('current DB',database)
      }

    return {
      dialect: 'mysql',
      host: 'localhost',
      port: 3306,
      username: 'ismaeil',
      password: 'root',
      database: database,
      autoLoadModels: true,
      synchronize: true,
    };
  }
}

Performance -- Production

in production we need to avoid create connection in every request so we will do some edits in user module and services.

Solution:

Our problem here is connection was create every request and this connection not close and also I don't reuse.
So we can use even close connection or use existing connection in new request or use both and manage when create and when close.

Closing connection

To close connection first wen need to access it.by naming connections we can access connection using @InjectConnection then in service we can close conection every time after finish .
So we need this edits

in user-service.module.ts


     SequelizeModule.forRootAsync({
      name: 'development',
       useClass:SequelizeConfigService
     }),
     SequelizeModule.forFeature([Users], 'development')], // use connection name in forFeature

and in user-service.service.ts

export class UserServiceService {
  constructor(
    @InjectConnection('development') private readonly sequelize: Sequelize, // access connection by name 'development'
    @InjectModel(Users, 'development')
  private readonly userModel: typeof Users){}
  async findAll() {
    let result =await this.userModel.findAll()
    this.sequelize.close() // after  every use will close connection
    return result;
  }
  /// the rest 
}

Use existing connection

In order to prevent the creation of SequelizeConfigService inside SequelizeModule and use a provider imported from a different module, you can use the useExisting syntax.

and we need to create external module that provide sequelize configuration .

create new file named user-config.module.ts


@Module({
  providers: [SequelizeConfigService],
  exports:[SequelizeConfigService]
})

export class UserConfigModule {}

then edit user-service.module.ts

  SequelizeModule.forRootAsync({
    imports:[UserConfigModule],
    useExisting: SequelizeConfigService,
  }),

Use both

if we add ability to use both ways code will be like this

user-service.module.ts

@Module({
   imports: [
  SequelizeModule.forRootAsync({
    imports:[UserConfigModule],
    name: 'development',
    useExisting: SequelizeConfigService,
  }),
    SequelizeModule.forFeature([Users], 'development')],
  controllers: [UserServiceController],
  providers: [UserServiceService],
})

export class UserServiceModule {}

user-service.service.ts

@Injectable()
export class UserServiceService {
  constructor(@InjectConnection('development') private readonly sequelize: Sequelize,
    @InjectModel(Users, 'development')
  private readonly userModel: typeof Users){}
  async findAll() {
    let result =await this.userModel.findAll()
    //console.log(this.sequelize.close())  // optional or you can manage it 
    return result;
  }

  async create( createUserDto:CreateUserDto):Promise<Users> {
    return this.userModel.create(<Users>createUserDto)

  }
}

and we have new module

user-config.module.ts

import { Module } from '@nestjs/common';
import { SequelizeConfigService } from './sequelize-config-service';

@Module({
  providers: [SequelizeConfigService],
  exports:[SequelizeConfigService]
})

export class UserConfigModule {}

Testing

After finishing the configuration, we need to do some work to test it because we need to map our localhost and IP to a domain.
I will try to use two ways to test the application locally but for the production, it will be a configuration in your domain provider.

1- Edit the hosts file in your local machine and edit this file every time you add a tenant

Go to the following file in Linux: /etc/hosts and in windows: c:\windows\system32\drivers\etc\hosts and add

## lines
127.0.0.1   example.com
127.0.0.1   company1.example.com
127.0.0.1   company2.example.com

2- Use local dns

In linux you can install dnsmasq and follow these steps

1- Install dnsmasq in NetworkManager.

2- Add the configuration file sudo nano /etc/NetworkManager/dnsmasq.d/dnsmasq-localhost.conf.

add this line in the file:

address=/.example.com/127.0.0.1

3- Restart the service sudo systemctl reload NetworkManager.

Source code available in git branch multi-database

Next in part 4 we will add a security level and user roles.

Create a multi tenant application in Nest.js Part 2 (database setup using Sequelize & mongoose)

ismaeil-shajar — Wed, 10 Nov 2021 21:26:39 +0000

In part 1, we setup nestjs framework, configured and tested the microservice application using nest.js.

Database

Nest gives us all the tools to work with any SQL and NoSQL database. You have a lot of options, you can also use almost all ORM and libraries in nodejs and typescript, like Sequelize, TypeORM, Prisma, and of course mongoose.

In this application, we will work with MySQL and MongoDB. We will also use the most popular js libraries; Sequelize as ORM for MySQL, and mongoose for MongoDB.

Database integration

Sequelize

To start using sequelize; we first need to install the required dependencies which include @nestjs/sequelize, mysql2 because we will connect to MySQL database and other needed dependencies.

$ npm install --save @nestjs/sequelize sequelize sequelize-typescript mysql2
$ npm install --save-dev @types/sequelize

In the services, we will import SequelizeModule in the main modules to set connection configuration:

ex: app.module.ts

@Module({
  imports: [
    SequelizeModule.forRoot({
      dialect: 'mysql',
      host: 'localhost',
      port: 3306,
      username: 'root',
      password: 'root',
      database: 'test',
      models: [],
    }),
  ],
})

The forRoot() method will include all the configuration properties. You can read more details here.

After configuring the connection, we need to create a table entity.
For example, we can set a user model in the user service (will add the connection in the service too) by creating user.model.ts which will look like:

user.model.ts

/// imports
@Table({tableName:'Users'})
export class Users extends Model<Users> {
    @Column( {allowNull: false })
    firstName: string;

    @Column( {allowNull: false })
    lastName: string;

    @Column( { allowNull: false,unique: true })
    email: string;

    @Column( {allowNull: false})
    password: string;    

    @Column( { allowNull: false})
    type: string;
}

We should also add the dto:

create-user-dto.ts

export class CreateUserDto{
    readonly firstName:string
    readonly lastName:string
   readonly   email:string
   readonly password:string
   readonly type:string
}

And don't forget to add Users in models array in forRoot()

Now let's complete the setup and configuration.
If you don't have a database you need to create an empty table and change Sequelize configuration by adding: autoLoadModels: true, synchronize: true .
Then in the module, you will add the repository by adding
SequelizeModule.forFeature([Users]) in the imports array.
In our case, we use the main module so it will be:

user-service.module.ts

@Module({
  imports: [SequelizeModule.forRoot({
    dialect: 'mysql',
    host: 'localhost',
    port: 3306,
    username: 'ismaeil',
    password: 'root',
    database: 'test',
    autoLoadModels: true,
    synchronize: true,
    models: [Users],
  }),SequelizeModule.forFeature([Users])],
  controllers: [UserServiceController],
  providers: [UserServiceService],
})

And we will edit main service to add findall and create method:

user-service.service.ts

@Injectable()
export class UserServiceService {
  constructor(
    @InjectModel(Users)
  private readonly userModel: typeof Users){}
  async findAll(): Promise<Users[]> {
    return this.userModel.findAll() ;
  }

  async create( createUserDto:CreateUserDto):Promise<Users> {
    return this.userModel.create(<Users>createUserDto)
  }
}

Lastly, edit the controller to enable the use of REST requests to access and edit the database:

user-service.controller.ts

@Controller('users')
export class UserServiceController {
  constructor(private readonly userServiceService: UserServiceService) {}

  @Get()
  async findAll(){
      return this.userServiceService.findAll();
  }

  @Post()
  async createUser(@Body() createUserDto:CreateUserDto){
    return  this.userServiceService.create(createUserDto)
  }

}

Now run the browser and test http://127.0.0.1:3003/users. This should access the database and create a table for the first time and return an empty array.

We can add data using a POST request:

Tips

If we have an existing database and need to import tables with types without a lot of work we can use sequelize-typescript-generator.

You can search about it to see how it works. But here are some simple steps:

1- Download and install npx

2- Create a folder to save output typescript models mkdir models

3- Install sequelize-typescript-generator in your machine: npm install -g sequelize-typescript-generator

4- Install mysql driver: npm install -g mysql2

5- Run the command: npx stg -D mysql -h localhost -p 3306 -d <databaseName> -u <username> -x <password> --indices --case camel --out-dir models --clean

Source code available in git branch database-connection

mongoose

Just like the previous one, we need to install dependencies to use MongoDB in Nest:

$ npm install --save @nestjs/mongoose mongoose

Import MongooseModule into the root Module

@Module({
  imports: [MongooseModule.forRoot('mongodb://localhost:27017/test')],
})

forRoot() accepts the same configuration as mongoose.connect() from the Mongoose package.

We will use the MongoDB database in the notification service. First, we will add forRoot() in the root module and will create a child module called a message to serve notification messages.

The root module will look like this:

notification.module.ts


@Module({
  imports: [MongooseModule.forRoot('mongodb://localhost:27017/test'),
  MessageModule],
  controllers: [NotificationController],
  providers: [NotificationService],
})

The message module files will be as follows:

Because we are using mongoose, we need to create a schema and after that import the repository in a module.

In src/message/schemes we will create message.schema.ts file which will look like this:

export type MessageSchemaDocument = MessageSchema & Document;

@Schema()
export class MessageSchema{
    @Prop()
    name: string    

    @Prop()
    createdAt: Date

    @Prop({type:mongoose.Schema.Types.Mixed})
    data: Record<string, any>
}

export const MessageSchemaSchema = SchemaFactory.createForClass(MessageSchema);

Put the following code in message.module:

message.module.ts

@Module({
  imports: [MongooseModule.forFeature([{name:MessageSchema.name,schema:MessageSchemaSchema}])],
  controllers: [MessageController],
  providers: [MessageService],
})

And put the following methods in the message service:

message.service.ts

@Injectable()
export class MessageService {
    constructor(@InjectModel(MessageSchema.name) private readonly messageModel: Model<MessageSchemaDocument>) {}
    async findAll () {
        return await this.messageModel.find().exec()
    }    
    async create (messageDto:MessageDto) {
        return await this.messageModel.create(messageDto)
    }
}

Create MessageDto:

export class MessageDto {
    readonly name: string    
    readonly createdAt:Date = new Date();
    readonly data?: any
}

For Request mapping:

message.controller.ts

@Controller('message')
export class MessageController {
  constructor(private readonly messagenService: MessageService) {}

  @Get()
  async findAll(){
    return this.messagenService.findAll();
  }

  @Post()
  @UsePipes(new ValidationPipe({ transform: true }))
  async create(@Body() messageDto:MessageDto){
    return this.messagenService.create(messageDto);
  }
}

*Note: Pipes are used in transforming and validating input data, in our case we can use @UsePipes(new ValidationPipe({ transform: true })) to set the empty properties in the Dto with default values. For more details refer to Pipes and validation.

Now you can test using a Post request to the URL http://127.0.0.1:3002/message with body:

    {
        "name":"validation",
        "data":{"message":"testing validation message if it success","status":"valid"}
    }

To retrieve all the records, use the Get request http://127.0.0.1:3002/message

Source code available in git branch mongodb-connection

In part 3, we will complete the database setup to use multiple databases depending on the request header.

Create a multi tenant application in Nest.js Part 1 (microservices setup)

ismaeil-shajar — Wed, 10 Nov 2021 21:20:40 +0000

Overview

In Saas applications; multitenancy is a mode of operation where multiple independent instances share the same environment. In plain English, it is when multiple tenants and businesses use the same Saas application.

Multitenancy Archituecture

I will not cover how to design a multitenancy application in this article, however, you can read about it in detail here: What is Multi-Tenant Architecture? By
Andy Patrizio

From here on, we are going to work on an example about the multi-tenants using multiple Databases and microservices architecture.

What are we going to build?

We will focus on ROLE-BASED ACCESS CONTROL with multi-database and will use two databases -MySQL and MongoDB- for each tenant:

We will use sample web services to test the flow, for example creating users, sending and receiving notifications... etc.

Nest.js

From nest js: nest (NestJS) is a framework for building efficient, scalable Node.js server-side applications. It uses progressive JavaScript, is built with and fully supports TypeScript (yet still enables developers to code in pure JavaScript), and combines elements of OOP (Object Oriented Programming), FP (Functional Programming), and FRP (Functional Reactive Programming).

Installation

You will need to install Node.js, and it must be version 10.13 or higher. I will install v12 LTS and npm. I recommend that you use nvm to install node.js

Setup

You need to install nest cli using npm:

npm install -g @nestjs/cli

then create a new nest.js project:

nest new multi_tenant

If you encountered an error during the installation like:

npm ERR! invalid json response body at https://registry.npmjs.org/ajv reason: Unexpected end of JSON input.

You can use this to fix it:

npm cache clean --force

Microservices setup

Although we can create a monolithic application, it is not usually the cause for mulit-tenancy applications, since they can be quite big, the better -and harder- approach is to use microservices.
Let's start the microservices architecture setup, first install dependency:

npm i --save @nestjs/microservices

We will use Redis as a transport layer so install Redis client using npm:

npm i --save Redis

We also need to install Redis server, I will use docker to install the Redis server:

docker run --name my-redis-container -p 6379:6379 -d redis

Now we need to test the application setup before we create another microservice.

Edit the following:

main.ts

In src/main.ts replace the bootstrap method by:

  const app = await NestFactory.create(AppModule);
  app.connectMicroservice<MicroserviceOptions>({
    transport:Transport.REDIS,
    options:{
      url:'redis://127.0.0.1:6379'
    },
  });

  await app.startAllMicroservices();
  await app.listen(3000);

Creating microservices in an application

We will start with two applications: notifications and user services. Using generate app command in nest cli:

nest g app user-service
nest g app notification

Now the application directory will look like this:

The new service will be created like a new project but will share the same project.json file. We need to change all services' main.ts to work with Redis and change their ports to be unique.

Got to /apps/user-service/src/main.ts and /apps/notification/src/main.ts and add Redis connection and microservice starter>

const app = await NestFactory.create(NotificationModule);

 // new connection
 app.connectMicroservice<MicroserviceOptions>({
    transport:Transport.REDIS,
    options:{
      url:'redis://127.0.0.1:6379'
    },
  });
 await app.startAllMicroservices();
 /// the rest
   await app.listen(<port>); // any service must have diffrent port or no port if not use http service and client

Before we start editing we need to start the services in dev mode using the following command:

npm run start:dev 
npm run start:dev notification

Currently, there is no need to start the user-service.

Edit configuration and controller

To send data between services; first we will start with the needed configuration and controller. To make it simple, we will send two integers to the notification service and return the user name and the summation of the two integers.

In the main service app.module, we need to add a client to send data to the notification.

But what does app.module.ts do? The @Module() decorator provides metadata that Nest makes use of to organize the application structure. For more details, you can visit Nest.js @Module()

Edit the module file to add microservices ClientsModule and configure it:

@Module will look like this:

@Module({
  imports: [ClientsModule.register([
    {
      name: 'NOTIFY_SERVICE',
      transport: Transport.REDIS,
      options: {
        url: 'redis://localhost:6379',
      },
    },
  ])],
  controllers: [AppController],
  providers: [AppService],
})

ClientsModule is a type of module called a dynamic module. This feature enables you to easily create customizable modules that can register and configure providers dynamically and you can read about it here

Now, in the app.service we will add a constructer to inject the transport client and edit the getHello method to send the data:

app.service.ts

  constructor(
      @Inject('NOTIFY_SERVICE') private readonly client: ClientProxy){}
 async getHello(): Promise<string> { // need to use async because we need to wait recieved data

    let recieve= await this.client.send<number>("notify",{user:"Ali",data:{a:1,b:2}}).toPromise();// notify if mapped key will used to in other hand 
     // without toPromise function will return Observable and will not see execute before subscribe so when convert to Promise will recieve data in variable 

    return "\t add 1+2="+recieve;
  }

The transporters support two methods: send() (for request-response messaging) and emit() (for event-driven messaging)

Then in the notification service we will just use it to receive a request and send a response.

In notification.controller.ts.

Add a new controller (let's call it notify) and use the annotition @MessagePattern with a mapped key notify the same key used in send() function.

  @MessagePattern('notify')
  async notify(data:NotifiyData){
    console.log('send')
    Logger.log("notificatoin data"+data.user);
    let a:number=data.data['a'];
    let b:number=data.data['b'];
    console.log(a,b)
    return a+b;
  }

We will add an interface in the same class file to map the received data to an object type:

interface NotifiyData{
  user: string;
  data: object;
}

run

Now run the main and notification services using:

npm run start:dev 
npm run start:dev notification

Go to the browser and open the main service URL http://localhost:3000/. The output will be add 1+2=3

Source code available in git branch microservices-setup

Go to part2 for database connection setup and test