GraphQL subscriptions with Nest: how to publish across multiple running servers

Georgii Rychko — Tue, 24 Sep 2019 23:23:04 +0000

Today we'll learn how to setup GraphQL (GQL) subscriptions using Redis and NestJS.

Prerequisites for this article:

An experience in GraphQL
Some basic knowledge of NestJS (If you don't know what NestJS is, then give it a try and come back after.)
Docker installed on your machine.

You may be asking yourself, "Why we need Redis at all?" A default subscriptions implementation provided by Apollo works out of the box just fine, right?

Well, it depends. When your server has a single instance, you don't need Redis.

But, when you scale the app and spawn extra instances of your server, you need to be sure that events published on one instance will be received by subscribers on the other. This is something default subscription cannot do for you.

So, let's start by building a basic GQL application with the default (in-memory) subscriptions.

First, install @nestjs/cli:

npm i -g @nestjs/cli

Then create a new NestJS project:

nest new nestjs-gql-redis-subscriptions

Now, open the nestjs-gql-redis-subscriptions/src/main.ts and change

await app.listen(3000);

to:

await app.listen(process.env.PORT || 3000);

This allows us to specify a port, when needed, through an env var.

NestJS has a very solid GQL support, but we need to install some extra dependencies to take advantage of it:

cd nestjs-gql-redis-subscriptions
npm i @nestjs/graphql apollo-server-express graphql-tools graphql graphql-subscriptions

We've also installed graphql-subscriptions, which brings subscriptions to our app.

To see the subscriptions in action, we're going to build a "ping-pong" app, in which ping gets sent via GQL mutation, and pong gets delivered using GQL subscription.

Under the src directory, create a types.graphql file and put there our schema:

type Query {
  noop: Boolean
}

type Mutation {
  ping: Ping
}

type Subscription {
  pong: Pong
}

type Ping {
  id: ID
}

type Pong {
  pingId: ID
}

Then go to app.module.ts, and import GraphQLModule as follows:

// ... other imports
import { GraphQLModule } from '@nestjs/graphql';
import { PubSub } from 'graphql-subscriptions';

@Module({
  imports: [
    GraphQLModule.forRoot({
      playground: true,
      typePaths: ['./**/*.graphql'],
      installSubscriptionHandlers: true,
    }),
  ],
  providers: [
    {
      provide: 'PUB_SUB',
      useValue: new PubSub(),
    },
  ],
})
export class AppModule {}

Let's go through the options we pass to GraphQLModule.forRoot:

playground - exposes GQL Playground on http:localhost:${PORT}/graphql. We'll be using this tool for subscribing to "pong" events and sending "ping" mutations.
installSubscriptionHandlers - enables subscriptions support
typePaths - path to our GQL type definitions.

Another interesting detail is:

{
  provide: 'PUB_SUB',
  useValue: new PubSub(),
}

This is a default (in-memory) implementation of a publish/subscribe engine, which allows us to publish events and create subscriptions.

Now, after we've configured GQL server, it's time to create resolvers. Under the src folder, create a file ping-pong.resolvers.ts, and enter there following:

import { Resolver, Mutation, Subscription } from '@nestjs/graphql';
import { Inject } from '@nestjs/common';
import { PubSubEngine } from 'graphql-subscriptions';

const PONG_EVENT_NAME = 'pong';

@Resolver('Ping')
export class PingPongResolvers {
  constructor(@Inject('PUB_SUB') private pubSub: PubSubEngine) {}

  @Mutation('ping')
  async ping() {
    const pingId = Date.now();
    this.pubSub.publish(PONG_EVENT_NAME, { [PONG_EVENT_NAME]: { pingId } });
    return { id: pingId };
  }

  @Subscription(PONG_EVENT_NAME)
  pong() {
    return this.pubSub.asyncIterator(PONG_EVENT_NAME);
  }
}

First, we need to decorate PingPongResolvers class with @Resolver('Ping'). The official NestJS docs does a good job of describing its purpose:

You can consult the Nest.js official documentation on Working with GraphQL

The @Resolver() decorator does not affect queries or mutations (neither @Query() nor @Mutation() decorators). It only informs Nest that each @ResolveProperty() inside this particular class has a parent, which is a Ping type in this case.

Then, we define our ping mutation. Its main responsibility is to publish pong event.

Finally, we have our subscription definition, which is responsible for sending the appropriate published events to subscribed clients.

Now we need to add PingPongResolvers to our AppModule:

// ...
@Module({
  // ...
  providers: [
    PingPongResolvers,
    {
      provide: 'PUB_SUB',
      useValue: new PubSub(),
    },
  ],
})
export class AppModule {}

At this point, we're ready to start the app, and have a look at our implementation in action.

Actually, to understand the issue with in-memory subscriptions, let's run two instances of our app: one on port :3000 and another - on :3001

In one terminal window, run:

# port 3000 is the default port for our app
npm start

After that, in another one:

PORT=3001 npm start

And here is a demo:

As you can see the instance that is running on :3001 didn't get any events that were published on the :3000 instance.

Just have a look at the image below to get a view from a different angle:

Clearly, there is no way for :3001 to see events published on :3000

Now, let's adjust our app a bit to address this issue. First, we need to install Redis subscriptions dependencies

npm i graphql-redis-subscriptions ioredis

graphql-redis-subscriptions provides a Redis-aware implementation of PubSubEngine interface: RedisPubSub. You've already used that interface previously, via it's in-memory implementation - PubSub.

ioredis - is a Redis client, which is used by graphql-redis-subscriptions.

To start using our RedisPubSub, we just need to tweak AppModule a bit.

Change this:

// ...
{
  provide: 'PUB_SUB',
  useValue: new PubSub(),
}
// ...

To this:

// ...
import { RedisPubSub } from 'graphql-redis-subscriptions';
import * as Redis from 'ioredis';
//  ...

// ...
{
  provide: 'PUB_SUB',
  useFactory: () => {
    const options = {
      host: 'localhost',
      port: 6379
    };

    return new RedisPubSub({
      publisher: new Redis(options),
      subscriber: new Redis(options),
    });
  },
},
// ...

We're going to start redis in a docker container, and make it available on the localhost:6379 (which corresponds to options we pass to our RedisPubSub instance above):

docker run -it --rm --name gql-redis -p 6379:6379 redis:5-alpine

Now we need to stop our apps, and restart them again (in different terminal sessions):

npm start

and

PORT=3001 npm start

At this point, subscriptions work as expected, and events, published on one instance of the app, are received by the client, subscribed to another instance:

Here is what's happening under the hood:

Summary:

In this article, we've learned how to use Redis and GQL subscriptions to publish events across multiple instances of the server app.

We should also better understand GQL subscriptions event pub/sub flow.

Source code:

https://github.com/rychkog/gql-redis-subscriptions-article

Enjoy this article? Head on over to This Dot Labs and check us out! We are a tech consultancy that does all things javascript and front end. We specialize in open source software like, Angular, React and Vue.

Forem: Georgii Rychko

GraphQL subscriptions with Nest: how to publish across multiple running servers