Forem: Daniel Rearden

Building a GraphQL server with GraphQL Helix 🧬

Daniel Rearden — Thu, 05 Nov 2020 19:58:14 +0000

Earlier this week I released GraphQL Helix, a new JavaScript library that lets you take charge of your GraphQL server implementation.

There's a couple of factors that pushed me to roll my own GraphQL server library:

I wanted to use bleeding-edge GraphQL features like @defer, @stream and @live directives.
I wanted to make sure I wasn't tied down to a specific framework or runtime environment.
I wanted control over how server features like persisted queries were implemented.
I wanted to use something other than WebSocket (i.e. SSE) for subscriptions.

Unfortunately, popular solutions like Apollo Server, express-graphql and Mercurius fell short in one or more of these regards, so here we are.

Existing libraries like Apollo Server provide you with either a complete HTTP server or else a middleware function that you can plug into your framework of choice. GraphQL Helix takes a different approach -- it just provides a handful of functions that you can use to turn an HTTP request into a GraphQL execution result. In other words, GraphQL Helix leaves it up to you to decide how to send back the response.

Let's see how this works in practice.

A Basic Example

We'll start by building an express application and adding a /graphql endpoint.

import express from "express";
import { schema } from "./my-awesome-schema";

const app = express();

app.use(express.json());

app.use("/graphql", async (res, req) => {
  // TODO
});

app.listen(8000);

Note that we're assuming here we already have a GraphQL schema we've created. However you build your schema (GraphQL Tools, TypeGraphQL,
graphql-compose, GraphQL Nexus, etc.) is irrelevant -- as long as you have a GraphQLSchema object, you're good to go.

Next, let's extract the relevant bits from our request into a standard GraphQL Helix object:

app.use("/graphql", async (res, req) => {
  const request = {
    body: req.body,
    headers: req.headers,
    method: req.method,
    query: req.query,
  };
});

More astute readers might note that we could have just used the req object as-is — and that's true! However, this step will look a little different depending on the framework or runtime we use, so I'm being more explicit about how we define this object.

Now let's extract the relevant parameters from the request and process them.

import {
  getGraphQLParameters,
  processRequest
} from "graphql-helix";

...

app.use("/graphql", async (res, req) => {
  const request = {
    body: req.body,
    headers: req.headers,
    method: req.method,
    query: req.query,
  };

  const {
    query,
    variables,
    operationName
  } = getGraphQLParameters(request);

  const result = await processRequest({
    schema,
    query,
    variables,
    operationName,
    request,
  })
});

processRequest still takes our Request object as a parameter, so why doesn't it just call getGraphQLParameters for us? As we'll see later on, this is an intentional design choice that gives us the flexibility to decide how the parameters are actually derived from the request.

So, we've processed our request and now have a result. Groovy. Let's do something with that result.

app.use("/graphql", async (res, req) => {
  const request = {
    body: req.body,
    headers: req.headers,
    method: req.method,
    query: req.query,
  };

  const {
    query,
    variables,
    operationName
  } = getGraphQLParameters(request);

  const result = await processRequest({
    schema,
    query,
    variables,
    operationName,
    request,
  })

  if (result.type === "RESPONSE") {
    result.headers.forEach(({ name, value }) => {
      res.setHeader(name, value)
    });
    res.status(result.status);
    res.json(result.payload);
  } else {
    // TODO
  }
});

Our result includes the headers we should send back, an HTTP status code and the response payload (i.e. an object containing the data and errors we get by actually validating and executing the request).

And that's it! We now have a working /graphql endpoint that can process our requests. Neat.

So why are we writing all this extra boilerplate when I could do the same thing in a few lines of code in Apollo Server? In a word: flexibility. If we swap out Express for another framework like Fastify, we only have to change how we construct our request object and how we handle the result. In fact, we could use the meat of our implementation in virtually any other runtime -- serverless, Deno or even in the browser.

Moreover, we can process the result however our business needs dictate. We have a GraphQL over HTTP specification, but if for some reason you need to deviate from it, you can. It's your application -- send back the status, headers or response that are right for your use case.

So... what's up with that else block? As it turns out, processRequest will return one of three types of results:

RESPONSE for standard queries and mutations,
MULTIPART_RESPONSE for requests that include the new @defer and @stream directives, and
PUSH for subscriptions

Again, it's up to us to implement how to send back these responses, so let's do that now!

Subscriptions

We'll implement our subscriptions using Server Sent Events (SSE). There's a lot of advantages of using SSE over something like WebSockets for subscriptions, like being able to use the same middleware for all your requests, but a deeper comparison of the two approaches will be the topic of a future article.

There's a few libraries out there that can make integrating SSE with Express easier, but we'll do it from scratch for this example:

if (result.type === "RESPONSE") {
  ...
} else if (result.type === "PUSH") {
  res.writeHead(200, {
    "Content-Type": "text/event-stream",
    Connection: "keep-alive",
    "Cache-Control": "no-cache",
  });

  req.on("close", () => {
    result.unsubscribe();
  });

  await result.subscribe((result) => {
    res.write(`data: ${JSON.stringify(result)}\n\n`);
  });
}

Here, our result includes two methods -- subscribe and unsubscribe. We call subscribe with a callback that's passed the result each time a new subscription event is pushed -- within this callback, we just write to the response with a SSE-compatible payload. And we call unsubscribe when the request is closed (i.e. when the client closes the connection) to prevent memory leaks.

Easy, peasy. Now let's take a look at MULTIPART_RESPONSE.

Multipart Responses

If our request includes @stream or @defer directives, our request needs to be sent down to the client in chunks. For example, with @defer, we send down everything except the deferred fragment and eventually send down the deferred fragment data when its finally resolved. As such, our MULTIPART_RESPONSE result looks a lot like the PUSH result with one key difference -- we do want to eventually end our response once all parts have been sent.

if (result.type === "RESPONSE") {
  ...
} else if (result.type === "PUSH") {
  ...
} else {
  res.writeHead(200, {
    Connection: "keep-alive",
    "Content-Type": 'multipart/mixed; boundary="-"',
    "Transfer-Encoding": "chunked",
  });

  req.on("close", () => {
    result.unsubscribe();
  });

  await result.subscribe((result) => {
    const chunk = Buffer.from(
      JSON.stringify(result),
      "utf8"
    );
    const data = [
      "",
      "---",
      "Content-Type: application/json; charset=utf-8",
      "Content-Length: " + String(chunk.length),
      "",
      chunk,
      "",
    ].join("\r\n");
    res.write(data);
  });

  res.end("\r\n-----\r\n");  
}

Note that the Promise returned by subscribe won't resolve until the request has been fully resolved and the callback has been called with all the chunks, at which point we can safely end our response.

Congrats! Our API now has support for @defer and @stream (provide you're using the correct version of graphql-js).

Adding GraphiQL

GraphQL Helix comes with two additional functions that can be used to expose a GraphiQL interface on your server.

shouldRenderGraphiQL takes a Request object and returns a boolean that indicates, as you may have already guessed, whether you should render the interface. This is helpful when you have a single endpoint for both your API and the interface and only want to return the GraphiQL interface when processing a GET request from inside a browser.

renderGraphiQL just returns a string with the HTML necessary to the render the interface. If you want to create a separate endpoint for your documentation, you can use this function without using shouldRenderGraphiQL at all.

app.use("/graphql", async (req, res) => {
  const request = {
    body: req.body,
    headers: req.headers,
    method: req.method,
    query: req.query,
  };

  if (shouldRenderGraphiQL(request)) {
    res.send(renderGraphiQL());
  } else {
    // Process the request
  }
});

The returned GraphiQL has a fetcher implementation that will work with multipart requests and SSE as shown in the examples above. If you need to do something else for your server, you can roll your own using renderGraphiQL as a template only.

Evolving your server implementation

GraphQL Helix is, by design, light-weight and unopinionated. Libraries like Apollo Server are bloated with a lot of features that you may never need.

However, that doesn't mean you can't add those features back if you need them. For example, we can add uploads to our server by adding the Upload scalar and using the appropriate middleware from graphql-upload

import { graphqlUploadExpress } from "graphql-upload";

app.use(
  "/graphql",
  graphqlUploadExpress({
    maxFileSize: 10000000,
    maxFiles: 10,
  }),
  (req, res) => {
    // Our implementation from before
  }
)

Similarly, we can add support for live queries with the @live directive by adding @n1ru4l/graphql-live-query and @n1ru4l/in-memory-live-query-store. We just need to add the directive to our schema and provide the appropriate execute implementation:

import {
  InMemoryLiveQueryStore
} from "@n1ru4l/in-memory-live-query-store";

const liveQueryStore = new InMemoryLiveQueryStore();

...

const result = const result = await processRequest({
  schema,
  query,
  variables,
  operationName,
  request,
  execute: liveQueryStore.execute,
});

Tracing, logging, persisted queries, request batching, response deduplication and any number of other features can be added just as easily without the bloat and without having to wrestle with some plugin API or unfriendly abstraction.

You can check the repository for more examples and recipes (I'll be adding more as time allows and also accepting PRs!).

Conclusion

So when should you use Apollo Server instead of GraphQL Helix? If you need to throw together a quick POC or tutorial, Apollo Server is great. If you want to use federation, you might want to stick with Apollo (and even then there are better alternatives to doing GraphQL with microservices).

GraphQL Helix offers a flexible, extensible approach to building a GraphQL server, without the bloat. If you're building something other than another to-do tutorial, I highly recommend checking it out :)

Making GraphQL Magic with Sqlmancer 🧙🔮✨

Daniel Rearden — Fri, 15 May 2020 03:09:32 +0000

Today I'm excited to announce the beta release of Sqlmancer! Sqlmancer is a Node.js library that empowers you to effortlessly and efficiently translate GraphQL queries into SQL statements. This article will walk through setting up a project with Sqlmancer and showcase a few of its many features:

Multiple dialect support. Sqlmancer supports Postgres, MySQL, MariaDB and SQLite, enabling you to incorporate it into existing projects regardless of what flavor of SQL you're using.
Performance. Avoid the N+1 problem by building a single SQL query to fetch all necessary data, regardless of query depth.
Robust filtering and sorting. Add complex filtering and sorting to your queries, including filtering using logical operators and filtering and sorting by fields and aggregate fields of related models.
Arbitrarily deep nesting. Define one-to-one, one-to-many and many-to-many relationships between models. Related models can be filtered, sorted and paginated just like root-level fields.
Mutations made easy. Create, update and delete records, with or without transactions, using a simple, fluent API.
Views and CTEs. Take advantage of existing views in your database or create inline ones using common table expressions.
Custom scalars. Use the scalars that make sense for your schema.
Abstract types. Utilize unions and interfaces in your schema using views or single table inheritance.

If you want to skip the tutorial and see a working example, you can check out this CodeSandbox container which includes all the code shown in this post. Peruse the official docs for additional examples and a detailed API reference. If you like the project, please ⭐ it on GitHub!

The database

For this example, we'll use SQLite and a sample database (that you can get here courtesy of SQLite Tutorial). Here's a diagram of the database we'll be using:

The models

Let's start by writing our type definitions. Let's start by adding a model for our customers table:

type Customer @model(
  table: "customers"
  pk: "CustomerId"
) {
  id: ID! @col(name: "CustomerId")
  firstName: String!
  lastName: String!
  email: String!
}

Sqlmancer uses schema directives like @model and @col shown here to decorate your existing types and show how they relate to your database. All your models are defined right inside your type definitions. We've created a model named Customer and associated it with the customers table in our database. We've also indicated that the primary key on this table is the CustomerId column.

Each field on your type (with some exceptions noted below) will be mapped to a column in your database. We can use the @col directive to map a field to a column with a different name than our field, as we've done with our id field.

Naming conventions between GraphQL fields and database columns are usually different -- in GraphQL it's camelCase while SQL databases frequently use either snake_case or PascalCase. Our sample database happens to use PascalCase. It'd be incredibly laborious to add the @col directive to every field, so let's tell Sqlmancer to instead apply a transformation to each field name when determining the column name:

type Query @sqlmancer(
  dialect: SQLITE
  transformFieldNames: PASCAL_CASE
) {
  customers: [Customer!]!
}

The @sqlmancer directive is a required directive that is applied to your Query type. The transformFieldNames tells Sqlmancer to convert a model's field name to PascalCase to determine its matching column name. Any fields with a @col directive will still override this derived value. While transformFieldNames is an optional argument, the dialect is required and indicates which database you're using.

NOTE: There's a number of other directives available to you when defining models. For example, @depends can be used to create "virtual" fields, while @private can be used to hide fields from your API while still adding them to your client. Check out the docs for more info!

Relationships

Next, let's add another model and relate it to our Customer model:

type Customer @model(
  table: "customers"
  pk: "CustomerId"
) {
  id: ID! @col(name: "CustomerId")
  firstName: String!
  lastName: String!
  email: String!
  invoices: [Invoice!]!
    @relate(on: { from: "CustomerId", to: "CustomerId" })
}

type Invoice @model(
  table: "invoices"
  pk: "InvoiceId"
) {
  id: ID! @col(name: "InvoiceId")
  total: Float!
  createdOn: String! @col(name: "InvoiceId")
  customer: Customer
    @relate(on: { from: "CustomerId", to: "CustomerId" })
}

We use the @relate directive to indicate a relationship between two models. The on argument indicates which columns to join the two model's tables on -- from is this model's column and to is the related model's column.

Filtering, sorting and pagination

Lastly, let's add the ability to sort, filter and paginate the customers and invoices we look up through our API.

type Query @sqlmancer(
  dialect: POSTGRES
  transformFieldNames: PASCAL_CASE
) {
  customers: [Customer!]! @where @orderBy @limit @offset
  invoices: [Invoice!]! @many
}

Sqlmancer includes a number of "utility" directives to help generate types from your models. Here, @where, @orderBy, @limit and @offset add arguments to a field with the corresponding name and appropriate input type.

Each directive can be applied separately, or you can use @many to apply all four. Also note that we can add these directives to our relation fields too!

  invoices: [Invoice!]!
    @relate(on: { from: "CustomerId", to: "CustomerId" })
    @many

Directives like these are a great way to quickly implement a schema with robust capabilities. However, they are completely optional. If you want more control, you can define the types yourself and as long as they are the appropriate shape, they'll work all the same.

The database client

Now that we've got our type definitions, we can use them to create a database client. Sqlmancer uses Knex.js under the hood. In order to initialize the Sqlmancer client, we'll need a Knex instance pointed at our database:

import Knex from "knex";

const knex = Knex({
  client: "sqlite3",
  connection: {
    filename: "./sample.db"
  }
});

Now, let's initialize the client, pointing it to where our type definitions our located. In this example, we're using the same file, but you can pass in a glob pattern to indicate any number of files elsewhere.

import { createSqlmancerClient } from "sqlmancer";

const client = createSqlmancerClient(__filename, knex);

Our client includes all the methods of our knex instance, as well as a models property that includes the generated models. Let's use destructuring assignment syntax and get our two models:

const { models: { Customer, Invoice } } = createSqlmancerClient(__filename, knex);

The models can be used inside your resolvers or in anywhere else in your application (REST APIs, webhooks, scripts, etc.). Adding the resolver can be as simple as this:

const resolvers = {
  Query: {
    customers: (root, args, ctx, info) => {
      return Customer.findMany()
        .resolveInfo(info)
        .execute();
    },
    invoices: (root, args, ctx, info) => {
      return Invoice.findMany()
        .resolveInfo(info)
        .execute();
    }
  }
};

findMany returns a query builder instance with a number of methods like select, where, orderBy, etc. used to build your SQL query. However, it also includes a resolveInfo method that can be used inside your resolvers to automatically parse the field's arguments and selection set and generate the appropriate query builder options! Once we're done building our query, we call the execute method to actually execute it and get our results.

The schema

The last step is creating our schema. Because Sqlmancer relies on schema directives, these need to be added to our schema as well. We could do this ourselves, but we can also use the convenient makeSqlmancerSchema function, which is just a wrapper around makeExecutableSchema from graphql-tools.

import { createSqlmancerClient, makeSqlmancerSchema } from "sqlmancer";

const schema = makeSqlmancerSchema({ typeDefs, resolvers });

Now we're good to go. We can use our schema with apollo-server, express-graphql or another HTTP library of our choice.

const apollo = new ApolloServer({ schema })

The magic

Let's spin up our server and navigate to the GraphQL Playground interface. Even though we hardly wrote any code, our schema supports all sorts of complex queries, like this one:

query {
  customers(
    where: { firstName: { like: "L%" } }
    orderBy: [{ invoices: { sum: { total: DESC } } }]
    limit: 5
  ) {
    id
    firstName
    lastName
    invoices(
      limit: 3
      orderBy: { total: DESC }
    ) {
      id
      total
    }
  }
}

Even though this GraphQL query includes filtering, sorting, pagination and a relationship field that is itself paginated and sorted, Sqlmancer compiles it to a single SQL query. And we added these features with a handful of directives and a single line of code in our resolver. You can see the magic for yourself using this CodeSandbox container, which includes all the code shown in this article.

BONUS: TypeScript integration

Sqlmancer was created with TypeScript in mind. You can use Sqlmancer's CLI to generate TypeScript types for your client. Just tell the CLI where your type definitions are and where to create the file with the typings:

sqlmancer generate ./some/glob/**/*.graphql ./generated.ts

then import and use the generated definition

import { createSqlmancerClient } from "sqlmancer";
import { SqlmancerClient } from "./generated";

const client = createSqlmancerClient<SqlmancerClient>(__filename, knex);

Now you will not only have type-safety and autocompletion for the different model methods, but you'll also be able to work with fully typed result objects -- down to the individual columns you've selected and associations you've loaded!

We've only scratched the surface of what you can accomplish with Sqlmancer, but I hope I've peaked your interest enough to take the library for a spin or to at least poke your nose in the docs! I'll be publishing subsequent tutorials on more advanced topics like mutations, inline views, custom scalars, aggregation, and more. Stay tuned!