Forem: Oleksandr Hrishchuk

Visual regression tests for Storybook with Loki

Oleksandr Hrishchuk — Wed, 22 Feb 2023 10:58:49 +0000

How to setup visual regression tests with Loki for your design system

To my knowledge, there are currently no leading standalone tools for visual regression testing. Many companies implement their own solutions, but there are no tools that are widely used and recommended as a standard.

Some companies actively explore OSS solutions and in this article, I will share my experience with one of them.

What is Loki?

Loki is a tool that makes it easy to test Storybook projects for visual regressions. Loki was used for the trial stage at Thoughtworks tech radar.

Why Loki?

There are a few visual regression tools for the web, but most either cannot be run headless or use phantomjs which is deprecated and a browser nobody is actually using. They usually also require you to maintain fixtures. With react-native it's now possible to target multiple platforms with a single codebase, but there's no single tool to test all to my knowledge.

How to setup Loki for your project?

1. Install Loki package



npm i loki --save-dev

or if you use yarn



yarn add loki -D

2. Create a Loki config file



yarn loki init

This will add standard configuration to your project. You can find it in the package.json file.

Example of the configuration:



{
  "name": "my-project",
  "version": "1.0.0",
  "loki": {
    "chromeSelector": "#my-decorator > *",
    "configurations": {
      "chrome.laptop": {
        "target": "chrome.app",
        "width": 1366,
        "height": 768
      },
      "chrome.iphone7": {
        "target": "chrome.docker",
        "preset": "iPhone 7"
      },
      "chrome.a4": {
        "target": "chrome.aws-lambda",
        "preset": "A4 Paper"
      },
      "ios": {
        "target": "ios.simulator"
      },
      "android": {
        "target": "android.emulator"
      }
    }
  }
}

I prefer not to pollute my package.json and extract configuration to a separate file loki.config.js:



// loki.config.js
module.exports = {
  diffingEngine: 'pixelmatch',
  configurations: {
    'chrome.laptop': {
      target: 'chrome.docker',
      width: 1366,
      height: 768,
      deviceScaleFactor: 1,
      mobile: false,
    },
    // 'chrome.iphone7': {
    //   target: 'chrome.docker',
    //   preset: 'iPhone 7'
    // }
  },
}

I'll go with the simplest configuration for now, but you can find more information about the configuration options in the Loki documentation.

I choose the pixelmatch engine for now, but you can also choose between GraphicsMagick (gm) and looks-same. Currently, three diffing engines are available. Worth to mention that if you'd like to use graphicsmagick diffing engine,
for image comparison, you need to install it on the machine / CI as an additional dependency.

3. Install Docker

Loki uses Docker to run tests in an image. If you don't have Docker installed, you can follow the installation instructions.

4. Start Storybook

When running tests locally, you need to run a local instance of the Storybook & start Docker to execute the tests.



yarn storybook

5. Generate reference images



yarn loki update

will create a reference image for each story in your Storybook & place them in the .loki/reference directory.

6. Run tests



yarn loki test

or if you want to run tests for a specific platform (e.g. laptop, iphone7 etc.)



yarn loki test laptop

On each test run, Loki will create new files in the .loki directory. Inside this directory, you will find the current, difference, and reference folders.

The current directory contains the images that were generated during the test run.
The difference directory contains the images that were generated during the test run and are the ones that were different from the reference images with, visual differences shown.
The reference directory contains the reference images that were generated during the loki update command. These are the images that are used to compare against the current images.

Images from the reference directory should be committed to the repository if you want to use them as a reference for your tests on CI.

7. Add Loki to CI (as GitHub action)

To run Loki on CI you need to add a new job to your CI pipeline.

Update NPM scripts



{
  "scripts": {
    "visual-tests": "loki test laptop",
    "visual-tests-ci": "build-storybook && loki --requireReference --reactUri file:./storybook-static"
  }
}

Add a new job to your CI pipeline as a GitHub action



# .github/workflows/ci.yml

visual-regression-tests:
  name: 'Visual tests: React DOM'
  runs-on: ubuntu-latest

  steps:
    - uses: actions/checkout@v3
    - name: Setup node from node version file
      uses: actions/setup-node@v3
      with:
        node-version: '16'
        cache: 'yarn'
        registry-url: 'https://npm.pkg.github.com/'
        scope: '@your-org'
    - name: Install dependencies
      run: yarn --frozen-lockfile --non-interactive --silent --ignore-scripts
      env:
        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
    - name: Run loki
      run: yarn visual-tests-ci
    - name: Archive screenshots
      if: ${{ failure() }}
      uses: actions/upload-artifact@v1
      with:
        name: design-system
        path: .loki

Skipping stories

If you want to skip some stories from the tests you can add loki.skip parameter to the story:



<!-- Badge.stories.mdx -->

import { Canvas, Story } from '@storybook/addon-docs'
import Badge from './Badge'

<Canvas>
  <Story
    name="warning"
    parameters={{
      loki: {
        skip: true
      }
    }}
  >
    <Badge variant="warning">
      On hold
    </Badge>
  </Story>
</Canvas>

Approve changes

If you want to approve changes you can run:



yarn loki approve

This will update images in the reference folder if changes are intended and valid.

Accessing artifact files from GitHub action

In case of a flaky test or a test that fails on CI but passes locally, you can download the artifact files and run the tests locally to see what's going on.

To access artifact files you can go to the "Actions" tab in your repository, find your workflow build, navigate to "Summary" and download the artifact from the latest run:

Pros

Relatively easy to setup
Multi devices/platforms support
Running in Docker
Different comparison engines to choose from

Cons

Building a Storybook on each test run, Loki requires it to perform tests on a static build output. The build takes time & resources on CI. A possible solution: cache this somehow
No handy way to visually compare/accept differences/changes between the test runs

Comparison stats - VRT tests

Tests were run on a local machine and on a CI machine (build time excluded).

The tests were run on a Storybook with 30 components and on average 3 stories per component.

The tests were run with 3 different comparison engines.

There are three currently available options to choose from when comparing images in Loki.

Tests runs	pixelmatch	gm	looks-same
Locally (M1)	~27.71s ✅	~27.7s ✅	~29.09s ✅

Conclusion

It's important to have a visual regression testing tool in your tool belt. It's a great way to catch visual bugs early and prevent them from reaching production.

There are many things to consider when implementing visual regression testing, it should not slow down your development process and release process.

Although there are some cons to using Loki, I'd give it a try and see if it works for you. It's a decent tool for catching visual bugs and it's relatively easy to set up.

If you have any questions or suggestions, feel free to reach out to me on Twitter @alxgri

Upload files to AWS S3 bucket using official Amazon Docker image and AWS CLI with GitLab CI

Oleksandr Hrishchuk — Mon, 19 Oct 2020 10:38:26 +0000

One thing I really enjoy doing when I get a chance is to work on "DevOps-related" tasks at work or pet-projects. For example, building CI pipelines configurations from scratch(say 'hi' to YAML) using different Docker images, running apps in the Docker or Kubernetes.

I know a lot of developers hate these types of tasks but not me. For me, it's really refreshing and I'm always completing them with extra interest.

Getting something running in the cloud brings joy to me. I knew some developers who don't even understand where End-to-End(E2E) Cypress tests are running, or what is ESLint job and workers on CI or Flow checks. CI server is just a "magical" black box for them from where they get "green" light (checks) to merge pull requests. That's fine, I guess.

It could be even something simple, like uploading configuration files via the official AWS CLI using amazon/aws-cli s3 sync command to GitLab ...or any CI. It doesn't really matter what you're using, CircleCI or some other. I have experience of different sorts: from custom CI/Jenkins to fine-tuned CircleCI plans, GitHub Actions, GitLab CI, and so on.

How to upload files to AWS S3 using amazon/aws-cli s3 sync command with GitLab CI

Today, we're going to take a look at how can we upload files(any files) using official AWS CLI from CI to an S3 bucket.

One of the real-world use cases that I'm working on right now that you might face in practical web development is consuming these files from your application at a runtime. Benefits from this method could be avoiding release cycle, in case of NPM JavaScript package - publishing these updates to NPM but rather consuming them directly after update, without need to reload. This technic is pretty popular when developing mobile apps and getting updates there without the need to re-publish application to the App Store (kinda React Native way of doing things).

Create .gitlab-ci.yml

Everything that runs on the GitLab CI start configuration in the .gitlab-ci.yml file. Let's create it and configure it to run different jobs.

# .gitlab-ci.yml

stages:
  - verify
  - deploy

variables:
  BASE_REGISTRY_URL: '0123456.dkr.ecr.aws-region.amazonaws.com'
  DEPLOY_IMAGE_URL: 'amazon/aws-cli'

.auth: &auth
  before_script:
    - echo /custom-host/repository/npm-packages/:_authToken=${NPM_TOKEN} >> .npmrc

verify:
  <<: *auth
  stage: verify
  image: node:12
  except:
    - /^v\d+\.[1-9]\d*\.\d+/
  script:
    - yarn install
    - yarn lint
    - yarn prettier-ci

We are adding two stages:

verify just to test pipeline, install dependencies, run linter and Prettier
deploy is the most interesting part

.auth is used here to set NPM_TOKEN to our .npmrc because we require to install some private npm package for our project to build. This is totally an option and just to demonstrate the purpose.

Note a few things here, we are defining variables like BASE_REGISTRY_URL - that's our amazon registry and DEPLOY_IMAGE_URL is our Docker image that we want to use.

amazon/aws-cli is an identifier of the official amazon image from Docker Hub with the most updated version of AWS CLI installed. Find out more about available command of the CLI here amazon/aws-cli.

How to use or migrate to the official CLI you can read more in this article Using the official AWS CLI version 2 Docker image.

Working on the deployment stage

deploy is the most interesting part. Let's say that we want to support multiple environments, like dev_deploy and stage_deploy, so we will add them:

# .gitlab-ci.yml

stages:
  - verify
  - deploy

variables:
  BASE_REGISTRY_URL: '0123456.dkr.ecr.aws-region.amazonaws.com'
  DEPLOY_IMAGE_URL: 'amazon/aws-cli'

.auth: &auth
  before_script:
    - echo /custom-host/repository/frontend/:_authToken=${NPM_TOKEN} >> .npmrc

verify:
  <<: *auth
  stage: verify
  image: node:12
  except:
    - /^v\d+\.[1-9]\d*\.\d+/
  script:
    - yarn install
    - yarn lint
    - yarn prettier-ci

dev_deploy:
  tags:
    - ci
  stage: deploy
  only:
    - master
  environment:
    name: dev
  script:
    - aws ecr get-login --region=eu-central-1 --no-include-email | bash
    - docker pull ${DEPLOY_IMAGE_URL}
    # prettier-ignore
    - >-
      docker run --rm
      -v $(pwd):/app
      -e "AWS_ACCESS_KEY_ID=${DEV_ACCESS_KEY_ID}"
      -e "AWS_SECRET_ACCESS_KEY=${DEV_SECRET_ACCESS_KEY}"
      -e "AWS_DEFAULT_REGION=eu-central-1"
      ${DEPLOY_IMAGE_URL} s3 sync --delete /app/dev s3://your-dev-bucket-name

stage_deploy:
  tags:
    - ci
  stage: deploy
  environment:
    name: stage
  only:
    - master
  script:
    - aws ecr get-login --region=eu-central-1 --no-include-email | bash
    - docker pull ${DEPLOY_IMAGE_URL}
    # prettier-ignore
    - >-
      docker run --rm
      -v $(pwd):/app
      -e "AWS_ACCESS_KEY_ID=${STAGE_ACCESS_KEY_ID}"
      -e "AWS_SECRET_ACCESS_KEY=${STAGE_SECRET_ACCESS_KEY}"
      -e "AWS_DEFAULT_REGION=eu-central-1"
      ${DEPLOY_IMAGE_URL} s3 sync --delete /app/stage s3://your-stage-bucket-name

This job will be executed on every merge to master/main branch. It will not run inside the merge/pull request.

only:
  - master

-e for passing environments variables.
${DEV_ACCESS_KEY_ID} and ${DEV_SECRET_ACCESS_KEY} are variables with your AWS access key and secret. They are coming from the GitLab CI repository level => Setting => CI / CD => Variables. You can set new there or revel already sat at any time.

Ok, the key component here is the amazon/aws-cli s3 sync --delete is a command which invokes aws-cli bin with one of the services which is called s3, s3 has sync command with --delete option which will copy files from current /app/stage folder to the AWS S3 bucket (target destination).

By default, the aws sync command (see documentation) does not delete files. It simply copies new or modified files to the destination.
Using the --delete option deletes files that exist in the destination but not in the source.

s3://your-stage-bucket-name is a path to your S3 bucket/storage. Note how it is conveniently starting with s3://. As you might notice for stage and dev I'm using different buckets of course.

Conclusion

We set up GitLab CI pipeline with files are being copied to the S3 bucket from CI using official Docker image and Amazon CLI v2. That's it! Hope you found it helpful. Thanks :)

How to setup Apollo Client in order to perform GraphQL queries with NextJS

Oleksandr Hrishchuk — Thu, 11 Jun 2020 15:39:47 +0000

In the previous article Setting up GraphQL API with MongoDB and Apollo Server for a NextJS app we've created a GraphQL API with Apollo Server setup and connection to the MongoDB. We worked mainly on the server side of the application.

It's recommended to read and follow along the first part if you haven't.

In this article, we're moving to the client side of the application and we'll learn how to consume an API that we've created in the first part. Meaning, performing queries from our NextJS application to the GraphQL API.

An idea

In order to perform queries to our GraphQL API from the client-side (and not only from the GraphQL Playground) we'll need to setup Apollo Client.

Using a custom implementation of the withApollo() high-order component we are going to wrap index file with it and make queries to the API.

Installing Apollo Client and related dependencies

yarn add apollo-client @apollo/react-hooks @apollo/react-ssr apollo-cache-inmemory apollo-link-http apollo-link-schema graphql-tag

Setting up an Apollo Client

At the project root let's create a folder called apollo/ with a client.js file inside.

// apollo/client.js
import React from 'react'
import Head from 'next/head'
import { ApolloProvider } from '@apollo/react-hooks'
import { ApolloClient } from 'apollo-client'
import { InMemoryCache } from 'apollo-cache-inmemory'

let globalApolloClient = null

/**
 * Creates and provides the apolloContext
 * to a next.js PageTree. Use it by wrapping
 * your PageComponent via HOC pattern.
 * @param {Function|Class} PageComponent
 * @param {Object} [config]
 * @param {Boolean} [config.ssr=true]
 */
export function withApollo(PageComponent, { ssr = true } = {}) {
  const WithApollo = ({ apolloClient, apolloState, ...pageProps }) => {
    const client = apolloClient || initApolloClient(apolloState)
    return (
      <ApolloProvider client={client}>
        <PageComponent {...pageProps} />
      </ApolloProvider>
    )
  }

  // Set the correct displayName in development
  if (process.env.NODE_ENV !== 'production') {
    const displayName =
      PageComponent.displayName || PageComponent.name || 'Component'

    if (displayName === 'App') {
      console.warn('This withApollo HOC only works with PageComponents.')
    }

    WithApollo.displayName = `withApollo(${displayName})`
  }

  if (ssr || PageComponent.getInitialProps) {
    WithApollo.getInitialProps = async (ctx) => {
      const { AppTree } = ctx

      // Initialize ApolloClient, add it to the ctx object so
      // we can use it in `PageComponent.getInitialProp`.
      const apolloClient = (ctx.apolloClient = initApolloClient())

      // Run wrapped getInitialProps methods
      let pageProps = {}
      if (PageComponent.getInitialProps) {
        pageProps = await PageComponent.getInitialProps(ctx)
      }

      // Only on the server:
      if (typeof window === 'undefined') {
        // When redirecting, the response is finished.
        // No point in continuing to render
        if (ctx.res && ctx.res.finished) {
          return pageProps
        }

        // Only if ssr is enabled
        if (ssr) {
          try {
            // Run all GraphQL queries
            const { getDataFromTree } = await import('@apollo/react-ssr')
            await getDataFromTree(
              <AppTree
                pageProps={{
                  ...pageProps,
                  apolloClient,
                }}
              />
            )
          } catch (error) {
            // Prevent Apollo Client GraphQL errors from crashing SSR.
            // Handle them in components via the data.error prop:
            // https://www.apollographql.com/docs/react/api/react-apollo.html#graphql-query-data-error
            console.error('Error while running `getDataFromTree`', error)
          }

          // getDataFromTree does not call componentWillUnmount
          // head side effect therefore need to be cleared manually
          Head.rewind()
        }
      }

      // Extract query data from the Apollo store
      const apolloState = apolloClient.cache.extract()

      return {
        ...pageProps,
        apolloState,
      }
    }
  }

  return WithApollo
}

/**
 * Always creates a new apollo client on the server
 * Creates or reuses apollo client in the browser.
 * @param  {Object} initialState
 */
function initApolloClient(initialState) {
  // Make sure to create a new client for every server-side request so that data
  // isn't shared between connections (which would be bad)
  if (typeof window === 'undefined') {
    return createApolloClient(initialState)
  }

  // Reuse client on the client-side
  if (!globalApolloClient) {
    globalApolloClient = createApolloClient(initialState)
  }

  return globalApolloClient
}

/**
 * Creates and configures the ApolloClient
 * @param  {Object} [initialState={}]
 */
function createApolloClient(initialState = {}) {
  const ssrMode = typeof window === 'undefined'
  const cache = new InMemoryCache().restore(initialState)

  // Check out https://github.com/vercel/next.js/pull/4611 if you want to use the AWSAppSyncClient
  return new ApolloClient({
    ssrMode,
    link: createIsomorphLink(),
    cache,
  })
}

function createIsomorphLink() {
  if (typeof window === 'undefined') {
    const { SchemaLink } = require('apollo-link-schema')
    const { schema } = require('./schema')
    return new SchemaLink({ schema })
  } else {
    const { HttpLink } = require('apollo-link-http')
    return new HttpLink({
      uri: '/api/graphql',
      credentials: 'same-origin',
    })
  }
}

Sharing schema for Apollo Client and Server

Our GraphQL schema is located now under the pages/api/graphql.js alongside with our Apollo Server setup.

Before we had all the setup in on file on the server, like so:

import { ApolloServer, gql } from 'apollo-server-micro'
import { makeExecutableSchema } from 'graphql-tools'
import { MongoClient } from 'mongodb'

require('dotenv').config()

const typeDefs = gql`
  type User {
    id: ID!
    firstName: String!
    lastName: String!
    blog: String
    stars: Int
  }

  type Query {
    users: [User]!
  }
`

const resolvers = {
  Query: {
    users(_parent, _args, _context, _info) {
      return _context.db
        .collection('users')
        .findOne()
        .then((data) => {
          return data.users
        })
    },
  },
}

const schema = makeExecutableSchema({
  typeDefs,
  resolvers,
})

let db

const apolloServer = new ApolloServer({
  schema,
  context: async () => {
    if (!db) {
      try {
        const dbClient = new MongoClient(process.env.MONGO_DB_URI, {
          useNewUrlParser: true,
          useUnifiedTopology: true,
        })

        if (!dbClient.isConnected()) await dbClient.connect()
        db = dbClient.db('next-graphql')
      } catch (e) {
        console.log('--->error while connecting with graphql context (db)', e)
      }
    }

    return { db }
  },
})

export const config = {
  api: {
    bodyParser: false,
  },
}

export default apolloServer.createHandler({ path: '/api/graphql' })

However we need to access schema on the client side as well so let's move typeDefs, resolvers and schema setup to the apollo folder on the client and just import it on the server.

Inside the apollo/ folder let's create type-defs.js, resolvers.js and schema.js files and move respective parts in there so you would have a file structure like this:

// apollo/type-defs.js
import gql from 'graphql-tag'

export const typeDefs = gql`
  type User {
    id: ID!
    firstName: String!
    lastName: String!
    blog: String
    stars: Int
  }

  type Query {
    users: [User]!
  }
`

// apollo/revolvers.js
export const resolvers = {
  Query: {
    users(_parent, _args, _context, _info) {
      return _context.db
        .collection('users')
        .findOne()
        .then((data) => {
          return data.users
        })
    },
  },
}

// apollo/schema.js
import { makeExecutableSchema } from 'graphql-tools'
import { typeDefs } from './type-defs'
import { resolvers } from './resolvers'

export const schema = makeExecutableSchema({
  typeDefs,
  resolvers,
})

After moving files to the client our Apollo Server setup looks nice and clean:

// pages/api/graphql.js
import { ApolloServer } from 'apollo-server-micro'
import { MongoClient } from 'mongodb'
import { schema } from '../../apollo/schema'

require('dotenv').config()

let db

const apolloServer = new ApolloServer({
  schema,
  context: async () => {
    if (!db) {
      try {
        const dbClient = new MongoClient(process.env.MONGO_DB_URI, {
          useNewUrlParser: true,
          useUnifiedTopology: true,
        })

        if (!dbClient.isConnected()) await dbClient.connect()
        db = dbClient.db('next-graphql')
      } catch (e) {
        console.log('--->error while connecting with graphql context (db)', e)
      }
    }

    return { db }
  },
})

export const config = {
  api: {
    bodyParser: false,
  },
}

export default apolloServer.createHandler({ path: '/api/graphql' })

After the refactoring our API still works as expected:

Create Users component and perform queries to the GraphQL API

Create components/Users.js component which will display a list of users:

// components/Users.js
import React from 'react'
import gql from 'graphql-tag'
import { useQuery } from '@apollo/react-hooks'

const UsersQuery = gql`
  query Users {
    users {
      id
      firstName
    }
  }
`

const Users = () => {
  const { loading, error, data } = useQuery(UsersQuery)

  if (loading) return 'loading users...'
  if (error) return 'error while loading users'

  return (
    <div>
      <table>
        <thead>
          <tr>
            <th>First Name</th>
          </tr>
        </thead>
        <tbody>
          {data.users.map((user) => (
            <tr key={user.id}>
              <td>{user.firstName}</td>
            </tr>
          ))}
        </tbody>
      </table>
    </div>
  )
}

export default Users

Wrap Index file `withApollo()` HOC

In order for GraphQL Queries to work we need to wrap our Index page withApollo() HOC.

Let's refactor our Index page to display the Users component and import withApollo().

// pages/index.js
import Head from 'next/head'
import Users from '../components/Users'
import { withApollo } from '../apollo/client'

const Index = () => {
  return (
    <div className="container">
      <Head>
        <title>Create Next App</title>
        <link rel="icon" href="/favicon.ico" />
      </Head>

      <main>
        <Users />
      </main>

      <footer>
        <a
          href="https://vercel.com?utm_source=create-next-app&utm_medium=default-template&utm_campaign=create-next-app"
          target="_blank"
          rel="noopener noreferrer"
        >
          Powered by{' '}
          <img src="/vercel.svg" alt="Vercel Logo" className="logo" />
        </a>
      </footer>

      <style jsx>{`
        .container {
          min-height: 100vh;
          padding: 0 0.5rem;
          display: flex;
          flex-direction: column;
          justify-content: center;
          align-items: center;
        }

        main {
          padding: 5rem 0;
          flex: 1;
          display: flex;
          flex-direction: column;
          justify-content: center;
          align-items: center;
        }

        footer {
          width: 100%;
          height: 100px;
          border-top: 1px solid #eaeaea;
          display: flex;
          justify-content: center;
          align-items: center;
        }

        footer img {
          margin-left: 0.5rem;
        }

        footer a {
          display: flex;
          justify-content: center;
          align-items: center;
        }

        a {
          color: inherit;
          text-decoration: none;
        }

        .description {
          line-height: 1.5;
          font-size: 1.5rem;
        }

        .logo {
          height: 1em;
        }
      `}</style>

      <style jsx global>{`
        html,
        body {
          padding: 0;
          margin: 0;
          font-family: -apple-system, BlinkMacSystemFont, Segoe UI, Roboto, Oxygen,
            Ubuntu, Cantarell, Fira Sans, Droid Sans, Helvetica Neue, sans-serif;
        }

        * {
          box-sizing: border-box;
        }
      `}</style>
    </div>
  )
}

export default withApollo(Index)

That's it! We can perform queries to our GraphQL API with Apollo Client and fetch data from the MongoDB.

graphql-apollo-mongodb-example

I created a supporting repository for this article so you can follow along commit-by-commit.

See deployed project preview here.

Originally posted on oleks.pro

Setting up GraphQL API with MongoDB and Apollo Server for a NextJS app

Oleksandr Hrishchuk — Mon, 08 Jun 2020 09:25:59 +0000

Recently, I got an interest in working with a Next.js React Framework for server-side-rendered (SSR)applications (not only). I'm using Next more and more for my pet projects. I was surprised by how intuitive, easy, and enjoyable working with NextJS is.

In this article, we're going to setup GraphQL API with MongoDB connection for a NextJS app. I was surprised that both NextJS and Apollo GraphQL examples lacked such a common use case with MongoDB. There were examples with PostgreSQL but I wanted to use a non-SQL database. After spending some time learning and reading I put together this step-by-step guide on how to connect your Apollo Server GraphQL API to MongoDB and read/write data to Mongo database in your GraphQL resolvers.

Initialize a NextJS default app

There are multiple ways to initialize a project with Next via create-next-app similar to create-react-app or manually.

I'm going to use create-next-app with Yarn (alternatively, you could use NPM) for this example:

yarn create next-app graphql-apollo-mongodb

Pick a template › Default starter app

After dependencies are installed:

cd graphql-apollo-mongodb
yarn dev

Cool! Our NextJS app is running on http://localhost:3000.

Setting up MongoDB Atlas

MongoDB Atlas is the global cloud database service for modern applications. Deploy fully managed MongoDB across AWS, Azure, or GCP. Best-in-class automation and proven practices guarantee availability, scalability, and compliance with the most demanding data security and privacy standards. Use MongoDB's robust ecosystem of drivers, integrations, and tools to build faster and spend less time managing your database.

I'm going to be using an instance of the Cloud Mongo database.

Navigate to the MongoDB Atlas page
Click "Start Free" and sign up for the MongoDB account
On the "Projects" page click on "New Project" give it a name and create
Add Members. You're already a member -> hit continue
Build Cluster -> Select Free Tier
Select Cloud Provider & Region and Create Cluster

After the cluster is initialized click on "connect"

Whitelist a connection IP address -> Add a Different IP Address -> Enter 0.0.0.0/0 to the IP Address to access this DB from anywhere.

Create a MongoDB User -> Enter Username and Password
You'll be using this user to connect to the DB instance. Finally hit Create MongoDB User

Choose a connection method -> Select Connect Your Application and select Node.js

Add your connection string to your application code
Copy and save your application string

"mongodb+srv://test:<password>@cluster0-yvwjx.mongodb.net/<dbname>?retryWrites=true&w=majority"

Nice. We have a URL to the cloud DB instance to which we can connect from our code but we don't have a DB yet. Let's go and create a new DB.

Navigate to the Collections tab and click Add my Own Data

Give DATABASE NAME/COLLECTION NAME and hit Create

After the setup you should see your cluster running:

We can insert some document/data into our database manually or via code execution. We are done here.

Setting up a GraphQL API with Apollo Server

Right now we don't have any graphql setup in our application. When we navigate to the http://localhost:3000/api/hello we'll see

{ "name": "John Doe" }

the output which is served from the pages/api/hello.js file.

What we need is to create a new endpoint under pages/api/graphql.js that's where our Apollo Server GraphQL setup will be located. Call to GraphQL API will be served from http://localhost:3000/api/graphql.

Install apollo-server-micro graphql mongodb

Let's install packaged required for the Apollo Server setup

yarn add apollo-server-micro graphql mongodb

Create a basic GraphQL server

Add graphql.js file under pages/api/.

// pages/api/graphql.js
import { ApolloServer, gql } from 'apollo-server-micro'

const typeDefs = gql`
  type Query {
    sayHello: String
  }
`

const resolvers = {
  Query: {
    sayHello(parent, args, context) {
      return 'Hello World!'
    },
  },
}

export const config = {
  api: {
    bodyParser: false,
  },
}

const apolloServer = new ApolloServer({ typeDefs, resolvers })
export default apolloServer.createHandler({ path: '/api/graphql' })

We created a new instance of the ApolloServer, passing our type definitions and resolvers to it and serving this graphql on /api/graphql path.

When you navigate to the http://localhost:3000/api/graphql you should see a GraphQL Playground where you could execute mutation/queries.

That's great but our API doesn't do much for the moment. It was just for testing. Let's add a MongoDB connection.

Adding MongoDB connection to our Apollo Server

Before adding a MongoDB connection let's talk about data. For the example purpose our application will be displaying a list of users from MongoDB.

Here's my data representation:

{
  "users": [
    {
      "id": 1,
      "firstName": "Alexander",
      "lastName": "Grischuk",
      "blog": "https://grischuk.de/",
      "stars": 5
    },
    {
      "id": 2,
      "firstName": "Max",
      "lastName": "Mustermann",
      "blog": "mustermann.de",
      "stars": 3
    }
  ]
}

I'll insert it manually into the MongoDB:

Creating executable schema and connecting mongo client to DB

Graphql schema is a combination of typeDefs and resolvers.

To make schema executable we need to install graphql-tools

yarn add graphql-tools

Let's describe a data query in our typeDefs and resolvers. We want to query a list of users from the MongoDB.

// pages/api/graphql.js
import { ApolloServer, gql } from 'apollo-server-micro'
import { makeExecutableSchema } from 'graphql-tools'
import { MongoClient } from 'mongodb'

const typeDefs = gql`
  type User {
    id: ID!
    firstName: String!
    lastName: String!
    blog: String
    stars: Int
  }

  type Query {
    users: [User]!
  }
`

const resolvers = {
  Query: {
    users(_parent, _args, _context, _info) {
      return _context.db
        .collection('users')
        .findOne()
        .then((data) => {
          return data.users
        })
    },
  },
}

const schema = makeExecutableSchema({
  typeDefs,
  resolvers,
})

let db

const apolloServer = new ApolloServer({
  schema,
  context: async () => {
    if (!db) {
      try {
        const dbClient = new MongoClient(
          'mongodb+srv://test:qwerty123@cluster0-yvwjx.mongodb.net/next-graphql?retryWrites=true&w=majority',
          {
            useNewUrlParser: true,
            useUnifiedTopology: true,
          }
        )

        if (!dbClient.isConnected()) await dbClient.connect()
        db = dbClient.db('next-graphql') // database name
      } catch (e) {
        console.log('--->error while connecting with graphql context (db)', e)
      }
    }

    return { db }
  },
})

export const config = {
  api: {
    bodyParser: false,
  },
}

export default apolloServer.createHandler({ path: '/api/graphql' })

Configuring .env variables

It's not recommended checking in your MongoDB URI directly to git for security and deployment convenience. We will make Mongodb URI accessible vie environment variables and pull it from there.

First, install dotenv npm package

yarn add dotenv

Create .env file at the project root with your MONGO_DB_URI

MONGO_DB_URI=mongodb+srv://test:qwerty123@cluster0-yvwjx.mongodb.net/next-graphql?retryWrites=true&w=majority

// pages/api/graphql.js
import { ApolloServer, gql } from 'apollo-server-micro'
import { makeExecutableSchema } from 'graphql-tools'
import { MongoClient } from 'mongodb'

require('dotenv').config()

const typeDefs = gql`
  type User {
    id: ID!
    firstName: String!
    lastName: String!
    blog: String
    stars: Int
  }

  type Query {
    users: [User]!
  }
`

const resolvers = {
  Query: {
    users(_parent, _args, _context, _info) {
      return _context.db
        .collection('users')
        .findOne()
        .then((data) => {
          return data.users
        })
    },
  },
}

const schema = makeExecutableSchema({
  typeDefs,
  resolvers,
})

let db

const apolloServer = new ApolloServer({
  schema,
  context: async () => {
    if (!db) {
      try {
        const dbClient = new MongoClient(process.env.MONGO_DB_URI, {
          useNewUrlParser: true,
          useUnifiedTopology: true,
        })

        if (!dbClient.isConnected()) await dbClient.connect()
        db = dbClient.db('next-graphql') // database name
      } catch (e) {
        console.log('--->error while connecting via graphql context (db)', e)
      }
    }

    return { db }
  },
})

export const config = {
  api: {
    bodyParser: false,
  },
}

export default apolloServer.createHandler({ path: '/api/graphql' })

Testing GraphQL API with MongoDB connection in GraphQL Playground

Navigate to http://localhost:3000/api/graphql and make a query

{
  users {
    id
    firstName
  }
}

Query results from MongoDB connection:

In our MongoClient setup, we initialize a new database connection using new MongoClient() with MongoDB cloud URI read from our .env file. We return db object { db } from our context function to be accessible via _context in our resolvers. That's it! As soon as you have access to the database in your resolvers, you can perform read/write operations there for your queries and mutations.

graphql-apollo-mongodb-example

I created a supporting repository for this article so you can follow along commit-by-commit.

How to create custom Create React App (CRA) templates

Oleksandr Hrishchuk — Tue, 14 Apr 2020 11:45:30 +0000

A Create React App (CRA) templates were introduced quite recently and there is not much info present on how to create them. In this article, I'm going to share my experience of developing a custom template and releasing it to the npm.

CRA templates are a way to automate the routine setup. Basic Create React App CLI execution generates an app without any React Router, linter, prettier setup, so you need to add all of these dependencies on your own and configure them, but why waste time when you know from the beginning that you'll be building an app with Redux, for example. Templates could help with that. It's a way to generate an app with all the deps and tools you need. Of course, if such a template is present ;)

CRA basic templates

Currently, there are only two official CRA templates present. cra-template and cra-template-typescript

you could generate CRA from a template by simply providing --template flag.

For example:

npx create-react-app my-app --template typescript

# or

yarn create react-app my-app --template typescript

Building your own CRA template

1. Starting a project

To start building CRA template I would start with generating a new project with CRA itself

yarn create react-app your-cra-template-name

Adjust package name in your package.json with cra-template-[your-custom-template-name].

Note, that it should start with the cra-template- prefix. This way CRA knows that this would be an actual template.

2. Remove deps and setting up tools

Remove everything you don't need from the CRA app and add everything you need.

3. Create a template

When your project is ready to become a template you need to do some preparation.

3.1 Create template/ folder

From your project root create template/ folder.

mkdir template/

3.2 Add gitignore file to the template/ folder

Add gitignore into the created template folder template/gitignore. You can copy gitignore from the CRA but don't forget to remove the . in front of the file.

cp .gitignore template/gitignore

Please note: it is important that the gitignore file inside the template folder should be without "." in front!

This way CRA would know that you're creating a template and it will replace this file later. Otherwise, you'll receive build failures.

3.3 Create `template.json` at your project root

List all of your dependencies and scripts that you need for the new app created from your template to have. For example

{
  "dependencies": {
    "@types/enzyme": "^3.10.4",
    "@types/jest": "^24.9.0",
    "@types/node": "^12.0.0",
    "@types/react": "^16.9.0",
    "@types/react-dom": "^16.9.0",
    "@types/react-redux": "^7.1.5",
    "@types/react-router-dom": "^5.1.3",
    "@typescript-eslint/eslint-plugin": "^2.16.0",
    "@typescript-eslint/parser": "^2.16.0",
    "enzyme": "^3.11.0",
    "enzyme-adapter-react-16": "^1.15.2",
    "eslint": "^6.8.0",
    "eslint-config-airbnb-typescript": "^6.3.1",
    "eslint-config-prettier": "^6.9.0",
    "eslint-plugin-import": "^2.20.0",
    "eslint-plugin-jsx-a11y": "^6.2.3",
    "eslint-plugin-prettier": "^3.1.2",
    "eslint-plugin-react": "^7.17.0",
    "eslint-plugin-react-hooks": "^2.3.0",
    "prettier": "^1.19.1",
    "react-redux": "^7.1.3",
    "react-router-dom": "^5.1.2",
    "react-scripts": "3.3.0",
    "redux": "^4.0.5",
    "redux-devtools-extension": "^2.13.8",
    "typescript": "~3.7.2"
  },
  "scripts": {
    "lint": "eslint --ext .js,.jsx,.ts,.tsx ./",
    "fix": "eslint --ext .js,.jsx,.ts,.tsx  --fix ./"
  }
}

3.4 Copy `src/` and `public/` folders into your `template/` folder

cp -a ./src/. template/src && cp -a ./public/. template/public

3.5 Copy any configuration files that you might want to include with your template

For example

cp .eslintignore .eslintrc .prettierrc tsconfig.json README.md template/

Prepare the template for publishing to the NPM

Prepare your `package.json`

Add the license, author, description, description, keywords, repository URL, bugs URL. You find a good example in my template package.json

Add `main` and `files` properties

It's important to point CRA where to look while building your app from the template. Adding 'main' and files points to proper files.

{
  "main": "template.json",
  "description": "A quick start Create React App template with React Router, Redux, TypeScript, Enzyme and custom ESlint configurations",
  "keywords": [
    "react",
    "create-react-app",
    "cra-template",
    "template",
    "enzyme",
    "eslint",
    "redux",
    "react-redux",
    "react-router",
    "react-router-dom",
    "typescript",
    "prettier"
  ],
  "files": ["template", "template.json"],
  "repository": {
    "type": "git",
    "url": "https://github.com/alexandr-g/cra-template-typescript-redux",
    "directory": "/"
  }
}

I'd suggest using pre-publish script to make it easier for incremental template updates

For your convenience, you can add pre-publish scripts

{
  "clean-files": "rm -rf ./template/public ./template/src && rm -f ./template/.eslintrc",
  "copy-files": "cp -a ./src/. template/src && cp -a ./public/. template/public && cp .eslintrc template/",
  "prepublishOnly": "yarn clean-files && yarn copy-files"
}

Publishing to the NPM

When everything is ready you can publish your package to the NPM.

Create npm account

If you haven't already, navigate to the npm sign up and create an account.

Login to npm via terminal

From your project root login to npm with your credentials

npm login

Publish your package to the npm with public access

npm publish --access public

This is how successful release looks like.

If the name of your package is already taken you'll receive an error on this step - just try another name. When publishing was successful you can start using your template. Happy coding!

A quick start Redux + TypeScript Create React App template

I created an opinionated quick start Create React App (CRA) template with configured Redux, TypeScript, React Router, Enzyme and custom ESlint configuration.

Just install and play with it already. It configures TypeScript with Redux project with tons of useful tools and configurations, like automatic code formatting and ESLint setup for the TypeScript.

Usage

npx create-react-app your-project-name --template typescript-redux

yarn create react-app your-project-name --template typescript-redux

npx command installs the most recent stable version of CRA from npm.

--template parameter points to this template, note that cra-template- prefix is omitted.

Motivation

You know the pain. You start a new project from scratch and need to configure it again and again. It needs routing, ok you setup Router, then you need Redux - ok, oh 😩 Redux boilerplate is taking so much time to type. Wait... what if you could have all the tools you want just from the beginning? I want to focus on building amazing projects and not spending hours configuring them. That's why I've created this template. It's here for you to use.

Try now!

Redux + TypeScript Create React App template on GitHub

Redux + TypeScript Create React App template on NPM

Originally posted on oleks.pro

Migrating tasks and notes from Wunderlist to Joplin

Oleksandr Hrishchuk — Mon, 13 Apr 2020 20:52:37 +0000

As a software developer living in Berlin, I like to support awesome apps, companies, and products developed here. I was an early adopter of N26(until they canceled my account), Blinkist and Wunderlist. Until recent.

For those of you who don't know Wunderlist is a To-do list, Reminders, Errands app with the support of multiple platforms, from the web to mobile.

Wundeslist was an awesome product, fast and simple. It becomes essential for me on day-to-day use. It was working perfectly for me.

Backstory

As it happens to many awesome products Microsoft Corporation bought the Wunderlist app. And they decided to use the Wunderlist team to build their own Microsoft To Do app.

What's is sad about that? Well, two things:

Wunderlist will be shut down on May 6, 2020
MS killed another awesome product

Okay, what to do

Some people are looking for alternative solutions right now and platforms to migrate their data.

In this article, I'll describe how to migrate your data from Wunderlist to a free, open-source and self-hosted solution keeping your notes and todos.

A lot of people might switch directly to the "new" Microsoft To Do app and that's completely fine. In my opinion, Microsoft To Do is not even close to Wunderlist and they even made some things worse with it. It's doesn't look like an app I would love to use daily for multiple reasons. You need MS account, you don't own or control your data and you don't know how it's been used or handled.

As an engineer, I always was interested in the open source, cloud and security.

Introducing Joplin

Joplin is a free, open source note taking and to-do application, which can handle a large number of notes organised into notebooks. The notes are searchable, can be copied, tagged and modified either from the applications directly or from your own text editor. The notes are in Markdown format.

When I found that it exists I couldn't believe it. THIS IS SO AMAZING!

Before jumping in and setting up Joplin, let's export out existing data from Wunderlist.

Step 1. Export your data from Wunderlist

Navigate to the https://export.wunderlist.com and export all your
data. Log in with your Wunderlist account and wait for the download to complete. You will get a zip-file.

Step 2. Convert the Wunderlist export to a Joplin import format

I used this Wunderlist2Joplin Python script to convert Wunderlist export to a Joplin import format. Follow instructions from the above-mentioned repository. They're pretty clear.

Make sure to have Python 3 on your machine
I had to install dateutil library before the script could work

You can install dateutil by running in the terminal:

pip install python-dateutil

# or if your default version of python is <= 2.x.x

pip3 install python-dateutil

If you installed Python with brew you can check which version do you have by running

brew info python

Step 3. Import converted raw data to Joplin

Import the output in Joplin via ‘File’ > ‘Import’ > ‘Raw Joplin Export Directory’. The result is a notebook Wunderlist Import with the same tree structure as your Wunderlist data, with the tasks as leaves, i.e. notes of the type to-do.

Rearrange the material as you wish. Tidy up the data.

One thing to mention it will export also all of your completed tasks.

Step 4. Enable End-to-end encryption for your notes

A nice thing is that Joplin supports End-to-end encryption of your notes so it will be transmitted in the encrypted state over the internet. It is encrypted with the password you choose, so-called Master Password and only you can decrypt the notes using and knowing the Master Password.

You need to activate End-to-end encryption in the Joplin. Navigate to "Preferences" => "Encryption". Activate it and give your password.

Step 5. Synchronize notes

Now, the final step. Your data will be synchronized to the cloud encrypted with your master password.

With Joplin, you can choose multiple synchronization options for your data.

The notes can be synchronised with various cloud services including Nextcloud, Dropbox, OneDrive, WebDAV or the file system (for example with a network directory)

I will use the default DropBox folder option.

Click on the "Synchronise" button at the bottom left of the Joplin Desktop application to initialize the sync process. Authorize Joplin on your dropbox following the instructions. I'd recommend initializing the sync process from the desktop app.

Sync process might take a while, it depends on how many items do you have in your lists. I'd recommend waiting and try do not interrupt it.

Step 6. Accessing notes from multiple devices on mobile and different machines

I'd recommend to start sync on the following devices only after the initial sync from your desktop app is done. This way you'll avoid conflicts. Do not rush and create on each new device its master password. Wait until initial sync is done, only then start to sync the second device and on the second device do not create any new master passwords! You have one already. That's enough.

If your notes are encrypted you'll need to wait until initial data sync is done and on a second device you will see a message "Press to set the decryption password. Click on that message and enter your master password to decrypt the notes.

This is how the Joplin app folder looks on the dropbox

This is how encrypted note stored in the dropbox (.md file)

Joplin Advantages

Fee
Open source
Fast
Sync capabilities to your own cloud (or services like DropBox)
Accessible anywhere. Desktop, mobile, cross-platform support
End-to-end encryption for your notes
Supports Markdown

Enjoy the control of your data :)

Forem: Oleksandr Hrishchuk

Visual regression tests for Storybook with Loki

What is Loki?

Why Loki?

How to setup Loki for your project?

1. Install Loki package

2. Create a Loki config file

3. Install Docker

4. Start Storybook

5. Generate reference images

6. Run tests

7. Add Loki to CI (as GitHub action)

Skipping stories

Approve changes

Accessing artifact files from GitHub action

Pros

Cons

Comparison stats - VRT tests

Conclusion

Upload files to AWS S3 bucket using official Amazon Docker image and AWS CLI with GitLab CI

How to upload files to AWS S3 using amazon/aws-cli s3 sync command with GitLab CI

Create .gitlab-ci.yml

Working on the deployment stage

Conclusion

How to setup Apollo Client in order to perform GraphQL queries with NextJS

An idea

Installing Apollo Client and related dependencies

Setting up an Apollo Client

Sharing schema for Apollo Client and Server

Create Users component and perform queries to the GraphQL API

Wrap Index file withApollo() HOC

graphql-apollo-mongodb-example

Setting up GraphQL API with MongoDB and Apollo Server for a NextJS app

Initialize a NextJS default app

Setting up MongoDB Atlas

Setting up a GraphQL API with Apollo Server

Install apollo-server-micro graphql mongodb

Create a basic GraphQL server

Adding MongoDB connection to our Apollo Server

Creating executable schema and connecting mongo client to DB

Configuring .env variables

Testing GraphQL API with MongoDB connection in GraphQL Playground

graphql-apollo-mongodb-example

How to create custom Create React App (CRA) templates

CRA basic templates

Building your own CRA template

1. Starting a project

2. Remove deps and setting up tools

3. Create a template

3.1 Create template/ folder

3.2 Add gitignore file to the template/ folder

3.3 Create template.json at your project root

3.4 Copy src/ and public/ folders into your template/ folder

3.5 Copy any configuration files that you might want to include with your template

Prepare the template for publishing to the NPM

Prepare your package.json

Add main and files properties

I'd suggest using pre-publish script to make it easier for incremental template updates

Publishing to the NPM

Create npm account

Login to npm via terminal

Publish your package to the npm with public access

A quick start Redux + TypeScript Create React App template

Usage

Motivation

Migrating tasks and notes from Wunderlist to Joplin

Backstory

Okay, what to do

Introducing Joplin

Step 1. Export your data from Wunderlist

Step 2. Convert the Wunderlist export to a Joplin import format

Step 3. Import converted raw data to Joplin

Step 4. Enable End-to-end encryption for your notes

Step 5. Synchronize notes

Step 6. Accessing notes from multiple devices on mobile and different machines

Joplin Advantages

Wrap Index file `withApollo()` HOC

3.3 Create `template.json` at your project root

3.4 Copy `src/` and `public/` folders into your `template/` folder

Prepare your `package.json`

Add `main` and `files` properties