Forem: Andrea Carraro

Unlocking API Interoperability: Converting OpenAPI to TypeScript JSON Schema

Andrea Carraro — Mon, 04 Sep 2023 07:51:44 +0000

Intro

This post briefly tells the story of why I wrote openapi-ts-json-schema to fill the unexpected gap existing between OpenAPI specification and TypeScript JSON schema format.

Prologue

In the last few weeks at work I was faced with an obvious task which I thought had an obvious technical solution, from my naive fronted engineer background:

Given an OpenAPI Specification, generate the relevant JSON schema files to be consumed and type-interpreted by a TypeScript application.

The main idea consists of leveraging a library like json-schema-to-ts to infer TypeScript types from the generated JSON schema files. This is a quite common pattern documented by Ajv and Fastify, to name a few:

import Ajv from 'ajv';
import type { FromSchema } from 'json-schema-to-ts';
import mySchema from 'path/to/generated/schemas/MyModel.ts';

const ajv = new Ajv();
// Perform data validation and type inference using the same schema
const validate = ajv.compile<FromSchema<typeof mySchema>>(mySchema);
const data: unknown = {};

if (validate(data)) {
  // data gets type inference
  console.log(data.foo);
} else {
  console.log(validate.errors);
}

Since the OpenAPI Specification is a "machine-readable interface definition language for describing, [...] web services", and TypeScript is practically the only available type solution for Node.js, I confidently googled:

"openAPI to typescript JSON schema"

...and found no relevant results.

OpenAPI to TypeScript JSON schema gap

If OpenAPI -> TypeScript types is mostly covered by openapi-typescript and OpenAPI -> JSON schema is covered by openapi-schema-to-json-schema, there is no clear way to generate TypeScript JSON schemas from an OpenAPI definition.

The main shortcomings consist of TypeScript not being able to:

Interpret/resolve JSON schema's $ref and $id props
Import JSON files as const (See relevant GitHub thread)

const dogSchema = {
  type: "object",
  properties: {
    name: { type: "string" },
    // TS cannot interpret/resolve $refs
    favoriteFood: { "$ref": "components/schemas/Food" },
  },
  required: ["name", "favoriteFood"],
} as const;

Considering the example above, in order to have a JSON schema which TypeScript can properly interpret, all $ref entries should be somehow statically resolved at generation time.

After an initial attempt to manually replace the $refs all around my JSON schemas, I ended up putting together a few lines to automate the process which I eventually abstracted into a JS NPM package: openapi-ts-json-schema.

The generation flow consists of:

Resolve external/remote $refs and dereference them with @apidevtools/json-schema-ref-parser (resolving $ref's)
Convert to JSON schema with @openapi-contrib/openapi-schema-to-json-schema and openapi-jsonschema-parameters
Generate one TypeScript JSON schema file for each definition (.ts files with as const assertion)
Store schemas in a folder structure reflecting the original OpenAPI definition structure

The example above would result into a TypeScript JSON schema file, ready to be imported and consumed:

import componentsSchemasFood from "./../components/schemas/Food"

export default {
  type: "object",
  properties: {
    name: { type: "string" },
    favoriteFood: componentsSchemasFood,
  },
  required: ["name", "favoriteFood"],
} as const;

Since v0.3.0, $ref definitions get generated as standalone TS JSON schema files and imported where needed.

Conclusion

Given the centrality of JSON schemas in most of web development, I would be glad to see TypeScript taking over and make JSON schemas first class citizens by adding native support for:

importing JSON schema import as const
inferring types natively from any JSON schema (currently achieved with the outstanding json-schema-to-ts)

TypeScript JSON schemas are 100% valid JSON schemas and would be able to represent the missing joining link between runtime type validation and static type check.

In the meanwhile feel free to take a look at the approach taken by openapi-ts-json-schema and start a discussion there in case something doesn't fit your use case.

Bringing together testing and development environment with Mock Service Worker

Andrea Carraro — Tue, 22 Mar 2022 16:52:54 +0000

This post is a brief write-up about how Mock Service Worker radically improved testing experience and -surprisingly- overall development flow on a couple frontend projects I recently worked on.

Testing: mock API to support test

I initially chose Mock Service Worker just as a Fetch mock replacement, for the only reason that it intercepts network requests at the network layer enabling mocking any requests including Axios's XMLHttpRequest ones.

msw approach favors building API mocks in a centralized way so that all tests run against the same set of mocks. (Still, any test can extend msw mock handlers on the fly, when strictly necessary).

As the projects have scaled, msw mocks ended up simplistically replicating most of the consumed API. This opened to...

Development: using the same mocks

Since msw handlers replicate most of API dependencies, I could run the same mocks used by tests in the browser to run the development environment. For free.

Running tests and development environment with the same mocks turned out to be a huge improvement for both testing and development.

Debugging tests

Broken tests could be debugged more easily be replicating the testing scenario in browser. Same mocks means same application state in tests and browser.

Decouple local development from API

Developing in the browser using local mocks resulted in a faster and more flexible development flow: local mocks take almost no time to load but the keep the exact same async flow of real API responses.

Local mocks are easy to extend or change, therefore replicating a new API scenario is trivial.

I ended up using local mocks most of my development time and contacting the actual API in very rare cases.

Storybook

In case you used Storybook, msw integrates seamlessy with it, too.

MSW inspector: asserting against mocked requests

msw encourages an end-to-end testing approach, decoupled from code's implementation details, which I also recommend:

Instead of asserting that a request was made, or had the correct data, test how your application reacted to that request.

Even though, I sometimes found it necessary asserting against a specific sent request. Like ensuring that a post/put request is issued with proper payload.

For this reason msw provides an API to listen to intercepted requests.

msw-inspector is a tiny utility I wrote to inspect msw intercepted requests and enable any testing framework to assert against them. It's open source and published as an npm package.

expect(mswInspector.getRequests('http://my.url/path')).toHaveBeenCalledWith({
  method: 'POST',
  headers: {
    'my-header': 'value',
  },
  body: {
    'my-body': 'value',
  },
  query: {
    'my-query': 'value',
});

Switching from React To Vue.js: don't lock yourself in

Andrea Carraro — Fri, 19 Mar 2021 14:52:39 +0000

This year I happened to onboard a Vue.js based company after several years of mainly React related activities. This is how I managed to approach this new ecosystem.

These notes are a work in progress. I'll keep them updated as long as I get familiar with Vue ecosystem.

Stay away from Vue cli tools

Vue provides a set of optional cli integrations to provide additional plug-and-play functionalities to the core one. Most of them are not extensible and poorly maintained.

My suggestion is to ignore them and use universal tools configured to work with Vue (eg. jest instead of @vue/cli-plugin-unit-jest, eslint instead of @vue/cli-plugin-eslint).

I'm currently just using vue-cli-service dev server to run my development environment. Compile with Rollup (even though rollup-plugin-vue is still quite flaky).

Vue + Typescript

Vue (at least v2) and Vue's Single File Components don't play nice with Typescript, but luckily there are tools that can partially fill this gap.

Vuex + Typescript

If using Vuex these helper libraries can help to bring store types into your components:

I eventually opted for Typed Vuex, which seems to be the current de facto official Vuex/TypeScript library. It definitely does its job with a minimal impact over the existing codebase.

The only minor downside consists on the fact that every component connected to the store is supposed to import a storeAccessor object exposed by the same store instance used by the application. Not a real blocker in my opinion.

Type checking Vue components

Vue components (and especially their Vue Template Syntax) are unfortunately invisible to typescript.

Vetur and VueDX (still in alpha) expose 2 cli component type checkers which I've not been able to use with a Vue 2 project:

Luckily Vetur provides a working component type checking in VSC by enabling experimental template interpolation service in your config:

vetur.experimental.templateInterpolationService: true

...not the best solution but better then nothing.

Replacing Vue template with tsx

Since Vue 2 type checking experience turned out to be quite a failure, I considered the option of skipping Vue templates (and Single File Component pattern) and just use TSX, which is type checkable by definition.

I've found a very promising library which provides the necessary glue between Vue and Typescript + TSX.

I've toyed with vue-tsx-support a couple hours with encouraging results, but I couldn't actually suggest my team to base our project on a library not officially supported by Vue.

Write framework-agnostic tests

When it comes to write unit tests, don't lock yourself into Vue ecosystem but use an abstraction on top of this.

I'm using vue-testing-library with msw (to mock network responses) and it works as well as react-testing-library.

General considerations

Vue ecosystem is not mature as React's. A significant number of Vue-specific libraries are solo projects with uncertain present and future.

As a general approach I'm trying to rely on Vue ecosystem only for what is strictly necessary and find a way of using tools built outside of Vue realm. On the long term I expect Vue to open to established standards of web frontend industry.

Just an example. Vue's Single File Component pattern seems a good idea in the first half hour, until you realize it places your code out of any language standard, preventing any third party tool from being able to parse it.

Contextual routing and modal routes in Next.js

Andrea Carraro — Sun, 18 Oct 2020 20:24:49 +0000

Contextual routing is a widespread UI technic made popular by applications like Facebook, Instagram and Reddit usually in the shape of Modal Routes.

Modal routes pattern consists of opening a modal while temporary replacing the current URL (usually with the one pointing to the resource being displayed in-modal):

Basic contextual routing

Next.js router recently added support for contextual routing by simply providing Link component with the relevant href + as props.

import Link from 'next/link';

<Link
  href="/post-list?postId=42"
  as="/post/42"
>
  <a>Open modal route</a>
</Link>

The above example would update browser address' path to /post/42 while rendering the page specified as href (/post-list with postId parameters equal to 42).

This leads to 2 possible outcomes:

user closes the modal and the URL is restored to the value where contextual routing was started (in our case /post-list)
user refreshes the page landing on the actual page described by the URL

HREF property

The minimal necessary information to render a Next.js page consists of:

page pathname (eg. /post/[id])
page params (eg. id=42)

The main idea behind contextual routing consists of creating a divergence between the URL displayed and the page actually rendered.

Since contextual navigation replaces the URL, it means that rendered pages cannot rely anymore on the URL to retrieve pathname and relative page params.

href prop plays therefore the crucial role of supplying the aforementioned information as a single string:

// page pathname + all required params as query string
const href = router.pathname + `?param1=1&param2=2`

Real world scenario

Before starting contextual routing navigation you need to know 3 information beforehand:

as path (the path displayed during contextual routing)
href path
return href (the path to return to to end contextual routing)

href and return href are trivial to get when the starting page has a static path, let's say: /post-list.

Things gets trickier when starting page path is not statically defined, in other words when the path includes dynamic parameters like: /post-list/bob (where bob is an author name).

This means href has to be generated from initial page pathname plus route params and then persisted during the whole contextual routing navigation in order to keep the page alive.

On top of this a developer might want to extend available route params with extra ones to be made available during contextual navigation. Eg:

const as = 'post/42'
const returnHref = 'post-list/bob'
const href = 'post-list/[author]?author=bob&id=42'

The example above provides id=42 as extra param to make the modal route aware of which post should be displayed.

This demo shows a basic implementation of what you just read.

The devil is in the details

There's is an extra pitfall worth mentioning: the application would loose reference to returnHref after a page reload followed by one or more back button presses.

This means the application won't be able to restore the initial URL while the modal route is open, breaking the user flow or forcing Next.js to reload the page.

One hook to rule them all

In order to overcome this issue and make contextual routing setup trivial, I wrapped the necessary boilerplate logic in a React hook published as next-use-contextual-routing.

You can read the source code and star it on github. It's fully tested and weights ~0.5 kb gzipped.

import Link from 'next/link';
import { useContextualRouting } from 'next-use-contextual-routing';

//...
const { makeContextualHref, returnHref } = useContextualRouting();

<Link
  as="/post/42"
  href={makeContextualHref({ id: 42 })}
  shallow
>
  <a>Open modal route</a>
</Link>

The hook returns 2 values:

makeContextualHref: a function returning the href value necessary to start contextual navigation. It optionally accepts an object providing extra href parameters.

returnHref: the path to return to to close contextual navigation.

The demo mentioned above makes use of next-use-contextual-routing hook. Check it out to make sure it can suit your needs.

DOM testing Next.js applications

Andrea Carraro — Sat, 26 Sep 2020 22:27:53 +0000

This article analyses the current status of DOM tests in Next.js and presents a utility library (next-page-tester) to get back DOM integration tests alongside Next.js apps.

The missing tile

Next.js comes with an outstanding tool belt out of the box, providing any web project with great dev experience and productivity since day one.

The value provided by Next.js is truly priceless, but there's a missing tile: DOM integration testing (React Testing Library or Enzyme tests, to put it simple) is quite uneffective and fragmented when it comes to Next.js.

Let's see why.

Why

An average Next.js project is made of disconnected pieces which are glued together by Next.js internals on next dev or next build.

This means less boilerplate code 👍but also no explicit relations among separate parts 😔. In other words: only Next.js knows how to put your application together.

This affects the way unit tests for Next.js apps are written: isolated tests targeting different parts and optimistically mocking what's in-between.

Let's be more explicit. This involves:

testing individual page components providing the expected server-side generated props
testing pages' data fetching methods (getServerSideProps and getStaticProps) providing the expected context object
mocking NextRouter object with relevant current route data (path, params, querystring...) when the tested page makes use of Next's Link, useRouter or withRouter
hoping that page file names are 100% correct, since Next.js relies on those to configure file system routing
rendering custom App and Document components, if the case

All this is feasible but, where are the times when we could write integration tests by rendering the whole components tree in Node.js with JSDOM?

Write tests. Not too many. Mostly integration.

https://twitter.com/rauchg/status/807626710350839808

Since business logic is spread among different disconnected places (and the fact that Next.js runs a web server), the currently suggested way of testing Next.js applications consists of running e2e tests against a fully fledged instance of the app.

With the rise of tools like Cypress and Test Café, e2e tester lives became significantly easier but, as everything in life, there's no silver bullet™ and browser tests make no exception. It'd be cool to be able to grab the right tool for the right task.

The internet is packed with resources documenting the trade-offs of different testing strategies. It's mostly a matter of cost, speed and determinism. This is a popular article by Kent C. Dodds to name one 🔗.

DOM Integration tests for Next.js

Ideally, a promising integration testing routine might consist of testing Next.js apps by route: given a route path, I receive the matching page element ready to be tested with any DOM testing library:

const Page = getNextJsPage('/blog/1');
render(Page);

In order to get to such testing routine we need to replicate part of Next.js glue:

resolving provided routes into the matching page component
calling Next.js data fetching methods (getServerSideProps, getInitialProps or getStaticProps)
set up the expected mocked next/router provider
wrapping page with custom _app/_document component
instantiating page component with the expected props
Emulate client side navigation via Link, router.push, router.replace

I put together this logic in next-page-tester, an utility library which enables approaching tests like:

import { screen, fireEvent } from '@testing-library/react';
import { getPage } from 'next-page-tester';

describe('Blog page', () => {
  it('renders blog page', async () => {
    const { render } =  await getPage({
      route: '/blog/1',
    });

    render();
    expect(screen.getByText('Blog')).toBeInTheDocument();

    fireEvent.click(screen.getByText('Link'));
    await screen.findByText('Linked page');
  });
});

next-page-tester is available on NPM and aims to make DOM integration tests first class citizens in Next.js.

It's written in Typescript, fully tested and open to receive any feedback from curious Next.js users. Hop over to next-page-tester GitHub page to see how it works or lend hand :).

Use next.js with react-router

Andrea Carraro — Sun, 28 Jul 2019 18:11:16 +0000

This repo documents an attempt of using Next.js (preserving native SSR features) with the following setup:

Single entry point (like Create React App and Hops). No file system-based routing
react-router as only routing system

This document is available as:

Disclaimers

Next.js team strongly advises against this approach.
This experiment was carried out at the times of Next.js v9.3: the framework has changed a lot since then.

Part one, basic setup

1 - Install Next.js

Relevant repo commit.

Install NextJS as usual and create the single entry point file at pages/index.js.

2 - Redirect all requests to single entrypoint

Relevant repo commit.

In order to skip file system-based routing, we'll configure a custom Next.js server to forward all the requests to our single entrypoint.

We'll use Next.js Server.render method to render and serve the entrypoint.



// server.js
const express = require('express');
const nextJS = require('next');

async function start() {
  const dev = process.env.NODE_ENV !== 'production';
  const app = nextJS({dev});
  const server = express();
  await app.prepare();

  // Redirect all requests to main entrypoint pages/index.js
  server.get('/*', async (req, res, next) => {
    try {
      app.render(req, res, '/');
    } catch (e) {
      next(e);
    }
  });

  server.listen(3000, err => {
    if (err) throw err;
    console.log(`> Ready on http://localhost:3000`);
  });
}

start();

Run the dev server, and the entrypoint page at pages/index.js should be served as response for any requested url. 👊

3 - Introduce react-router

Relevant repo commit.

In order to get different responses according to the requested url we need a routing system.

We'll use react-router (see it's docs about SSR) and wrap the application with a StaticRouter or a BrowserRouter based on the environment application environment (server or browser).

Install react-router and react-router-dom:



npm i react-router react-router-dom -S

...and update the pages/index.js entrypoint to use some Link and Route components from react-router-dom (see repo).

Let's now declare a withReactRouter HOC to wrap the application with the proper router:



// next/with-react-router.js
import React from 'react';
import {BrowserRouter} from 'react-router-dom';
const isServer = typeof window === 'undefined';

export default App => {
  return class AppWithReactRouter extends React.Component {
    render() {
      if (isServer) {
        const {StaticRouter} = require('react-router');
        return (
          <StaticRouter
            location={this.props.router.asPath}
          >
            <App {...this.props} />
          </StaticRouter>
        );
      }
      return (
        <BrowserRouter>
          <App {...this.props} />
        </BrowserRouter>
      );
    }
  };
};

...and wrap the application with withReactRouter HOC:



// pages/_app.js
import App, {Container} from 'next/app';
import React from 'react';
import withReactRouter from '../next/with-react-router';

class MyApp extends App {
  render() {
    const {Component, pageProps} = this.props;
    return (
      <Container>
        <Component {...pageProps} />
      </Container>
    );
  }
}

export default withReactRouter(MyApp);

Run the dev server, and you should be able to see your routes live and server side rendered.

Part two, context information

One of my favourite react-router features consists of the possibility of adding context information during the rendering phase and returning server side responses based on the information collected into the context object.

This enables client side code to take control of the responses returned by the node server like returning a HTTP 404 instead of a "not found page" or returning a real HTTP 302 redirect instead of a client side one.

In order to achieve this behaviour we have to configure Next.js to do the following:

render the requested page providing a context object to the app router
check whether context object was mutated during the rendering process
decide whether to return the rendered page or do something else based on context object

4 - Provide context object to the router

Relevant repo commit.

We'll inject an empty context object into Express' req.local object and make it available to the router application via React Context.

Let's inject context object into Express' req.local object:



// server.js
server.get('/*', async (req, res, next) => {
  try {
+   req.locals = {};
+   req.locals.context = {};
    app.render(req, res, '/');

Next.js provides a req and res objects as props of getInitialProps static method. We'll fetch req.originalUrl and req.locals.context and handle it over to the static router.



// next/with-react-router.js
  return class AppWithReactRouter extends React.Component {
+   static async getInitialProps(appContext) {
+     const {
+       ctx: {
+         req: {
+           originalUrl,
+           locals = {},
+         },
+       },
+     } = appContext;
+     return {
+       originalUrl,
+       context: locals.context || {},
+     };
+   }

  // Code omitted
          <StaticRouter
-           location={this.props.router.asPath}
+           location={this.props.originalUrl}
+           context={this.props.context}
          >

5 - Separate rendering and response

Relevant repo commit.

Since we want to provide extra server behaviours based on req.locals.context in-between SSR and server response, Next.js Server.render falls short of flexibility.

We'll re-implement Server.render in server.js using Next.js Server.renderToHTML and Server.sendHTML methods.

Please note that some code was omitted. Refer to the source code for the complete implementation.



// server.js
  server.get('/*', async (req, res, next) => {
    try {
+     // Code omitted

      req.locals = {};
      req.locals.context = {};
-     app.render(req, res, '/');
+     const html = await app.renderToHTML(req, res, '/', {});
+
+     // Handle client redirects
+     const context = req.locals.context;
+     if (context.url) {
+       return res.redirect(context.url)
+     }
+
+     // Handle client response statuses
+     if (context.status) {
+       return res.status(context.status).send();
+     }
+
+     // Code omitted
+     app.sendHTML(req, res, html);
    } catch (e) {

Before sending the response with the rendered HTML to the client, we now check the context object and redirect or return a custom HTTP code, if necessary.

In order to try it out, update the pages/index.js entrypoint to make use of <Redirect> and <Status> components and start the dev server.

Summary

We showed how it's be possible to setup Next.js take full advantage of react-router, enabling single entrypoint approach and fully preserving SSR.

In order to do so we:

Redirected all server requests to a single entrypoint
Wrapped the application (using HOC) with the proper react-router router
Injected req server object with a locals.context object
Provided HOC wrapper with req.locals.context and req.originalUrl
Extended next.js Server.render to take into account req.locals.context before sending HTML

The re-implementation of Server.render in userland code is the most disturbing part of it, but it might be made unnecessary by extending a bit Server.render API in Next.js.

Results

`react-router` rendered server side

react-router's <Route> components get statically rendered on the server based on received req.originalUrl url.

HTTP 302 redirect triggered by client code

When server rendering process encounters <Redirect from="/people/" to="/users/" /> component, the server response will return an HTTP 302 response with the expected Location header.

HTTP 404 triggered by client code

When server rendering process encounters <Status code={404}/> component, the server response returns an HTTP response with the expected status code.

Further consideration

I'm sure this setup is way far from being optimal. I'll be happy take into account any suggestions, feedbacks, improvements, ideas.

Issues

Static pages not being exported
Dev mode cannot build requested route on demand
getInitialProps not implemented

Forem: Andrea Carraro

Unlocking API Interoperability: Converting OpenAPI to TypeScript JSON Schema

Intro

Prologue

OpenAPI to TypeScript JSON schema gap

Conclusion

Bringing together testing and development environment with Mock Service Worker

Testing: mock API to support test

Development: using the same mocks

Debugging tests

Decouple local development from API

Storybook

MSW inspector: asserting against mocked requests

Switching from React To Vue.js: don't lock yourself in

Stay away from Vue cli tools

Vue + Typescript

Vuex + Typescript

Type checking Vue components

Replacing Vue template with tsx

Write framework-agnostic tests

General considerations

Contextual routing and modal routes in Next.js

Basic contextual routing

HREF property

Real world scenario

The devil is in the details

One hook to rule them all

DOM testing Next.js applications

The missing tile

Why

Write tests. Not too many. Mostly integration.

DOM Integration tests for Next.js

Use next.js with react-router

Disclaimers

Part one, basic setup

1 - Install Next.js

2 - Redirect all requests to single entrypoint

3 - Introduce react-router

Part two, context information

4 - Provide context object to the router

5 - Separate rendering and response

Summary

Results

react-router rendered server side

HTTP 302 redirect triggered by client code

HTTP 404 triggered by client code

Further consideration

Issues

`react-router` rendered server side