Forem: Kevin Simons

Fixing a 3 second lockup in our app by switching from Apollo Client to URQL

Kevin Simons — Thu, 09 Feb 2023 09:16:53 +0000

A few weeks ago, the Kitemaker team was working on diagnosing a performance problem one of our larger customers reported. A few seconds after loading Kitemaker, the app would freeze up for several seconds before becoming responsive again. After digging into the problem, we were able to pinpoint that the lockup happened during the fetching of data from our servers. Kitemaker has a two-phased approach to loading data - first the data is loaded from a local offline cache (IndexedDB) in order to get a snappy load, and then we do a fetch from the server to ensure the client always has the latest data. Based on the timing, we could see it was the fetch from the server that was causing the freeze.

There we bunch of places in the code that could be to blame for such a performance issue. Maybe it was our client-side search indexing hogging the CPU? Maybe it was an issue of loading a lot of data all at once into our state management library Recoil? It wasn’t until we did some analysis using Chrome’s profiling tools that we spotted the culprit:

After the network responses were received, > 3 seconds was spent in our GraphQL client, Apollo. Specifically, we were spending a bunch of time and a ton of CPU cycles in the cache layer (InMemoryCache) of Apollo that we’d specifically tried (and apparently failed) to disable.

We suspected we needed to take some drastic action and rip out the Apollo client. But before we get into the details, let’s take a quick walk down memory lane to see how we got here.

A brief history of GraphQL at Kitemaker

We started with GraphQL at Kitemaker on day 1. And in the beginning, we were a pretty traditional-looking GraphQL application. Each screen fetched just the queries it needed from the server and we used Apollo’s state management for everything. This meant Apollo’s InMemoryCache cached all of our data and we frequently wrote updaters to keep that cache in sync after mutations and subscription events.

However, we began to hit some real challenges with Apollo in terms of making it reliable when our users had bad/no internet. Writing optimistic cache updaters was an error-prone pain and we never got the offline experience (persisting mutations client-side until the client came back online, etc.) to work the way we wanted it to.

Therefore, we decided to drop Apollo’s state management in favor of writing our own syncing layer, which was still based on GraphQL. Since the basic functionality of GraphQL was working fine with the Apollo client, we decided to stick with it. We configured it to not do any caching (we thought) and we went on our way:

new ApolloClient({
  link: getLink(),
  cache: new InMemoryCache({ fragmentMatcher }),
  defaultOptions: {
    watchQuery: {
      fetchPolicy: 'no-cache',
      errorPolicy: 'ignore',
    },
    query: {
      fetchPolicy: 'no-cache',
      errorPolicy: 'all',
    },
  },
}),

Apollo’s normalization

So if we’d disabled Apollo’s caching, why were we getting massive amounts of time spent in Apollo’s InMemoryCache? Turns out, the cache actually does two things:

Caches the results of queries (what we disabled)
Normalized caching

What’s normalization mean in this case? If you have a GraphQL query that looks like this:

query MyQuery {
  space {
    labels {
      id
      name
      color
      createdAt
      updatedAt
    }
    workItems {
      id
      title
      description
      labels {
        id
      }
    }
  }
}

Apollo will attempt to match the various objects by ID and create a graph of all of the data in the cache. This means, if the labels on workItems are the same IDs as the ones at the top level of the space query, Apollo will recognize this and build a graph of the data, where accessing labels via the workItems will include all of the properties like name and color even though they weren’t specified. It’s pretty neat! It’s also very costly when you have a lot of data, especially when you’re never actually accessing that graph that Apollo builds.

We tried ripping out the cache entirely, but we didn’t manage to get it to work. It was time to find a new client.

URQL to the rescue

URQL is a lightweight GraphQL client that had been on our radar for a while. It’s almost a drop-in replacement for Apollo if you’re not using Apollo for your state management. However, unlike Apollo, it only (by default) provides caching of GraphQL query results - it does not perform any normalization of the data.

In the places where we were using the Apollo client directly (calling methods like query() and mutation()), switching to URQL was just a matter of massaging parameters into the correct format. Similarly, for the places where we used React hooks (like useQuery() and useMutation()), switching was basically figuring out that the hooks took arguments in a slightly different format and returned some extra objects/callbacks (that we didn’t really care about).

Error handling was a bit of work since there were some places where the Apollo client threw exceptions where URQL just quietly returned an error. Overall though, URQLs CombinedError (which wraps GraphQL errors and network errors together in a single object) is easy to work with and helped us clean up our error handling code.

Additionally, we created a bit more work for ourselves by upgrading the library we use for GraphQL subscriptions over web sockets, moving from the seemingly unmaintained subscriptions-transport-ws to the active graphql-ws project (which is URLQ’s library of choice for subscriptions).

The work took about a week including upgrading some Apollo Server libraries while we were at it.

After we switched to URQL, the performance timeline looked like this:

While there’s still some room for improvement, the lockup is gone and the app spends much less time processing the response.

Wrapping it up

Having made the switch of libraries, we were delighted to see that the application no longer locked up after fetching data in the background. Three seconds we were previously wasting processing data into a format we didn’t need was gone. As an added bonus, we upgraded some GraphQL-related dependencies to their latest-and-greatest versions. If any team is using Apollo but doesn’t require any of the caching or state management aspects, we definitely recommend taking a peek at URQL.

We're building Kitemaker, an alternative to your issue tracker that helps product managers, designers, and engineers collaborate better from ideation to delivery. Click here to learn more.

Images:
Top photo by Stanos

How to write great one-pagers, PRDs, Specs, and more

Kevin Simons — Thu, 19 Jan 2023 09:39:52 +0000

When building a product, it’s important that everyone on the team has a shared understanding of what to build, why it’s important, and a plan for how to make it real. The most common way to get the team aligned is to write a document containing all the required information.

These documents go by many names: one- or six-pagers, PRDs, specification documents, project posters, problem-to-solve documents, and many more. Internally, we just refer to them as feature or bug descriptions, so we'll just call them descriptions here. They all have a similar purpose: to align the team what needs to be built and why, and to outline a plan on how to execute.

We have written many descriptions for teams in larger tech companies and startups. Here are some of our key takeaways:

Start with the team. Don’t start writing a descriptions until you have talked things through with the team. Some PMs don’t involve engineers and designers from the start, but we see that most collaboration challenges within teams are easily solved by having the collaboration start early. Writing great descriptions should be a collaborative task where all the knowledge and expertise of the team should be utilized. By having the team be a part of discovering and understanding user problems and figuring out solutions, you'll get a better product and a more engaged team.
Know your audience. You are writing to a team of product managers, designers, engineers, and anyone else involved in the development process. It’s key to know what they care about and need to know in order to do a good job, as it differs between roles. Which takes us to the next point.
Know what you need to convey. This is going to be different for every team and each feature. However, there are some best practices. Some formats like Jobs-To-Be-Done, user stories, and the templates here covers most of these in various forms. But in general, these are the core parts:
- Write about the user context, the problem to be solved, and why this is important for the user and the business. People make better choices and are more motivated when they understand the business context of what they are working on. To us at Kitemaker, understanding users' problems is a vital part of the process. We won't build anything new until we understand these problems. We will follow up throughout the building process and verify that the problems are solved when the new thing is shipped.
- Be clear about what is expected to be built. There are various ways to do this. F.ex. user stories with acceptance criteria or Jobs-To-Be-Done are a popular models. When we build Kitemaker, we typically describe in enough detail how things are expected to work, and put a lot of the specification into the design files, so we are all on the same page. However, we make sure we don’t over-design the feature upfront, as we want to explore at the same time as we build.
- Make a plan for how to get there. This could involve technical designs, a development plan, the specific tasks that need to be delivered, and so forth. To us, this mostly means a list of things that need to be designed and code that needs to be written. Our engineers love to-dos, so there is usually a long list of them 🙂
Make it concise and to the point. People don’t read documents because they like them, they read them because they contain information that is important to them. Practice writing concisely, and going back to the first three points, information that is not important to share should be put somewhere else.
Make it a living document. In most teams, the document isn’t updated when things change during development. Often one finds that a certain solution needs to be adjusted because of new learnings or because better technical solutions have been found. If someone coming in later in the cycle (like QA) having a nicely written up-to-date description is the easiest way for them to catch up on the context and the intended goal.

Real-world example from Kitemaker

There is no one-size-fits-all approach to writing descriptions, so you need to figure out what works best for you and your team. However, seeing real-world examples might inspire you to find new ways to write them. Here are some examples from descriptions we have written for Kitemaker.

This is the snippet we use to start all new initiatives in Kitemaker (the included todo is there to ensure we never forget tracking):

Why
[Explain why this is needed, cite users and background.
Provide a helpful context for the team to solve this]

What
[Explain the solution or bet we want to make]

How
[] Add tracking

We use the “Why” section to describe the background, the problem users are experiencing, and the context of why this is important.

In some tools you may want to link to user research here. We use the Kitemaker’s insights module for this (coming soon), allowing us to have everything in the same tool and connected to where the team executed.

Here is an example of a “Why” section that we are currently using in Kitemaker:

We use the “What” section to describe what we believe is the solution. Sometimes on high level items (like roadmap themes) it will only be a description of the hypothesis for solutions, while on concrete deliveries we will include designs and, when needed, more detailed explanation of the feature.

Here is an example from when we built a new version of “Insights” in Kitemaker. In this case we flesh out an outline in the document, and the design files contains the details of the functionality.

The “How” section is where we make the plan for how to get things . We write a lot of todos in Kitemaker, since they are easy to use and connect directly to commits and PRs in Github/GitLab.

Here is an example from a new feature we were working on, where a bug resulted in some edge cases where deletion could cause users to get a 404 error:

We don’t believe there is only one way to write great descriptions. Some have great success using models like Shape up, Jobs-To-Be-Done and/or user stories. Others choose a less structured approach like we do in Kitemaker. All are good ways to do it, as long as the audience that reads it find it easy to comprehend and it contains the information they need to do a great job. The important part is to figure out what works for you and your team to get alignment and a common understanding.

Want to learn how Kitemaker will make these documents front and center for your team? Send us a ping, and we’ll help you set it up!

Images: Toop photo by Annie Spratt

Lessons learned from moving to Recoil.js

Kevin Simons — Wed, 12 Oct 2022 11:49:39 +0000

At Kitemaker, we recently made the leap to Recoil.js for our React state management needs. Before using Recoil, Kitemaker used a simple state management solution built upon useReducer(). We built Kitemaker to be super fast, responding to every user interaction instantly. However, in organizations with lots of data, we sometimes had a difficult time achieving this due to unnecessary re-renders. Kitemaker has a sync engine under the hood that is constantly syncing data in the background between clients. With useReducer() this always triggered a top-down re-render and we had to rely on memoization to keep things snappy.

We reached for Recoil to help us minimize re-renders in order to keep Kitemaker fast and responsive regardless of what changes are flowing in via background syncs. We chose it over other competing frameworks like MobX because we liked the explicitness of its API and its similarity to Redux which we were already familiar with. Additionally, the fact that the Meta team designed Recoil for the purpose of building performant UIs with large datasets was a major draw.

While we’ve been very happy with the decision and we have seen the benefits we’d hoped for, it did come with some gotchas.

A little terminology

Let’s get everyone on the same page about Recoil’s terminology first:

An atom is a piece of state stored by Recoil. In Kitemaker, we have items for things like work items, labels, users, comments and more
A selector is a derived piece of state. It fetches data from atoms (or other selectors), and transforms them in some way. An example of a selector in Kitemaker would be “fetch all of the comments for this particular work item”

Write fine-grained selectors

In order to make your UI perform well with Recoil, it’s import to understand what actually causes renders. Fortunately, it’s pretty simple - if a selector returns a simple value (string, number, boolean), it triggers a re-render whenever that value changes. If a selector returns an object or an array, Recoil re-renders if the returned value is not referentially equal (i.e. ===) to the previous value.

What this means in practice is that you want to write selectors that are quite fine-grained. Stick to simple values, avoiding arrays and objects where possible.

Take for example the board view in Kitemaker:

Let’s say we want to render the title of a card. We could write a selector that returns the entire atom, but if we did this, every single time that atom changed, we’d cause a re-render of the component. We don’t care about all the properties of that atom, we just want the title, so we should write a selector like this:

const workItemTitleSelector = selectorFamily({
  key: 'WorkItemTitle',
  get:
    (workItemId: string | undefined | null) =>
      ({ get }) => {
        return get(workItems(workItemId)?.title;
    },
});

// only fetch what we need!
function WorkItemCard({ workItemId }: { workItemId: string }) {
  const workItemTitle = useRecoilValue(workItemTitleSelector(workItemId));
  return (
    <div>
      {workItemTitle}
    </div>
  );
}

Now we’re only fetching the title, so only changes to the title will cause a re-render. Win!

Dealing with selectors that return objects and arrays

Before we said you should try to return as specific a value as possible from your selectors. But what if we have no choice but to return an object or an array from your selectors? For example, in Kitemaker, we often return a sorted list of IDs that we’ll then render in some sort of a list or board. Unfortunately, Recoil just doesn’t handle these cases very well out of the box.

We found a solution that allowed us to provide our own equality checks for selectors (or selector families). We have a custom selector family implementation that looks like this:

function equalSelectorFamily<T, P extends SerializableParam>(
  options: EqualSelectorFamilyOptions<T, P>
) {
  const inner = selectorFamily<T, P>({
    key: `${options.key}_inner`,
    get: options.get,
  });

  const priorValues: Map<P, T | undefined> = new Map();

  return selectorFamily<T, P>({
    ...options,
    key: options.key,
    get:
      (param: P) =>
      ({ get }) => {
        const latest = get(inner(param));
        const prior = priorValues.get(param);
        if (prior != null && options.equals(latest, prior)) {
          return prior;
        }
        priorValues.set(param, latest);
        return latest;
      },
  });
}

The idea is that if two consecutive executions of the selector result in the same value (as defined by the supplied equality operator), the selector just returns the old value, thus preserving referential equality and preventing a re-render.

To use it we do something like this:

const filteredWorkItemsByStatusSelector = equalSelectorFamily({
  key: 'FiltereWorkItemsByStatus',
  get:
    ({ spaceId, filterString }: { spaceId: string | undefined; filterString: string }) =>
    ({ get }) => {
      const itemIds: string[] = getFilteredWorkItemsSomehow(get, spaceId, filterString);
      return itemIds:
    },
  equals: _.isEqual,
});

Now if the two arrays returned are equal (as per lodash’s isEqual function in this case), we don’t re-render.

Hopefully Recoil will solve this for us in the future in a proper way 🤞There’s a promising but yet undocumented equality parameter on selector cachePolicy that should allow selectors to use value equality instead of referential equality. However, in our initial testing, this still failed to prevent re-renders from selectors that returned objects or arrays.

Leverage transactions

In Kitemaker, we have some complex operations. For example, adding a backlog to a space in Kitemaker creates a new board object, adds various columns to it, and more.

In order to minimize re-renders (and to prevent users from seeing weird intermediate states) we want to apply all of our changes in a single atomic operation. Fortunately, Recoil has an awesome solution for this - transactions.

In the case of Kitemaker, we always use transactions inside of a useRecoilCallback() function (though there is a useRecoilTransaction_UNSTABLE() hook as well:

interface BulkItemCreator {
  create(items: Array<{ id: string; name: string }>): void;
}

export function useBulkCreateExample() {
  return useRecoilCallback(
    ({ transact_UNSTABLE }) =>
      (callback: (creator: BulkItemCreator) => void) => {
        transact_UNSTABLE(({ set, get }) => {
          callback({
            create(items) {
              // all of these set() calls happen in one atomic operation
              for (const item of items) {
                set(itemAtomFamily(item.id), item);
              }
              // you can use get() here, but only for atoms, not selectors
            },
          });
        });
      }
  );
}

// Simple example of using a transaction
function SomeComponent() {
  const bulkCreate = useBulkCreateExample();
  const handleClick = React.useCallback(() => {
    bulkCreate((creator) => {
      creator.create([
        { id: '123', name: 'Bob' },
        { id: '234', name: 'Susan' },
      ]);
    });
  }, []);

  return <button onClick={handleClick}>Create</button>;
}

These transactions are considered an unstable feature (thus the naming) but in our experience they work great. There are some gotchas however - the main one being that you may not access any selectors inside of a transaction. This is because selectors are not updated as the individual operations of a transaction are applied to the recoil state, so the data in the selectors would be stale. This hasn’t been a big problem for us as the data fetching needs of this transaction code tends to be very simple. Also there’s a performance overhead with creating the snapshot inside of a transaction, so keep this in mind.

Watch your atom count

As the number of atoms in Recoil increases, there’s some performance impact. There is a lot of bookkeeping for dependency checking, etc. inside of Recoil that does not scale linearly. Additionally, whenever you need to grab a snapshot, it gets slower based on how large your set of atoms is. As a result, you need to think a bit about how you bucket your data into atoms. Does every single object need to be a separate atom? Or maybe if you always access a list of items together as a unit, you can shove the entire list into an atom. Then you can still write nice specific selectors on top (possibly using the equality tricks above) to fetch just the data you need to minimize re-renders.

Of course selectors are not free and in some cases changes to data may cause selectors to be frequently reevaluated. That means it’s a bit of a balancing act between specific selectors and managing your atom count. A rule of thumb is to group up atoms that are generally rendered together and which change relatively infrequently.

Wrapping it up

To summarize, using Recoil we were able to eliminate unnecessary renders in React. Instead of top-down, full screen re-renders with a bunch of memoization, we now have a very minimal number of re-renders whenever our data changes. However, Recoil isn’t magic. It really pays to take the time to understand what things causes re-renders and other performance issues so you can avoid those landmines in your implementation.

Want to learn more about Recoil or how we built Kitemaker? Always happy to chat on Twitter or email.