Forem: Denis

Reduce memory usage of NodeJS apps inside Docker

Denis — Thu, 27 Jul 2023 10:59:35 +0000

Before diving in, I must admit that the title is a bit vague. Let me describe the issue first, cause the solution below is not a silver bullet and it's suitable for specific runtime environments.

In our company we use Google Cloud Run to deploy web applications, and every app is built into a docker image. For now we use the default memory limit by Cloud Run which is 256 MB per container. Recently we started to notice that the part of applications go beyond this limit, causing a container to restart and in some cases even resulting to downtime of a service.

These applications run as Node processes (NextJS server and SvelteKit server), and docker entrypoint is just a pnpm run start, that executes the corresponding command under the hood.

Troubleshooting

Unfortunately, it's impossible to get inside a running instance in Cloud Run, so the only way to debug a problem is via logs. Web interface doesn't provide you any information about running processes, just a chart with memory utilization from your containers:

On this chart, we can see that sometimes it's just not enough memory for a service.

My initial idea was to periodically run top inside a container and log the output. Then try to see how different conditions like huge traffic affect the memory. There's a tool that helps you run multiple processes inside a docker container called multi-start. I slightly modified its code to execute top -bcn1 -o %MEM -w256 by some interval and log the result to stdout. But just after installing it to the image and deploying it to Google Cloud I noticed something strange:

1) Total memory consumption according to top is 160 MB. But the chart above says it takes up to 90% of memory. Well, memory calculation can be interesting sometimes
2) For some reason, the container thinks it has 1 Gb RAM allocated, yet the limit in Cloud Run is set to 256 MB
3) pnpm run start consumes almost the same amount of memory as NextJS server! Yet it's just a runner for an actual command

Solution

In fact, the only thing I had to do is to replace pnpm run start with a corresponding script from package.json, so in case of NextJS apps it was node ./node_modules/next/dist/bin/next start and node ./build for SvelteKit (you might have a different setup).

The results are prominent:

It immediately saved almost 60 MB or memory, which is a lot in such circumstances. See how the changes are reflected in the dashboard:

Conclusion

This indeed may sound like an edge case, but if you run your node application in an environment with such a limited resources, just try to invoke the scripts directly, without wrapping them into pnpm command.

It also sounds very strange to me why pnpm is this eager. If anyone has some ideas, please share them.

Hope it helps 👋

Apity: A typed HTTP client for Svelte(Kit)

Denis — Wed, 12 Apr 2023 15:55:06 +0000

Most frontend projects need to make HTTP requests, but there's no convention how to handle them. We have a fetch function, but it would be inconvenient to call it directly each time, so often developers create their own API wrappers to call logical functions instead, for example:

const products = await API.fetchProducts({ userId })
// or
const response = await listProducts({ userId })

I've seen a lot of code like this. Sometimes backend is implemented with the same technology as frontend and a framework provides helpers for network communication, for example SvelteKit has form actions. But in certain cases backend is a separate instance and you operate with its API, so such functions are unavoidable.

The main question with this approach is how to manage request parameters and data in responses. If you're a TypeScript user then you know the benefits of proper typing. TS warns you when you're trying to access wrong properties of objects and a lot of mistakes can be caught even before running the code. And of course having auto completion in your IDE makes development experience much better. So by returning a raw response.json() you lose a lot of benefits.

However, there's a couple of issues with this approach:

1) Writing the types by hand is tedious and error prone
2) Backend changes over time, it can be a new property added to a response, or a new parameter in request body - so you have to keep both codebases in sync
3) Sometimes the types are implemented by backend team already, so duplicating them on frontend sounds like a double work

Combining all these concerns together and trying to find a proper solution resulted in Apity - a Svelte/SvelteKit library for making typed HTTP requests. In this article I'll try to show its bright sides and how to use it.

Installation and configuration

A pre-requisite before taking Apity into use is having an OpenAPI spec of a server. It should be either stored as a local file, or accessible by an URL. If you're not familiar with OpenAPI - it's a JSON or YAML file that lists server routes, expected parameters and responses. A lot of backend technologies support schema generation, for example FastAPI has it out of the box.

In this post we'll work with Petstore schema, so the first steps will be to install the library and generate types from OpenAPI schema (assuming you already have a SvelteKit application).

npm install @cocreators-ee/apity

npx openapi-typescript https://petstore3.swagger.io/api/v3/openapi.json --output src/petstore.ts

Now we have all the necessary information about the backend in src/petstore.ts - it contains all the routes and schemas for requests and responses. Using these objects, we can construct and API wrapper for the routes almost automatically:

import { Apity } from '@cocreators-ee/apity'
import type { paths } from 'src/petstore'

const apity = Apity.for<paths>()

apity.configure({
  baseUrl: 'https://petstore.swagger.io/v2',
})

export const findPetsByStatus = apity
  .path('/pet/findByStatus')
  .method('get')
  .create()
export const addPet = apity.path('/pet').method('post').create()

Yes, you have to define each API route by hand, but this was an intentional decision. You will operate with these functions in your code, so it's up to you how to name them. Assigning the names automatically from the spec could result in worse naming, or you might want to separate several parts into different wrappers.

Don't panic however, now TS knows your backend structure, so defining the routes will be rather quick, and it will warn you if you for example try to create a GET method for a POST route:

Usage

Now when you have your desired routes defined, just import the functions and call them:

const request = findPetsByStatus({ status: 'available' })
const response = await request.result

Response object has a typed data property that's available when request is finished successfully. This can be checked with ok property:

This is crucial to check if request was successful before accessing the data, because failed request is also a valid case. This semantic forces you to handle bad cases as well, for example showing an error toast to the user.

Request parameters are also automatically suggested and validated for correctness:

Now that you've got a first impression of the library, let's dive into its features and see how it fits into different scenarios.

Server Side Rendering (SSR)

You can use the library in SvelteKit's load functions, to request data on server:

import { findPetByStatus } from 'src/api.ts'

export async function load({ fetch }) {
  const request = findPetByStatus({ status: 'sold' }, fetch)
  const resp = await request.result
  if (resp.ok) {
    return { pets: resp.data, error: undefined }
  } else {
    return { pets: [], error: 'Failed to load pets' }
  }
}

Note that in this case you need to explicitly provide server's fetch implementation.

Await syntax

If you like to await promises in Svelte templates, then you can use ready store from the request object:

<script lang="ts">
  import { findPetByStatus } from 'src/api.ts'
  const request = findPetByStatus({ status: 'sold' })
  const petsReady = request.ready
</script>

<div>
  {#await $petsReady}
    <p>Loading..</p>
  {:then resp}
    {#if resp.ok}
      {#each resp.data as pet}
        <p>{pet.name}</p>
      {/each}
    {:else}
      <p>Error while loading pets</p>
    {/if}
  {/await}

  <button on:click={() => {request.reload()}}>
    Reload pets
  </button>
</div>

Autosuggestion works there as well!

Retrying a request

The curious reader has noticed that API call doesn't return a response directly, but a request object. It's designed to satisfy different needs:

subscribe to a ready store and wait for updates
await for response in template
await the result promise right away (for example in server code)
trigger a reload of the same request with the same parameters

For example you might want to periodically refresh the page content, and the parameters of such request stay the same:

import type { components } from 'src/petstore'

let pets: components['schemas']['Pet'][] = []
const request = findPetsByStatus({ status: 'available' })

request.resp.subscribe((resp) => {
    if (resp && resp.ok) {
        pets = resp.data
    }
})

setInterval(() => {
    request.reload()
}, 5000)

This example also shows that you can access backend types programmatically by importing them:

Error handling

Errors can be expected and unexpected. First ones usually have an HTTP status code starting from 4, e.g 404 when a resource is not found on server, or 422 when your payload is incorrect. Sometimes you even have special handlers for them on frontend side. Often these requests have a JSON body in order to indicate what was wrong.
However under certain conditions a request can fail unexpectedly, for example when backend is down, or due to unstable network. These requests might not have a body, they usually throw an error on client side.

We thought that it's a good idea to provide a developer a single interface for handling both kinds of errors. Thus, each response has ok, status and error properties to debug the issue properly.

ok is set to false for every failed request
status is a HTTP status code and it's set by backend most of the times, but for exceptional cases like network errors it will be a negative value like -1
error contains a JSON body from backend if provided or set to undefined otherwise.

Please always care about error cases to build better services!

Bonus - pre-commit hooks

Now you have a mechanism to generate frontend types from OpenAPI spec, so backend becomes a single source of truth and you don't need to repeat a lot of code by hands. But there's still no answer on how to keep types in sync. One of the possible approaches is to use pre-commit hooks, like pre-commit or Husky.

Pre commit hooks are set of commands to execute every time you run git commit. If you have a monorepo, then you can configure hooks to export OpenAPI spec and run openapi-typescript consequently. If your spec is accessible over network, then just create an npm script with a URL as a parameter. Possible example for Petsore:

package.json:

{
  "devDependencies": {
    "openapi-typescript": "^6.2.0",
  },
  "scripts": {
    "make-api": "openapi-typescript https://petstore3.swagger.io/api/v3/openapi.json --output src/lib/openapi.ts"
  }
}

.pre-commit-config.yaml:

repos:
  - repo: local
    hooks:
      - id: make-api
        name: make-api
        entry: npm run make-api
        language: system
        pass_filenames: false

Conclusion

We hope that managing your API requests will be easier now thanks to Apity. We already use it in our SvelteKit projects and the experience is rather fun so far. Please try it out if you find this approach interesting enough, and your feedback is always appreciated.

Also I built a simple demo application using same Petstore spec, you can check it out at https://apity-demo.vercel.app.
Each page has a source code snippet for your convenience.

Thanks for reading!

Reducing Docker image size of a Nuxt SSR application

Denis — Mon, 17 May 2021 14:02:17 +0000

Recently I had to create a deployment of a NuxtJS application which is running in SSR mode. I have a separate backend that is already packed in Docker image, so it sounds pretty tempting to dockerize the frontend application as well and to run both using docker-compose. Basically, server side rendering implies that the frontend application becomes a server too, to some extent.

To demonstrate the concept, I will show you two Dockerfiles, one is straightforward, without any optimizations, and another is what goes into production.

First obvious idea is to get the smallest node-based image available. Of course, it's an Alpine one.

So let's consider this Dockerfile, assuming we don't care about a final size too much:

FROM node:15.11.0-alpine3.12 as frontend

WORKDIR /src

ADD frontend ./
RUN yarn install && yarn build

ENTRYPOINT ["npx", "nuxt", "start"]
EXPOSE 3000

Now check the size:

➜ docker images | grep demo-frontend
demo-frontend     latest     151ebafca257   1 minute ago   782MB

I couldn't bear the thought that a simple frontend application will take almost 800MB of disk space. It's not a surprise though, cause node_modules is enormous. We could of course use multi-stage builds and install only production dependencies for runtime, but it would not cost the effort:

➜ yarn install
➜ du -sh node_modules
386M    node_modules

➜ yarn install --production
➜ du -sh node_modules
276M node_modules

And now the trick. Let's check what's inside of a .nuxt folder, that is generated by nuxt build:

➜ yarn build
➜ du -sh .nuxt/dist/*
5.5M    .nuxt/dist/client
1.2M    .nuxt/dist/server

It looks pretty strange that client-side code takes more space than the server-side, isn't it? 🤔
Apparently, server-side code is relying on third-party libraries stored in the node modules. They are not bundled.

The good thing is that Nuxt offers a solution, a --standalone option that solves this issue. Let's try to rebuild and compare.

➜ yarn build --standalone
➜ du -sh .nuxt/dist/*
5.5M .nuxt/dist/client
 39M .nuxt/dist/server

Yep, something has changed for sure. Dependencies for a server runtime are now stored in .nuxt folder, so we don't need all the node_modules anymore.

And now the final insight: you don't need the entire nuxt package to run your code using nuxt start. There's a separate package that is optimized only for running bundles in SSR mode: nuxt-start. So the final step is to install this package in a runtime Docker image and skip the rest.

Let's have a look on the final Dockerfile:

FROM node:15.11.0-alpine3.12 as frontend-build

WORKDIR /src

ADD frontend/yarn.lock frontend/package.json ./
RUN yarn install

ADD frontend ./
RUN yarn build --standalone

FROM node:15.11.0-alpine3.12

ENV NUXT_VERSION=2.15.6

WORKDIR /app

RUN yarn add "nuxt-start@${NUXT_VERSION}"

COPY --from=frontend-build /src/.nuxt /app/.nuxt
COPY --from=frontend-build /src/nuxt.config.ts /app/
COPY --from=frontend-build /src/static /app/

ENTRYPOINT ["npx", "nuxt-start"]
EXPOSE 3000

In case you wonder what we've just done:

In build image (that is not used in production):

Install the dependencies from package.json
Build an application in a standalone mode, so .nuxt folder contains everything we need

In runtime image (that is running in production)

Install nuxt-start, a package that will run our app
Copy the .nuxt folder from the build image, as well as static folder and NuxtJS config
Run the app

Now, how much the final image weighs?

demo-frontend     latest     f41a130ae000   21 seconds ago   208MB

Yep, that's true 🙂 We've just saved 574 MB of a disk space, final image became 3.75 times thinner than initial!

Of course, it highly depends on the size of your dependencies, but I'm sure you got the idea. Please also keep in mind that it's a good idea to install nuxt-start with the same version as nuxt from your package.json.

TL;DR:

Get Alpine as a base image
Leverage multi stage builds
Bundle dependencies into server code
Run server using nuxt-start package

Happy deploying! 🚀

Edit on May, 21:

There was a suggestion in comments by @artalus to not use multi-stage builds, but to put all the logic into single RUN statement instead. In theory, it should result into even more space savings due to reducing the number of docker layers. Let's try it out!

FROM node:15.11.0-alpine3.12

ENV NUXT_VERSION=2.15.6

WORKDIR /app

ADD frontend ./
RUN : \
  && yarn install \
  && yarn build --standalone \
  && rm -rf node_modules \
  && rm package.json \
  && yarn add "nuxt-start@${NUXT_VERSION}" \
  && yarn cache clean \
  && :

ENTRYPOINT ["npx", "nuxt-start"]
EXPOSE 3000

🥁 ...

demo-frontend     latest     2e2ca36f6c2e   30 seconds ago   195MB

Awesome! 13MB may not sound that great, but now the total result is less than 200! Or officially 4x times thinner than initial version.

I intentionally haven't modified original post to show you the idea of applying optimizations step-by-step.

Please also note that using single RUN statement slows down your build to some point, cause yarn install step is not cached anymore. However, this is only relevant if you have caching enabled on your CI agent.

Cheers! 🎉

Deploying to GitHub Pages without extra dependencies

Denis — Thu, 18 Mar 2021 09:06:52 +0000

There are a lot of articles already describing how to deploy you static website to GitHub Pages from GitHub Actions. Almost all of them suggest to use 3rd party Actions available on marketplace.

Some of them are extremely popular, like this one.
Some less, but still doing their job as you expect from them.

And long story short, you can just use them, it's pretty convenient to delegate this logic to something and avoid maintenance effort. However, you need to understand that it's a security implication and you pass your secret token of the repository to 3rd party code. Which in theory can contain some sneaky bug or even be malformed. Yeah, they're open source and you can check the code and validate it yourself, but.. would you?

I still don't get why GitHub doesn't provide out-of-the-box functionality to connect 2 of their services together, and why you need to look for extra packages rather than use some built-in feature, like in Travis..

On the other hand, deploying to GitHub Actions is pretty straightforward: you just need to push desired files to some branch in your repo, and that's basically it. Why to complicate things then and use untrusted javascript code for simply invoking git commit && git push? That's a fair question.

One of awesome GitHub Action features is that you can access your secret token without any extra effort. So you basically don't have to deal with git authentication in any way.

Combining this together, here's a snippet of a pipeline you can use to deploy static files from your repo to GitHub Pages:

It's simple, customisable, and most importantly, doesn't leak your secrets to anywhere.

Hope it helps!