Forem: Ryosuke Iwanaga

Using GitHub Releases for patching Monorepo npm package

Ryosuke Iwanaga — Thu, 29 Jul 2021 12:40:12 +0000

When you find a bug in a npm package, you'll send a Pull Request (PR) to the original package. It would be great if your PR are merged quickly and released in the new version in npm registry. However, sometime it takes time but you want to use your bug fix right away. How can we do that?

It is simple if the package is the root of GitHub repository i.e. not Monorepo because package.json supports GitHub URL in the dependencies. Since you have already sumitted the PR, you should have your own fork with the bug fix. Therefore, you can simply specify your branch in your package.json and npm command handles the rest including running prepare.

Things are not so easy if the package is managed by a repository having multiple packages in it a.k.a. Monorepo. Unfortunately, there is no universal support by package.json because each monorepo is managed differently. Therefore, you need to build the package on your forked branch and use it from your application.

There are a bunch of solutions here, but I'd like to introduce a simple solution using GitHub Releases. In this solution, you just need a few manual steps:

Build the monorepo e.g. lerna run build
Create .tgz package for your patched version e.g. cd packages/package/dis && npm pack
Publish GitHub Releases i.e. gh release create ... package.tgz
Use the artifact URL in your application's dependencies

You need to run 1-3 whenever you update the branch, and probably need to reinstall packages in your application. You can automate these steps by GitHub Actions but you need to add action.yaml to your branch which is unrelated to your PR. Thus, I think this solution will work better with a repository getting less frequent update in upstream.

Steps

Let me give you a concrete example I did. I needed this fix in my application right away. The repository is monorepo and managed by lerna. I've already checked the forked repository out and had a branch named fix-classname on my laptop.

Build the monorepo

Simply built the monorepo:

➜  use-twind-with git:(fix-classname) npx lerna run build

This created dist directory under the package and it contains JavaScript files transpiled from TypeScript:

➜  use-twind-with git:(fix-classname) cd packages/preact
➜  preact git:(fix-classname) ls ./dist
CHANGELOG.md         package.json         preact.d.ts          preact.esnext.js.map preact.mjs
LICENSE              preact.cjs           preact.d.ts.map      preact.js            preact.umd.js
README.md            preact.cjs.map       preact.esnext.js     preact.js.map        preact.umd.js.map

Create `.tgz` package

I just needed dist directory only, so I ran npm pack there:

➜  preact git:(fix-classname) cd dist
➜  dist git:(fix-classname) npm pack
npm notice
npm notice 📦  @twind/preact@1.0.5
npm notice === Tarball Contents ===
npm notice 1.4kB CHANGELOG.md
npm notice 1.1kB LICENSE
npm notice 2.5kB README.md
npm notice 1.3kB package.json
npm notice 3.9kB preact.cjs
npm notice 3.7kB preact.cjs.map
npm notice 751B  preact.d.ts
npm notice 92B   preact.d.ts.map
npm notice 1.6kB preact.esnext.js
npm notice 3.4kB preact.esnext.js.map
npm notice 1.7kB preact.js
npm notice 3.4kB preact.js.map
npm notice 108B  preact.mjs
npm notice 1.5kB preact.umd.js
npm notice 3.3kB preact.umd.js.map
npm notice === Tarball Details ===
npm notice name:          @twind/preact
npm notice version:       1.0.5
npm notice filename:      @twind/preact-1.0.5.tgz
npm notice package size:  6.9 kB
npm notice unpacked size: 29.8 kB
npm notice shasum:        b258372501fb0c0f392274e237f230575c56342f
npm notice integrity:     sha512-y+zDKMLhQNSUu[...]UpgzvPXvPto7w==
npm notice total files:   15
npm notice
twind-preact-1.0.5.tgz

Now, I got twind-preaxt-1.0.5.tgz file.

Note: If you run npm pack in the package root, the .tgz file points TypeScript file as main and won't work with npm install.

Publish GitHub Releases

To release, I needed a Git tag. I used a convention like <original-tag>-<branch-name>. In this case, original tag was @twind/preact@1.0.5 and the branch name was fix-classname. I needed to create and push the tag first. Then, published the tag with the packed .tgz file:

➜  dist git:(fix-classname) git tag @twind/preact@1.0.5-fix-classname
➜  dist git:(fix-classname) git push --tags
➜  dist git:(fix-classname) gh release create \
    --target fix-classname \
    --title @twind/preact@1.0.5-fix-classname \
    --notes '' \
    @twind/preact@1.0.5-fix-classname twind-preact-1.0.5.tgz

This is the actual release page: https://github.com/riywo/use-twind-with/releases/tag/%40twind%2Fpreact%401.0.5-fix-classname

Note: gh command can be installed by https://github.com/cli/cli#installation

Use the artifact URL

Now, I got the .tgz public URL https://github.com/riywo/use-twind-with/releases/download/%40twind%2Fpreact%401.0.5-fix-classname/twind-preact-1.0.5.tgz. The last step was simply adding this to my application's package.json:

  "dependencies": {
    "@twind/preact": "https://github.com/riywo/use-twind-with/releases/download/%40twind%2Fpreact%401.0.5-fix-classname/twind-preact-1.0.5.tgz",
    ...
  },

Then, ran npm install. That's it! I compared the files under node_modules/@twind/preact and they were at the patched version:

➜  app git:(main) ✗ ls node_modules/@twind/preact
CHANGELOG.md         package.json         preact.d.ts          preact.esnext.js.map preact.mjs
LICENSE              preact.cjs           preact.d.ts.map      preact.js            preact.umd.js
README.md            preact.cjs.map       preact.esnext.js     preact.js.map        preact.umd.js.map

Alternatives

You can do the similar thing without the public artifact, for example with Git Submodule/Subtree + custom npm scripts. This would be great because you don't have to manage GitHub Releases artifacts. However, the build process of your application might become complicated and take more time.

Conclusion

I explained my solution to use a forked branch of monorepo packages via GitHub Releases. It requires a few manual steps but is relatively simple and easy to maintain.

How to refresh next-i18next content update automatically?

Ryosuke Iwanaga — Sun, 13 Jun 2021 08:51:04 +0000

tl;dr

If you want to refresh your Next.js app using next-i18next automatically, you can do it like below:

import { GetStaticProps } from "next";
import { i18n } from "next-i18next";
import { serverSideTranslations } from "next-i18next/serverSideTranslations";
import { useRouter } from "next/router";
import { useEffect } from "react";
import Home from "../components/home";

export const getStaticProps: GetStaticProps = async ({ locale }) => {
  if (process.env.NODE_ENV === "development") {
    await i18n?.reloadResources();
  }
  return {
    props: {
      ...(await serverSideTranslations(locale!, ["common"])),
    },
  };
};

const Index = () => {
  if (process.env.NODE_ENV === "development") {
    const router = useRouter();
    useEffect(() => {
      const timer = setInterval(() => {
        router.replace(router.asPath, undefined, {
          scroll: false,
        });
      }, 5000);
      return () => clearTimeout(timer);
    });
  }
  return <Home />;
};

export default Index;

What was the issue?

When I started using next-i18next, I realized next dev server only loaded the translation files once when initialized and never updated even if I reloaded the page on my browser since server side doesn't change. Whenever I updated the translation, I needed to restart next dev server, which was bad developer experience.

Alternative solutions

That is a known limitation and there are multiple GitHub issues like this. I was able to find two approaches:

Restart next dev automatically whenever the content is updated.
Poll API endpoint to monitor the content update and refresh the content.

The option 1 is simple and nodemon can easily achieve the goal. However, this is not "Fast Refresh" and takes a while.

The option 2 seems better because next dev server keeps running, but too complicated to implement internal API. It can be done without API like next-remote-watch which monitors files and calls Next.js's internal method to reload the page. I tried it but it still requires implementation of content refresh by calling i18n.reloadResources() anyway. Also, page refresh is not "Fast Refresh" neither.

Solution

Then, I realized this can be done much simpler. First of all, it anyway requires polling from client side because there is no public method to execute "Fast Refresh" from Next.js server side. Using internal method like next-remote-watch does is not sustainable. Therefore, client side polling is the best way.

However, setting up an API (i.e. /api/something) for such a simple polling seems overkill. I thought it's probably enough by just re-rendering the page. With this approach, unless the virtual DOM of React has been updated, nothing happens on client side (I think).

Now, how I can tell the translation files' change to the client? Next.js has a good mechanism to provide props to page i.e. GetStaticProps which is already used by next-i18next installation. I found a great solution to trigger this from client side.

In addition, I found that it can call i18n.reloadResources() there because i18n instance is stored in a global value. I lazily implemented it with reloading i18n at every request because my project doesn't have large translation files. This can eliminate file watcher logic at all.

Conclustion

Now, by adding a simple SetInterval() to refresh the page every 5 seconds on client side and reloading i18n on every GetStaticProps call, my Next.js pages are always in sync within 5 seconds. This is Next.js/React refresh, not browser refresh nor server restart, thus it's fast enough.

Let me know if you have a better solution or if you find a drawback of this solution :)

Notes

When your URL has hash (#foo), router.replace() always scrolls up to the anchor and doesn't reload preps from the server. This is a known issue and there is a discussion on GitHub repository: https://github.com/vercel/next.js/discussions/13804