Forem: Max Belsky

Dockerizing a Workspaced Node.js Application

Max Belsky — Wed, 11 Nov 2020 18:32:28 +0000

Re-usage of build cache is one of the most important things in Docker images creating.

To efficiently dockerize an app you need to split source code copying and dependencies installation in a few steps:

Copy dependencies files.
Install dependencies.
Copy source code.

For a node.js application these steps look like:

COPY package.json yarn.lock ./

RUN yarn install

COPY . .

However, this solution does not work with yarn workspaced application because the root package.json and yarn.lock are not enough to install whole project dependencies.

When I faced this task the first time I thought: what if I find all nested package.json files and copy them to a src directory:

COPY src/**/package.json src/

src/**/package.json pattern matches all package.json's that I need. But COPY works as not I expected. And instead of the expected directories structure I've got a single file under the src.

# The original project's tree
app
├── package.json
├── src
│   ├── backend
│   │   ├── backend.js
│   │   └── package.json
│   ├── notifier
│   │   ├── notifier.js
│   │   └── package.json
│   └── scraper
│       ├── package.json
│       └── scraper.js
└── yarn.lock

# The expected tree
app
├── package.json
├── src
│   ├── backend
│   │   └── package.json
│   ├── notifier
│   │   └── package.json
│   └── scraper
│       └── package.json
└── yarn.lock

# The result tree
app
├── package.json
├── src
│   └── package.json
└── yarn.lock

For a second I thought I could replace the single pattern line with a COPY operation for every workspace. But I wanted to have a scalable solution, a solution without duplication.

Shell solution

I've googled some alternative solutions. Commonly they suggest wrapping docker build with a script that creates a tmp folder, build the expected package.json's tree there and COPY the folder in the image.

And the "shell solution" is much better than the previous "copy-paste" solution. But it did not make me feel pleased.

Multi-stage builds solution

At some point, I thought of multi-stage builds. I used it in another project to build a tiny production image. "What if I will prepare the tree on a first stage and copy it on a second stage?"

In addition to the root package.json and yarn.lock files I copied the src directory and removed all not package.json files:

COPY package.json yarn.lock ./
COPY src src

# Remove not "package.json" files
RUN find src \! -name "package.json" \
  -mindepth 2 \
  -maxdepth 2 \
  -print \
  | xargs rm -rf

On a second stage I copied the tree and installed dependencies:

COPY --from=0 /app .

RUN yarn install --frozen-lockfile --production=true

Under the hood yarn workspaces use symlinks. So it's important to create them after copying src directory:

COPY . .

# Restore workspaces symlinks
RUN yarn install --frozen-lockfile --production=true

The final solution Dockerfile

FROM node:14.15.0-alpine3.10

WORKDIR /app
COPY package.json yarn.lock ./
COPY src src

# Remove not "package.json" files
RUN find src \! -name "package.json" -mindepth 2 -maxdepth 2 -print | xargs rm -rf

FROM node:14.15.0-alpine3.10

ENV NODE_ENV production

WORKDIR /app
COPY --from=0 /app .

RUN yarn install --frozen-lockfile --production=true

COPY . .

# Restore workspaces symlinks
RUN yarn install --frozen-lockfile --production=true

CMD ["yarn", "start"]

React-Redux apps on Rx steroids

Max Belsky — Sun, 05 Jul 2020 19:47:18 +0000

Since 2014 almost every week I've heard something about Rx. I've read many Rx docs, articles, and code examples. But I still had no idea why I should move to reactive programming paradigm if I can successfully solve my tasks with element.addEventListener(…).

In 2020 I found an awesome way to make complex react-redux components dumb and testable with Rx. And I'd like to share it with you in this article.

This article doesn't include Rx basics. If need you may find it here.

To see the Rx power let's start with an example.

Imagine there is a website that lists famous quotes. This website's users may do the following things:

read quotes
mark a quote as favorite

And this website may look like on the image below.

As you may see this webpage highlight favorite quotes for signed-in users only. So this website also has a sign in popup.

So now we imagine a component connected to redux that has quotes list, user data, and favorite quotes ids. And it makes a request to get favorite quotes list when a user is signed in.

class QuotesComponent extends React.Component {
  componentDidUpdate(prevProps) {
    const {user} = this.props

    if (undefiend === prevProps.user && undefined !== user) {
      dispatch(loadFavorites(user.id))
    }
  }

  // other methods
}

With current realization, this component knows not only about things to render. This component also contains some logic to make requests at the right moment.

And Rx comes here.

redux-observable is an awesome library that allows you to work with redux actions and state like with a stream. The core primitive to do that is Epic.

To make the QuotesComponent simpler we will do the following steps:

replace the componentDidUpdate implementation with componentDidMount
create an epic to load favorite quotes ids for a user

With redux-observable we don't need to track the user prop more. So we may just enqueue favorites loading in componentDidMount:

class QuotesComponent extends React.Component {
  componentDidMount() {
    dispatch(enqueueLoadFavorites())
  }

  // other methods
}

Why we don't need to track the user prop? Because we will create an epic that will do that for us.

const enqueueLoadFavoritesEpic = (action$, state$) => action$.pipe(
  filter(action => action.type === 'enqueueLoadFavorites'),
  switchMap(() => {
    const user = selectUser(state$.value)

    if (undefined === user) {
      // wait for signing in before loading favorites

      return state$.pipe(
        map(state => selectUser(state)),
        filter(user => user !== undefined)
        take(1),
        switchMap(() => {
          return of(loadFavorites(user.id))
        }),
      )
    }

    return of(loadFavorites(user.id))
  }),
)

const loadFavoritesEpic = () => {
  // do some async work, requests, get data and dispatch result action
}

The enqueueLoadFavoritesEpic doing the following steps:

wait for enqueue action
check for signed-in user

dispatch load action if a user is signed in
create a store observer that waits for a signed-in user to dispatch load action

Well as you see redux-observable is a great tool to handle pending actions depends on the store. redux-observable makes your components dumb and gets a way to test pending actions with unit tests.

useHover hooked a fool. The fool was me

Max Belsky — Sun, 23 Feb 2020 19:37:05 +0000

TLDR: useHover may trigger unnecessary re-renders: demo, solution

This weekend I found The Guardian's blog post about their website migration to React. At the end of the post, they mention DCR. It's a frontend rendering framework for theguardian.com and it's available on Github. I was so interested in how it is designed inside so I've started my research.

Performance is one of things that I was interested in. So I've tried to find any usages of React.memo, PureComponent or shouldComponentUpdate. And I was so surprised when I found nothing.

I've gone back to their site and started profiling it. And I didn't found any unnecessary re-renders because it just renders article. The data never changes, the page doesn't have any tricky handlers. So any optimizations will be just extra costs here. But then I've found this.

It's a simple component to render most viewed articles aside. And it re-renders on a hover event because its content styled programmatically. So it looks logical: you hover X and it re-renders because now it's hovered. After you hover Y and X with Y re-render because the state of both components changed. But what if I show you this?

Blue and orange boxes is not a part of the component. React profiler shows it when a component re-renders

Mouse movement inside the component still trigger re-renders, but I had no idea why. I've wrapped internal components with React.memo, callbacks with useCallback and other things that usually help. But this component was still re-render. React profiler showed that props still change. Then I thought maybe useHover has some issues. But it didn't.

So I've written a plain html+css+js demo and shared with some friends to whine and complain why mouse movement inside a hovered element triggers mouseout and mouseover events. And they helped me. I have forgotten about a core mechanic of javascript events. The mechanic is event bubbling and capture. The demo extended with extra logging currentTarget shows it.

Unfortunately e.stopPropagation does not solve current issue, so I've implemented a throttling mechanism for that hook with setTimeout.

The original version of the demo is available here

As before, mouse event handlers trigger immediately (you may track it by console logs), but setState (prefixed with Deferred) will be called on the next event loop tick only. So if we have two or more serial events, the hook won't call setState on every event, it will work only once with the latest true/false value.

It was a good reminder to don't forget javascript basics because React is just a library that builded on them.

Send message as a Telegram bot. What may go wrong?

Max Belsky — Tue, 17 Dec 2019 13:45:11 +0000

Last month I've worked on @hltvFeatured – it's a Telegram bot to get notifications about upcoming Counter-Strike: Global Offensive matches featured by HLTV.org.

After a few weeks in production I've got an alert that the bot fails on sending notifications to subscribers. I had no access to my PC, so it was so nervous. I didn't know what may go wrong.

When I back to home first of all I opened IDE and started debugging. There were no issues with database, the app's code or network. But the Telegram API returns error 400: Bad Request: can't parse entities. I've started analyze what wrong with the messages and why it didn't fail before.

Telegram API allows to format messages in two styles: Markdown and HTML. I've selected Markdown as less verbose and wrote a small function to convert match data entity to a Markdown string:

function convertToMessage({ event, href, stars, title, unixTimestamp }) {
  const when = new Date(unixTimestamp).toUTCString()
  const date = formatUTCString(when).replace(/\s/g, NBSP)

  return `
[${title.replace(/\s/g, NBSP)}](${href})
Rating: ${'☆'.repeat(stars) || '–'}
_${date} @ ${event}_
`.trim()
}

That day matches had an interesting event name: cs_summit 5 and I immediately noticed it. As you can see the convert function makes date and event italic: _${date} @ ${event}_. So the message contained three underscores. I didn't expect that Telegram API can't parse a message like that, so I've started search for a dependency to escape symbols like _, * and [ in matches data before inject it in message template.

You have no idea how I was surprised when Telegram API answered with the same error for a message with escaped symbols. This time I went to google it. The solution was a suggestion to use HTML markup and escape symbols. I've updated my function and... it works!

function convertToMessage({ event, href, stars, title, unixTimestamp }) {
  const when = new Date(unixTimestamp).toUTCString()
  const date = formatUTCString(when).replace(/\s/g, NBSP)

  return `
<a href="${href}">${escapeHtml(title).replace(/\s/g, NBSP)}</a>
Rating: ${'☆'.repeat(stars) || '–'}
<i>${date} @ ${escapeHtml(event)}</i>
`.trim()
}

I can't imagine a case when Markdown is a good choice to markup messages delivered by a bot. And if you can, please, share in comments :)

Pair husky with Git LFS in your JavaScript project

Max Belsky — Mon, 14 Oct 2019 14:36:18 +0000

As you may know Git has a way to fire off custom scripts when certain important actions occur. This is hooks.

I have worked on a project which uses husky as tool to run formatters and linters on pre-comimit hooks. One day we have decided to cover our forms with screenshot tests to be sure that our changes do not break UI. We have needed to store binary files in our repo, so we choose Git LFS to make git operations like git pull and git clone faster. If you are not familiar with Git LFS check this awesome Atlassian’s guide.

This seemed like a good and simple solution. I’ve started work on integration Git LFS in our project. Its Getting Started looks so easy: just download and run three commands in your terminal. I did fail on this first:

$ git lfs install
Hook already exists: pre-push
…
To resolve this, either:
  1: run `git lfs update --manual` for instructions on how to merge hooks.
  2: run `git lfs update --force` to overwrite your hook.

Both suggested solutions could fix this issue, but not really. The second overwrites a few husky’s hooks:

post-checkout
post-commit
post-merge
pre-push

So husky won’t run scripts if you configured any of listed above.

The first is more compatible with husky. However there is still an issue: anyone who will clone that repo should merge hooks manually. That is why I come with one more solution.

Installation

$ rm -rf .git/hooks
$ git lfs install
$ mv .git/hooks ./lfs-hooks
# Uninstall this dependency to restore husky hooks with `npm install`
$ rm -rf node_modules/husky
$ npm install

At this moment husky hooks will be installed in .git/hooks and Git LFS hooks in ./lfs-hooks. Now you need to configure Git LFS hooks running with husky:

  "husky": {
    "hooks": {
      "post-checkout": "echo $HUSKY_GIT_STDIN | lfs-hooks/post-checkout $HUSKY_GIT_PARAMS",
      "post-commit": "echo $HUSKY_GIT_STDIN | lfs-hooks/post-commit $HUSKY_GIT_PARAMS",
      "post-merge": "echo $HUSKY_GIT_STDIN | lfs-hooks/post-merge $HUSKY_GIT_PARAMS",
      "pre-push": "echo $HUSKY_GIT_STDIN | lfs-hooks/pre-push $HUSKY_GIT_PARAMS"
    }
  },

Thanks @mattrabe for this snippet

Finish Git LFS installation with git lfs track <binary files> to set up .gitattributes.

Now save, commit and push. Your collaborators will not need anything to do to start with husky and Git LFS.

Note: When someone clones your repo first of all she need to remove .git/hooks directory because Git LFS creates some hooks by default and husky does not have an overwrite option yet.

$ rm -rf .git/hooks
$ npm install

That is all. Happy coding!

From bash to zsh. My little adventure

Max Belsky — Tue, 01 Oct 2019 11:39:24 +0000

A few months ago Apple announced the upcoming version of its desktop OS macOS Catalina, which is going to replace the default command line shell bash with zsh for all newly created accounts. This was the start of my “adventure”. It wasn’t so long, however it was full of pitfalls. So I came here in the hope to save a few hours for someone.

First of all lets change a command line shell for your user. To do that we need locate zsh and set as default shell:

# get path to `zsh` on your machine and use it with `chsh` command
$ which zsh
/bin/zsh
$ chsh -s /bin/zsh $USER

Restart Terminal.app and make sure that your changes applied:

$ ps -p $$ -oargs=
-zsh

Now we can make some customizations. Every day I operate with git from terminal so there are two important things for me: git-prompt and git-autocomplete. Zsh provides git-autocomplete out of the box. So there is some commands to set up prompt:

$ cd ~/
$ curl -o .git-prompt.sh https://raw.githubusercontent.com/git/git/master/contrib/completion/git-prompt.sh
$ touch ~/.zshrc

We almost done 😁 Open ~/.zshrc in your favourite text editor and put there:

#!/bin/zsh

# Set up git-prompt
source ~/.git-prompt.sh
setopt PROMPT_SUBST; PS1='%~ $(__git_ps1 "(%s)")\$ '

autoload -U compinit; compinit

# If you have aliases or $PATH extensions  in your .bash_profile paste it below

Restart your terminal one more time and enjoy your new shell.