Forem: Sergey Labut

Google Tag Manager Loading Strategies in Next.js

Sergey Labut — Fri, 13 Feb 2026 09:51:06 +0000

Performance, Analytics Accuracy, and Risk-Aware Implementation

Google Tag Manager (GTM) can significantly affect Next.js apps. Choosing the right loading strategy requires balancing performance, tracking reliability, and implementation risk.

This guide covers seven strategies, with performance / analytics / complexity & risk explained for each, including common mistakes and implementation examples.

Why GTM Loading Matters in Next.js

Next.js applications differ from traditional websites in important ways:

Pages are server-rendered and then hydrated on the client
JavaScript execution during hydration affects interaction metrics (INP)
Client-side navigation does not reload the page
Page views must be tracked manually on route changes

Because of this, GTM loading strategy directly affects:

Core Web Vitals (especially INP and LCP)
Hydration speed and responsiveness
Analytics accuracy and attribution

1. Balanced Approach – After-Interactive Loading (Recommended Default)

Performance: Good

Analytics: High

Complexity / Risk: Low

What this means

GTM loads after the page becomes interactive using strategy="afterInteractive".
Hydration is not blocked, but GTM may execute while hydration is still finishing.

Common mistakes

Forgetting SPA page view tracking
Assuming afterInteractive alone guarantees good INP
Leaving GTM’s default page view trigger enabled and causing duplicates

import Script from "next/script";

const GTM_ID = process.env.NEXT_PUBLIC_GTM_ID!;

export function GTM() {
  return (
    <Script
      id="gtm"
      strategy="afterInteractive"
      dangerouslySetInnerHTML={{
        __html: `
          // Ensure dataLayer exists before GTM loads
          window.dataLayer = window.dataLayer || [];

          (function(w,d,s,l,i){
            w[l]=w[l]||[];
            w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});
            var f=d.getElementsByTagName(s)[0],
                j=d.createElement(s),
                dl=l!='dataLayer'?'&l='+l:'';
            j.async=true;
            j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;
            f.parentNode.insertBefore(j,f);
          })(window,document,'script','dataLayer','${GTM_ID}');
        `,
      }}
    />
  );
}

SPA page view tracking — required
Because Next.js uses client-side routing, page views must be sent manually on route changes.

Note: If you do this, disable GTM’s default Page View trigger to avoid duplicates.

// SPA page view tracking — Pages Router
import { useRouter } from "next/router";
import { useEffect } from "react";

export function usePageViews() {
  const router = useRouter();

  useEffect(() => {
    const handleRouteChange = (url: string) => {
      window.dataLayer?.push({
        event: "page_view",
        page_path: url,
        page_location: window.location.href,
      });
    };

    router.events.on("routeChangeComplete", handleRouteChange);
    return () =>
      router.events.off("routeChangeComplete", handleRouteChange);
  }, [router]);
}

// SPA page view tracking — App Router
"use client";

import { usePathname, useSearchParams } from "next/navigation";
import { useEffect } from "react";

export function PageViewTracker() {
  const pathname = usePathname();
  const searchParams = useSearchParams();

  useEffect(() => {
    const pagePath =
      pathname + (searchParams.toString() ? `?${searchParams.toString()}` : "");

    window.dataLayer?.push({
      event: "page_view",
      page_path: pagePath,
      page_location: window.location.href,
    });
  }, [pathname, searchParams]);

  return null;
}

2. Max Tracking – Early Loading (Before-Interactive)

Performance: Moderate / Risky
Analytics: Maximum
Complexity / Risk: Low

What this means

GTM loads before React hydration starts using beforeInteractive.

<Script strategy="beforeInteractive" src={`https://www.googletagmanager.com/gtm.js?id=${GTM_ID}`}/>

Common mistakes

INP regressions caused by long GTM tasks
Duplicate page views when default GTM page view triggers are combined with SPA tracking
Assuming “early load” only affects LCP (INP is usually the real casualty)
Letting marketing add heavy tags without performance review

When to use

Marketing-heavy pages
Conversion funnels where attribution accuracy is critical

3. Performance-First – Lazy Loading (After Page Fully Loads)

Performance: Very Good
Analytics: Medium
Complexity / Risk: Low

What this means

GTM loads only after the browser load event.

<Script id="gtm" strategy="lazyOnload" src={`https://www.googletagmanager.com/gtm.js?id=${GTM_ID}`}/>

Common mistakes

Assuming Lighthouse ignores GTM entirely
Missing page views from short sessions

When to use

Content-heavy sites
SEO-driven pages where analytics is helpful but not critical

4. Extreme Performance – Interaction-Based Loading with Event Buffering

Performance: Excellent
Analytics: Medium–High
Complexity / Risk: Medium

What this means

GTM loads only after user interaction. Important events are buffered until GTM is ready.

Common mistakes

Users who never interact are not tracked
Event buffer flushed too late or never
Overengineering simple setups

// Example buffering logic
// Initialize dataLayer early
window.dataLayer = window.dataLayer || [];

const bufferedEvents: any[] = [];

// Track events before GTM is ready
export function trackEvent(event: any) {
  if (!window.gtmReady) {
    bufferedEvents.push(event);
  } else {
    window.dataLayer.push(event);
  }
}

// Call this when GTM finishes loading
export function onGtmReady() {
  bufferedEvents.forEach(event => window.dataLayer.push(event));
  window.gtmReady = true;
}

5. Conditional Loading – Consent, Region, or Route-Based

Performance: Good
Analytics: Medium
Complexity / Risk: Medium–High

What this means

GTM is loaded only if certain conditions are met (for example, user consent).

{hasConsent && <GTM />}

Common mistakes

Inconsistent analytics across regions
Consent state not persisted correctly
Forgetting to update rules for new routes

When to use

GDPR-heavy environments
Multi-region or multi-brand sites

6. Server-Side GTM – High Reliability, High Complexity

Performance: Good
Analytics: Very High
Complexity / Risk: High

What this means

The browser sends events to your server, which forwards them to GTM and analytics tools.

Common mistakes

Poor client/server event parity
Added latency without batching
Underestimating maintenance cost

When to use

High-traffic applications
Long-term analytics infrastructure
The user's browser extensions are blocking analytics

7. Web Worker GTM (Partytown) – Maximum Frontend Performance

Performance: Excellent
Analytics: High
Complexity / Risk: High

What this means

GTM runs in a Web Worker using Partytown, keeping heavy scripts off the main thread.

Common mistakes

DOM-dependent tags silently failing
Missing forwarded APIs
Difficult debugging

import { Partytown } from "@builder.io/partytown/react";
import Script from "next/script";

<Partytown forward={["dataLayer.push"]} />

<Script
  type="text/partytown"
  src={`https://www.googletagmanager.com/gtm.js?id=${GTM_ID}`}
/>

or just

<Script src={`https://www.googletagmanager.com/gtm.js?id=${GTM_ID}`} strategy="worker" />

Final Recommendations

Default choice: After-Interactive loading + manual SPA page views
Maximum analytics reliability: Early loading or Server-Side GTM
Extreme performance: Interaction-based loading with buffering or Partytown
Regularly audit GTM tags — loading strategy alone won’t fix heavy scripts
Always validate SPA tracking when using the App Router

Strategy	Performance Impact	Tracking Reliability	Complexity	Recommended For
After Interactive (Recommended Default)	Good	High (manual SPA page views required)	Low	Most Next.js apps; balanced performance & analytics
Before Interactive (Early Load)	Moderate	Very High	Low	Marketing-heavy pages; critical attribution tracking
Lazy Onload (After Page Fully Loads)	Very Good	Medium	Low	Content-focused or SEO-driven pages; analytics not critical
Interaction-Based Load	Excellent	Low	Medium	Blogs or informational pages; extreme performance priority
Interaction with Event Buffering	Excellent	High	Medium-High	Performance-first sites needing reliable early tracking
Conditional Load	Good	Medium	Medium-High	GDPR-heavy, multi-region, or multi-brand setups
Server-Side GTM	Good	Very High	High	High-traffic apps; long-term analytics & reliability
Web Worker (Partytown)	Excellent	High	High	Performance-critical sites; heavy third-party scripts

Playwright testing on CI using Github Actions and Vercel

Sergey Labut — Wed, 07 Aug 2024 11:43:01 +0000

In this article, we will share our experience of testing on CI using Playwright and GitHub Actions.

Intro
Overview of GitHub Actions
Default example
Real world example
Summary

Intro

If you are new to Playwright, please check out the previous article. If you are new to the concept of CI/CD (Continuous Integration and Continuous Deployment), don't worry, this article doesn't require any special knowledge and introduces each concept step by step.

Overview of GitHub Actions

GitHub Actions is a CI/CD tool integrated directly into GitHub. It allows developers to automate their workflows for building, testing, and deploying code.

Key concepts:

workflow: A workflow is an automated process made up of one or more jobs. It's defined by a YAML file in the .github/workflows directory of your repository. Workflows can be triggered by various events, such as pushing code, creating pull requests, or on a schedule.
job: A job is a set of steps executed on the same runner. Jobs run in parallel by default, but can be configured to run sequentially if dependencies exist between them.
step: A step represents a single task to be performed, such as running a command or using an action.
action: Actions are custom applications for the GitHub Actions platform that can be used to perform complex tasks. Actions can be created by anyone and shared via GitHub Marketplace. Action can have inputs and outputs.
runner: Runners are the servers that run your workflows. GitHub provides hosted runners with commonly used software preinstalled, or you can use self-hosted runners.
event: A specific activity that triggers a workflow to run. Examples include push, pull_request, schedule, and workflow_dispatch.
artifact: Files generated during a workflow that can be retained and shared between jobs or used as a record of workflow execution. Artifacts are useful for persisting build outputs, test results, and other files for later analysis or download.

Default example

Now we can look at the example from the official documentation and understand what's going on there.

# Workflow name
name: Playwright Tests

# Trigger the workflow on push and pull request events to the main or master branches
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]

jobs:
# Job name
  test:
    # Set a timeout of 60 minutes for the job
    timeout-minutes: 60
    # Use the latest Ubuntu runner
    runs-on: ubuntu-latest

    steps:
      # Step to check out the repository
      - uses: actions/checkout@v4

      # Step to set up Node.js environment
      - uses: actions/setup-node@v4
      # Inputs for the action
        with:
          node-version: 18  # Specify Node.js version 18

      # Step to install project dependencies using npm
      - name: Install dependencies
        run: npm ci

      # Step to install Playwright browsers along with necessary dependencies
      - name: Install Playwright Browsers
        run: npx playwright install --with-deps

      # Step to run Playwright tests
      - name: Run Playwright tests
        run: npx playwright test

      # Step to upload Playwright test report as an artifact
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}  # Upload only if the job is not cancelled
      # Inputs for the action
        with:
          name: playwright-report  # Name of the artifact
          path: playwright-report/  # Path to the report
          retention-days: 30  # Retention period for the artifact

This workflow is basic, but it works for almost everything except snapshots testing. Snapshot testing can be part of regression testing - you take a screenshot of a page/component as a reference (master branch or production) and compare it to the current version (changes in PR). This means that for this type of testing we first need to create control screenshots, then create test screenshots and compare them. We need a successful deployment of preview to start testing.

There are four main steps to consider here:

preview deployment,
run tests on deployed master branch,
run tests on deployed current PR branch,
upload artifact to see detailed report.

The simplest version would look like this.

.github/workflows/e2e-test.yml

name: Playwright Tests
on:
  deployment_status:
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    # We need a successful deployment of preview to start testing.
    if: github.event.deployment_status.state == 'success'
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: 18
    - name: Install dependencies
      run: npm ci
    - name: Install Playwright
      run: npx playwright install --with-deps
    - name: Create control screenshots
      run: npx playwright test --update-snapshots
      env:
       # We should assign baseURL in playwright.config.ts to the value of this variable. You also need to set the action secret named E2E_MASTER_BRANCH_PREVIEW_URL to the URL of your deployed master branch.
        PLAYWRIGHT_TEST_BASE_URL: ${{ secrets.E2E_MASTER_BRANCH_PREVIEW_URL }}
    - name: Run Playwright tests
      run: npx playwright test
      env:
      # In this case we can get the deployed preview URL from the event
        PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
    - uses: actions/upload-artifact@v4
      with:
        name: playwright-report
        path: playwright-report/
        retention-days: 30

What are the potential downsides here? The most obvious one is that for each test, we re-run the test on the master branch to get screenshots. This is not very efficient. It is much better to run these tests once on the master branch, save the results, and then use them for all subsequent tests. Also, caching node_modules can save some time. Playwright caching itself is not easy, but we found a way to reduce the time using the official Docker image. We also added HTML report deployment and notifications for failed tests in Slack. For recurring tasks, we used composite actions.

We can still keep everything in one workflow, but we will have jobs that will be triggered by different events: master branch preview deployment and current branch preview deployment.

We will have two completly independent jobs:

on master preview deployment: run tests to create control screenshots, save screenshots and node_modules as artifact,
on current branch preview deployment: pull artifacts with screenshots and node_modules, run tests, upload artifact to see detailed report

Note: Cache may seem more natural to use as a storage for such things. But not quite. If we check the official documentation, we will see that for screenshots, for example, we cannot use cache. So if we can no longer use cache everywhere, the choice was made in favor of artifacts.

name: Playwright Tests
on:
  deployment_status:
env:
  PLAYWRIGHT_TEST_BASE_URL: ${{ secrets.E2E_MASTER_BRANCH_PREVIEW_URL }}
  NEXT_PREVIEW_TOKEN: ${{ secrets.NEXT_PREVIEW_TOKEN }}
  SLACK_INCOMING_WEBHOOK_URL_SNAPSHOTS: ${{ secrets.SLACK_INCOMING_WEBHOOK_URL_SNAPSHOTS }}
jobs:
  create-screenshots-master:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    # Run only on successful deployment of your master branch preview. your-vercel-project should be replaced with the name of your project.
    if: ${{ github.event.deployment_status.state == 'success' && github.event.deployment.environment == 'Production – your-vercel-project'}}
    steps:
      - name: Install dependencies
        run: yarn --prefer-offline
      # We use the official Docker image to speed things up
      - name: Run Playwright tests
        uses: docker://mcr.microsoft.com/playwright:v1.45.0-jammy
        with:
          args: npx playwright test --update-snapshots
      # Delete all ts files to leave only images
      - name: Prepare a screenshots folder
        run: |
          cd e2e
          find . -name "*.ts" -type f -delete
          cd ..
      - uses: actions/upload-artifact@v4
        with:
          name: 'screenshots-master'
          path: e2e/
          retention-days: 7
      - uses: actions/upload-artifact@v4
        with:
          name: node_modules
          path: node_modules/
          retention-days: 7
  create-screenshots-pull-request:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    # Run only on successful deployment of your current branch preview. your-vercel-project should be replaced with the name of your project.
    if: ${{ github.event.deployment_status.state == 'success' && github.event.deployment.environment == 'Preview – your-vercel-project' && github.event.deployment_status.deployment.ref != 'refs/heads/master'}}
    steps:
      # Find the ID of the last successful run. Where e2e-test.yml is the name of the workflow. And store it to a var run_id
      - name: Get Run ID
        id: get_run_id
        run: |
          # Fetch the latest run of the workflow
          response=$(curl -X GET \
          -H "Authorization: Bearer ${{ github.token }}" \
          -H "Accept: application/vnd.github.v3+json" \
          "https://api.github.com/repos/${{ github.repository }}/actions/workflows/e2e-test.yml/runs?status=success&branch=master&per_page=1")

          # Extract the run ID from the response
          run_id=$(echo "$response" | jq -r '.workflow_runs[0].id')

          echo "run_id=${run_id}" >> $GITHUB_OUTPUT
      - name: Print Run ID
        run: |
          echo "Latest Run ID: ${{ steps.get_run_id.outputs.run_id }}"
      - name: Check if artifact with master screenshots exists
        # Now we need to check if the run with the id has artifacts with screenshots. And store it to a var is_exist
        if: steps.get_run_id.outputs.run_id != 'null'
        id: check_screenshots_artifact
        run: |
          response=$(curl -s \
          -H "Authorization: Bearer ${{ github.token }}" \
          -H "Accept: application/vnd.github.v3+json" \
          "https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ steps.get_run_id.outputs.run_id }}/artifacts" \
          | jq ".artifacts[] | select(.name == \"screenshots-master\")")

          if [ "$(echo "$response" | jq -e '. != null')" ]; then
            echo "The artifact exists."
            echo "is_exist=true" >> $GITHUB_OUTPUT
          else
            echo "The artifact does not exist."
            echo "is_exist=false" >> $GITHUB_OUTPUT
          fi
        # Download artifact if exists
      - name: Download screenshots artifact
        if: steps.check_screenshots_artifact.outputs.is_exist != 'false'
        uses: actions/download-artifact@v4
        with:
          name: screenshots-master
          github-token: ${{ github.token }}
          repository: ${{ github.repository }}
          run-id: ${{ steps.get_run_id.outputs.run_id }}
          path: e2e/
      # Check if the run with the id has artifacts with node_modules. And store it to a var is_exist
      - name: Check if artifact with node_modules exists
        if: steps.get_run_id.outputs.run_id != 'null'
        id: check_modules_artifact
        run: |
          response=$(curl -s \
          -H "Authorization: Bearer ${{ github.token }}" \
          -H "Accept: application/vnd.github.v3+json" \
          "https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ steps.get_run_id.outputs.run_id }}/artifacts" \
          | jq ".artifacts[] | select(.name == \"node_modules\")")

          if [ "$(echo "$response" | jq -e '. != null')" ]; then
            echo "The artifact exists."
            echo "is_exist=true" >> $GITHUB_OUTPUT
          else
            echo "The artifact does not exist."
            echo "is_exist=false" >> $GITHUB_OUTPUT
          fi
      - name: Download node_modules artifact
        if: steps.check_modules_artifact.outputs.is_exist != 'false'
        uses: actions/download-artifact@v4
        with:
          name: node_modules
          github-token: ${{ github.token }}
          repository: ${{ github.repository }}
          run-id: ${{ steps.get_run_id.outputs.run_id }}
          path: node_modules
      # We need to explicitly set permissions on this file to make it possible to run in a Docker container. But there is one catch, please check the note right after the code block.
      - run: chmod +x node_modules/.bin/playwright
        if: steps.check_modules_artifact.outputs.is_exist != 'false'
      # If node_modules is not cached, install it as usual.
      - name: Cached dependencies are not found. Install dependencies
        if: steps.check_modules_artifact.outputs.is_exist == 'false'
        run: yarn --prefer-offline
      # Here we run the screenshot tests again, even if the main screenshots are loaded. The only reason for this is to cover the case where we add a new test that is not in the master, and therefore no screenshot is created for it. If this is not your case, just skip this step, but if it is, you should handle this in your test code to skip a specific test if a screenshot already exists, otherwise the screenshot will be overwritten.
      - name: Run Playwright tests
        uses: docker://mcr.microsoft.com/playwright:v1.45.0-jammy
        with:
          args: npx playwright test --update-snapshots
      - name: Prepare a screenshots folder
        run: |
          cd e2e
          find . -name "*.ts" -type f -delete
          cd ..
      - uses: actions/upload-artifact@v4
        with:
          name: screenshots
          path: e2e/
          retention-days: 7
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    needs: create-screenshots-pull-request
    if: success()
    steps:
      - uses: actions/checkout@v4
      - name: Get Run ID
        id: get_run_id
        run: |
          # Fetch the latest run of the workflow
          response=$(curl -X GET \
          -H "Authorization: Bearer ${{ github.token }}" \
          -H "Accept: application/vnd.github.v3+json" \
          "https://api.github.com/repos/${{ github.repository }}/actions/workflows/e2e-test.yml/runs?status=success&branch=master&per_page=1")

          # Extract the run ID from the response
          run_id=$(echo "$response" | jq -r '.workflow_runs[0].id')

          echo "run_id=${run_id}" >> $GITHUB_OUTPUT
      - name: Print Run ID
        run: |
          echo "Latest Run ID: ${{ steps.get_run_id.outputs.run_id }}
      - name: Check if artifact with node_modules exists
        if: steps.get_run_id.outputs.run_id != 'null'
        id: check_modules_artifact
        run: |
          response=$(curl -s \
          -H "Authorization: Bearer ${{ github.token }}" \
          -H "Accept: application/vnd.github.v3+json" \
          "https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ steps.get_run_id.outputs.run_id }}/artifacts" \
          | jq ".artifacts[] | select(.name == \"node_modules\")")

          if [ "$(echo "$response" | jq -e '. != null')" ]; then
            echo "The artifact exists."
            echo "is_exist=true" >> $GITHUB_OUTPUT
          else
            echo "The artifact does not exist."
            echo "is_exist=false" >> $GITHUB_OUTPUT
          fi
      - name: Download node_modules artifact
        if: steps.check_modules_artifact.outputs.is_exist != 'false'
        uses: actions/download-artifact@v4
        with:
          name: node_modules
          github-token: ${{ github.token }}
          repository: ${{ github.repository }}
          run-id: ${{ steps.get_run_id.outputs.run_id }}
          path: node_modules
      - run: chmod +x node_modules/.bin/playwright
        if: steps.check_modules_artifact.outputs.is_exist != 'false'
      - name: Cached dependencies are not found. Install dependencies
        if: steps.check_modules_artifact.outputs.is_exist == 'false'
        run: yarn --prefer-offline
      - name: Download artifacts
        uses: actions/download-artifact@v4
        with:
          name: screenshots
          path: e2e/
      - name: Run Playwright tests
        uses: docker://mcr.microsoft.com/playwright:v1.45.0-jammy
        with:
          # The expression in args means: run the tests and, if they fail, create a is_test_failed.txt file and set the permissions on the playwright-report. Think of this file as a flag or artifact that allows other code to see whether the tests failed or not.
          args: >
            sh -c "
             yarn test:e2e || echo "is_test_failed=true" > is_test_failed.txt &&
             chmod -R 755 playwright-report


        env:
          PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
        id: screenshot_test
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 1
      # Manually check if the file exists and throw an error if it does
      - name: Check if it is failed
        run: |
          if [ -f "is_test_failed.txt" ]; then
            echo "Test failed"
            exit 1
          else
           echo "Test was succesfull"
          fi
  deploy-report:
    # Run only if tests fail
    if: failure()
    env:
      VERCEL_ORG_ID: ${{ secrets.VERCEL_TEAM_ID }}
      VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}
    needs: test
    runs-on: ubuntu-latest
    environment:
      name: preview
      url: ${{ env.UI_TEST_REPORT_URL }}
    steps:
      - uses: actions/checkout@v4
      - name: Check if artifact with playwright-report exists
        if: ${{ github.run_id  }} != 'null'
        id: check_artifact
        run: |
          response=$(curl -s \
          -H "Authorization: Bearer ${{ github.token }}" \
          -H "Accept: application/vnd.github.v3+json" \
          "https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ steps.get_run_id.outputs.run_id }}/artifacts" \
          | jq ".artifacts[] | select(.name == \"playwright-report\")")

          if [ "$(echo "$response" | jq -e '. != null')" ]; then
            echo "The artifact exists."
            echo "is_exist=true" >> $GITHUB_OUTPUT
          else
            echo "The artifact does not exist."
            echo "is_exist=false" >> $GITHUB_OUTPUT
          fi
      # No report. Nothing to deploy.
      - name: Check if report exists
        if: steps.check_artifact.outputs.is_exist == 'false'
        run: exit 1
      - name: Install Vercel CLI
        run: npm install --global vercel@latest
      - name: Pull Vercel Environment Information
        run: vercel pull --yes --environment=preview --token=${{ secrets.VERCEL_UI_TEST_REPORT_TOKEN }}
      - name: Download artifacts
        uses: actions/download-artifact@v4
        with:
          name: playwright-report
      # Prepare report for static deploy
      - name: Copy artifacts
        run: |
          mkdir -p .vercel/output/static
          mv * .vercel/output/static/
      # Create a config.json reqiured for static deployment
      - name: Create config.json
        run: |
          touch .vercel/output/config.json
          echo '{"version": 3}'  >> .vercel/output/config.json
      - name: Deploy Project Artifacts to Vercel
        run: echo "UI_TEST_REPORT_URL=$(vercel deploy --prebuilt --token=${{ secrets.VERCEL_UI_TEST_REPORT_TOKEN }})" >> $GITHUB_ENV

Note: A few words about a potential issue with playwright CLI. Playwright has two packages with similar structure: playwright and @playwright. And if you use both packages and try to use the CLI to run tests, you may get an error. In our case, we started getting an error with the npx playwright test command without any changes from our side. For some reason, we had both packages installed, and it seems that at some point the CLI was linked to the wrong package. And the fix for our case was as described in these comments: first and second.

The code above is too verbose. We can use composite actions to make it look prettier. A composite action is basically a way to externalize a piece of repetitive code to make it reusable across jobs. An action can have inputs and outputs. An action can be in the same repository or from any public repository. We've seen composite actions before: when we download or download artifacts for example. We can create our own for checking run id, checking artifact and so on.

Below is a real-world example of using composite actions.

Real world example

name: E2E test
on:
  deployment_status:
env:
  PLAYWRIGHT_TEST_BASE_URL: ${{ secrets.E2E_MASTER_BRANCH_PREVIEW_URL }}
  NEXT_PREVIEW_TOKEN: ${{ secrets.NEXT_PREVIEW_TOKEN }}
  SLACK_INCOMING_WEBHOOK_URL_SNAPSHOTS: ${{ secrets.SLACK_INCOMING_WEBHOOK_URL_SNAPSHOTS }}
  CI_E2E_TEST: true
jobs:
  seed-screenshots-master:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: ${{ github.event.deployment_status.state == 'success' && github.event.deployment.environment == 'Production – easy-park-front'}}
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/e2e-seed
        with:
          store-period: 7
          artifact-name: 'screenshots-master'
          is-master: true
      - uses: actions/upload-artifact@v4
        with:
          name: node_modules
          path: node_modules/
          retention-days: 7
  seed-screenshots-pull-request:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: ${{ github.event.deployment_status.state == 'success' && github.event.deployment.environment == 'Preview – easy-park-front' && github.event.deployment_status.deployment.ref != 'refs/heads/master'}}
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/e2e-seed
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    needs: seed-screenshots-pull-request
    if: success()
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/e2e-get-last-run-id
        id: last-run
      - uses: ./.github/actions/e2e-install
        with:
          run-id: ${{ steps.last-run.outputs.run_id }}
      - name: Download artifacts
        uses: actions/download-artifact@v4
        with:
          name: screenshots
          path: e2e/
      - name: Run Playwright tests
        uses: docker://mcr.microsoft.com/playwright:v1.45.0-jammy
        with:
          args: >
            sh -c "
             yarn test:e2e || echo "is_test_failed=true" > is_test_failed.txt &&
             chmod -R 755 playwright-report &&
             node src/utils/patchPlaywrightReport.js"
        env:
          PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
        id: screenshot_test
      - uses: actions/upload-artifact@v4
        with:
          name: is_test_failed
          path: is_test_failed.txt
          retention-days: 1
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 1
  deploy-report:
    if: success() || failure()
    env:
      VERCEL_ORG_ID: ${{ secrets.VERCEL_TEAM_ID }}
      VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}
    needs: test
    runs-on: ubuntu-latest
    environment:
      name: preview
      url: ${{ env.UI_TEST_REPORT_URL }}
    steps:
      - uses: actions/checkout@v4
      - name: Install Vercel CLI
        run: npm install --global vercel@latest
      - name: Pull Vercel Environment Information
        run: vercel pull --yes --environment=preview --token=${{ secrets.VERCEL_UI_TEST_REPORT_TOKEN }}
      - name: Download artifacts
        uses: actions/download-artifact@v4
        with:
          name: playwright-report
      - name: Copy artifacts
        run: |
          mkdir -p .vercel/output/static
          mv * .vercel/output/static/
      - name: Create config.json
        run: |
          touch .vercel/output/config.json
          echo '{"version": 3}'  >> .vercel/output/config.json
      - name: Deploy Project Artifacts to Vercel
        run: echo "UI_TEST_REPORT_URL=$(vercel deploy --prebuilt --token=${{ secrets.VERCEL_UI_TEST_REPORT_TOKEN }})" >> $GITHUB_ENV
      - uses: ./.github/actions/e2e-check-artifact
        with:
          run-id: ${{ github.run_id  }}
          artifact-name: 'is_test_failed'
        id: check-if-test-failed
      - name: Get Deployment Info
        id: get-info
        run: |
          # Fetch the commit info from the API
          commit_name=$(curl -s -H "Authorization: token ${{ github.token }}" \
            "https://api.github.com/repos/${{ github.repository }}/commits/${{ github.event.deployment.sha }}" | jq -r '.commit.message')
          # Fetch the PR info if it exists
          pr_number=$(curl -s -H "Authorization: token ${{ github.token }}" \
            "https://api.github.com/repos/${{ github.repository }}/commits/${{ github.event.deployment.sha }}/pulls" | jq -r '.[0].number')
          pr_link=""
          if [ "$pr_number" != "null" ]; then
            pr_link="https://github.com/${{ github.repository }}/pull/${pr_number}"
          fi
          # Print the results
          echo "Commit name: ${commit_name}"
          echo "Commit SHA: ${{ github.event.deployment.sha }}"
          echo "PR Link: ${pr_link}"
          # Set the output variables
          echo "COMMIT_NAME=${commit_name}" >> "$GITHUB_ENV"
          echo "COMMIT_SHA=${{ github.event.deployment.sha }}" >> "$GITHUB_ENV"
          echo "PR_LINK=${pr_link}" >> "$GITHUB_ENV"
      - uses: actions/checkout@v4
      - uses: ./.github/actions/report-error
        if: steps.check-if-test-failed.outputs.is_exist != 'false'
        env:
          SLACK_INCOMING_WEBHOOK_URL: ${{ secrets.SLACK_INCOMING_WEBHOOK_URL }}
          NEXT_PUBLIC_PRODUCTION_DOMAIN: ${{ secrets.NEXT_PUBLIC_PRODUCTION_DOMAIN }}
          ERROR_EMAIL_LIST: ${{ secrets.ERROR_EMAIL_LIST }}
        with:
          args: "'Snapshot testing' 'Changes detected. Click link in slug to review changes' ${{ env.UI_TEST_REPORT_URL }} ${{ env.PR_LINK }} '${{ env.COMMIT_NAME }}'"
          mode: CI_SNAPSHOTS_REPORT

Composite action to check for the presence of an artifact.

name: E2E check artifact
description: Check if an artifact with a specific name exists
inputs:
  artifact-name:
    description: 'Name of the artifact'
    required: true
  run-id:
    description: 'Last run id'
    required: true
    default: 'null'
outputs:
  is_exist:
    description: 'Does the artifact exist?'
    value: ${{ steps.check_artifact.outputs.is_exist }}
runs:
  using: 'composite'
  steps:
    - name: Check if Artifact Exists
      if: inputs.run-id != 'null'
      id: check_artifact
      run: |
        response=$(curl -s \
        -H "Authorization: Bearer ${{ github.token }}" \
        -H "Accept: application/vnd.github.v3+json" \
        "https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ inputs.run-id }}/artifacts" \
        | jq ".artifacts[] | select(.name == \"${{ inputs.artifact-name }}\")")
        if [ "$(echo "$response" | jq -e '. != null')" ]; then
          echo "The artifact exists."
          echo "is_exist=true" >> $GITHUB_OUTPUT
        else
          echo "The artifact does not exist."
          echo "is_exist=false" >> $GITHUB_OUTPUT
        fi
      shell: bash

Composite action to get the ID of the last successful run in the master branch.

name: E2E get last run id
description: Get id of the last successful seed run on master branch
outputs:
  run_id:
    description: 'Last run id'
    value: ${{ steps.get_run_id.outputs.run_id }}
runs:
  using: 'composite'
  steps:
    - name: Get Run ID
      id: get_run_id
      run: |
        # Fetch the latest run of the workflow
        response=$(curl -X GET \
        -H "Authorization: Bearer ${{ github.token }}" \
        -H "Accept: application/vnd.github.v3+json" \
        "https://api.github.com/repos/${{ github.repository }}/actions/workflows/e2e-test.yml/runs?status=success&branch=master&per_page=1")
        # Extract the run ID from the response
        run_id=$(echo "$response" | jq -r '.workflow_runs[0].id')
        echo "run_id=${run_id}" >> $GITHUB_OUTPUT
      shell: bash
    - name: Print Run ID
      run: |
        echo "Latest Run ID: ${{ steps.get_run_id.outputs.run_id }}"
      shell: bash

Composite action to handle installation of node_modules.

name: E2E dependencies
description: Install all necessary dependencies to run e2e tests
inputs:
  run-id:
    description: 'Last run id'
    required: true
    default: 'null'
  is-master:
    description: 'Is master seed'
    required: true
    default: 'false'
runs:
  using: 'composite'
  steps:
    - uses: ./.github/actions/e2e-check-artifact
      with:
        run-id: ${{ inputs.run-id }}
        artifact-name: 'node_modules'
      id: check-dependencies-cache
    - name: Download artifacts
      if: inputs.is-master == 'false' && steps.check-seed-cache.outputs.is_exist != 'false'
      uses: actions/download-artifact@v4
      with:
        name: node_modules
        github-token: ${{ github.token }}
        repository: ${{ github.repository }}
        run-id: ${{ inputs.run-id }}
        path: node_modules
    - run: chmod +x node_modules/.bin/playwright
      if: inputs.is-master == 'false' && steps.check-seed-cache.outputs.is_exist != 'false'
      shell: bash
    - name: Cached dependencies are not found. Install dependencies
      if: inputs.is-master != 'false'
      run: yarn --prefer-offline
      shell: bash

Composite action for taking screenshots in the main branch.

name: E2E seed
description: Seed master screenshots
inputs:
  store-period:
    description: 'Artifact retention period in days'
    required: true
    default: 1
  artifact-name:
    description: 'Name of the artifact'
    required: true
    default: 'screenshots'
  is-master:
    description: 'Is master seed'
    required: true
    default: 'false'
runs:
  using: 'composite'
  steps:
    - uses: ./.github/actions/e2e-get-last-run-id
      id: last-run
    - uses: ./.github/actions/e2e-install
      with:
        run-id: ${{ steps.last-run.outputs.run_id }}
        is-master: inputs.is-master
    - uses: ./.github/actions/e2e-check-artifact
      with:
        run-id: ${{ steps.last-run.outputs.run_id }}
        artifact-name: 'node_modules'
      id: check-seed-cache
    - name: Download artifacts
      if: inputs.is-master == 'false' && steps.check-seed-cache.outputs.is_exist != 'false'
      uses: actions/download-artifact@v4
      with:
        name: screenshots-master
        github-token: ${{ github.token }}
        repository: ${{ github.repository }}
        run-id: ${{ steps.last-run.outputs.run_id}}
        path: e2e/
    - name: Run Playwright tests
      uses: docker://mcr.microsoft.com/playwright:v1.45.0-jammy
      with:
        args: yarn test:seed
    - name: Prepare a screenshots folder
      run: |
        cd e2e
        find . -name "*.ts" -type f -delete
        cd ..
      shell: bash
    - uses: actions/upload-artifact@v4
      with:
        name: ${{ inputs.artifact-name }}
        path: e2e/
        retention-days: ${{ inputs.store-period }}

Summary

There are tools for regression testing, but they seem to be paid. With Playwright you can start testing your app today and not everyone needs a solution as complex as the one above, you can start simple. With some of the optimizations presented above, your code will have reasonable execution time.

Testing web application performance with Playwright

Sergey Labut — Wed, 07 Aug 2024 11:42:55 +0000

In this article, we will discuss how to monitor your website's performance using Playwright. If you are new to playwriting and testing in general, please read this article first.

Why is this important?
When and where should such tests be conducted?
What will we measure and how?
Test example
Summary

Why is this important?

If the performance of your website is important to you and you want to know if there are any issues with it, there are plenty of tools available on the market for collecting real user data (Real User Monitoring). However, if there is a performance issue, this data is often not enough to resolve it. Performance tests allow you to measure metrics in an isolated environment, ensuring reproducibility and enabling you to track how changes in the code affect test results.

When and where should such tests be conducted?

The idea is to run these tests every time we want to push changes to the main branch. This can serve as the last check to ensure that performance has not been compromised. These tests should be run on CI on every preview deployment. This article does not cover how to set up CI for Playwright, but you can find information about that in this article.

What will we measure and how?

We are going to measure Core Web Vitals metrics in a browser environment. This is not quite the same as what we get in Lighthouse; instead, it’s what we can see in the Performance tab in Chrome. Firstly, we believe this data is more reliable than Lighthouse data. Secondly, we can control test conditions such as screen size, data transmission speed, CPU speed, etc.
Ultimately, we want to measure LCP, CLS, and FCP for a mobile device under various network conditions: fast 3G, slow 3G, and slow 4G. We can also slow down the CPU to 6X, for example. This is optional since the CI worker is already slow enough.

Note: This is all set up as a test, so we should skip it for devices that do not use Chrome, as this functionality is only available there.

Test example

import { expect, test } from '@playwright/test';
import type { CDPSession, Page, TestInfo } from '@playwright/test';

// Define types for various structures used in the test
type Test = { name: string; value: number };

type Devices = {
  mobile: Test[];
  desktop: Test[];
};

type Results = {
  slow3g: { normal: Devices };
  fast3g: { normal: Devices };
  slow4g: { normal: Devices };
};

type Network = {
  download: number;
  upload: number;
  latency: number;
  noThrottle?: boolean;
  name: keyof Results;
};

type CPU = {
  rate?: number;
  noThrottle?: boolean;
  name: keyof Results['fast3g'];
};

const SLUG = '/';

// Function to collect performance vitals
async function collectVitals(
  page: Page,
  session: CDPSession,
  testInfo: TestInfo,
  network: Network,
  cpu: CPU,
  prefix: 'no cache' | 'cache',
) {
  // Collect LCP (Largest Contentful Paint) metric
  const LCP: string = await page.evaluate(
    () =>
      new Promise((resolve) => {
        new PerformanceObserver((list) => {
          const entries = list.getEntries();
          const lcp = entries.at(-1);
          return resolve(Number(lcp?.startTime).toFixed(2));
        }).observe({
          type: 'largest-contentful-paint',
          buffered: true,
        });
      }),
  );

  // Collect TTFB (Time to First Byte) metric
  const TTFB = await page.evaluate(
    () =>
      new Promise((resolve) => {
        new PerformanceObserver((list) => {
          const entries = list.getEntries();
          if (entries.length > 0) {
            const navigationEntry = entries[0];
            const ttfb =
              //@ts-ignore
              navigationEntry.responseStart - navigationEntry.fetchStart;
            resolve(ttfb.toFixed(2));
          }
        }).observe({
          type: 'navigation',
          buffered: true,
        });
      }),
  );

  // Collect CLS (Cumulative Layout Shift) metric
  const CLS: number = await page.evaluate(
    () =>
      new Promise((resolve) => {
        new PerformanceObserver((list) => {
          let total = 0;
          for (const entry of list.getEntries()) {
            // @ts-ignore
            total += entry.value;
          }
          return resolve(+total.toPrecision(2));
        }).observe({ type: 'layout-shift', buffered: true });
      }),
  );

  // Collect long tasks metric
  const longTasks = await page.evaluate(
    () =>
      new Promise((resolve) => {
        new PerformanceObserver((list) => {
          let total = 0;
          for (const entry of list.getEntries()) {
            total += entry.duration;
          }
          return resolve(total);
        }).observe({ type: 'longtask', buffered: true });
      }),
  );

  // Collect navigation metric
  const L = await page.evaluate(
    () =>
      new Promise((resolve) => {
        new PerformanceObserver((list) => {
          let max = 0;
          list.getEntries().forEach((entry) => {
            if (max < entry.duration) {
              max = entry.duration;
            }
          });
          return resolve(max.toFixed(2));
        }).observe({
          type: 'navigation',
          buffered: true,
        });
      }),
  );

  // Collect paint timings
  const paintTimingJson = await page.evaluate(() =>
    JSON.stringify(window.performance.getEntriesByType('paint')),
  );

  const paintTiming = await JSON.parse(paintTimingJson);

  const perfData = {
    CLS,
    longTasks,
    LCP,
    FCP: 0,
    DCL: '',
    L,
    TTFB: 0,
    INP: 0,
  };

  // Measure INP (Interaction to Next Paint) for buttons
  for (const button of await page.getByRole('button').all()) {
    const className = await button.getAttribute('class');
    const classWithLangSelector = className
      ?.split(' ')
      .filter((e) => e.includes('StyledOpenButton'));

    if (classWithLangSelector?.length) {
      const selector = '.' + classWithLangSelector[0];
      perfData.INP = await page.evaluate(async (selector) => {
        return new Promise(async (resolve) => {
          requestAnimationFrame(async () => {
            const startTime = performance.now();
            // @ts-ignore
            document.querySelector(selector)?.click();
            requestAnimationFrame(async () => {
              const endTime = performance.now();
              resolve(+(endTime - startTime).toFixed(2));
            });
          });
        });
      }, selector);
    }
  }

  // Extract FCP (First Contentful Paint) from paint timings
  for (const metric of paintTiming) {
    if (metric.name === 'first-contentful-paint') {
      perfData.FCP = metric.startTime.toFixed(2);
    }
  }

  // Get performance metrics from CDP session
  const performanceMetrics = await session.send('Performance.getMetrics');
  results[network.name][cpu.name][testInfo.project.name as keyof Devices] =
    performanceMetrics.metrics;

  // Add paint timings to results
  paintTiming.forEach((element: any) => {
    results[network.name][cpu.name][
      testInfo.project.name as keyof Devices
    ].push({
      name: element.name,
      value: element.startTime,
    });
  });

  // Extract DCL (DOMContentLoaded) from performance metrics
  for (const metric of performanceMetrics.metrics) {
    if (metric.name === 'DomContentLoaded') {
      perfData.DCL = metric.value.toFixed(2);
    }
  }

  // Assert performance metrics against thresholds
  expect(+perfData.LCP).toBeLessThan(2500);
  expect(+perfData.FCP).toBeLessThan(1800);
  expect(+perfData.CLS).toBeLessThan(0.1);
  expect(+perfData.TTFB).toBeLessThan(800);
  expect(+perfData.INP).toBeLessThan(200);
}

// Basic test function to set up and run performance tests
async function basicTest(
  page: Page,
  network: Network,
  cpu: CPU,
  testInfo: TestInfo,
) {
  // Create a new connection to an existing CDP session to enable performance metrics
  const session = await page.context().newCDPSession(page);
  // Enable performance metrics collection in CDP session
  await session.send('Performance.enable');

  // Apply CPU throttling if needed
  if (!cpu.noThrottle && cpu.rate) {
    await session.send('Emulation.setCPUThrottlingRate', { rate: cpu.rate });
  }

  // Apply network throttling if needed
  if (!network.noThrottle) {
    await session.send('Network.enable');
    await session.send('Network.emulateNetworkConditions', {
      offline: false,
      downloadThroughput: network.download,
      uploadThroughput: network.upload,
      latency: network.latency,
    });
  }

  // Navigate to the specified URL
  await page.goto(SLUG);
  // Collect performance vitals
  await collectVitals(page, session, testInfo, network, cpu, 'no cache');
}

// Define network conditions for different test scenarios
const networkConditions = {
  slow3g: {
    download: ((500 * 1000) / 8) * 0.8,
    upload: ((500 * 1000) / 8) * 0.8,
    latency: 400 * 5,
    name: 'slow3g' as keyof Results,
  },
  fast3g: {
    download: ((1.6 * 1000 * 1000) / 8) * 0.9,
    upload: ((750 * 1000) / 8) * 0.9,
    latency: 150 * 3.75,
    name: 'fast3g' as keyof Results,
  },
  slow4g: {
    download: ((1.6 * 1000 * 1000) / 8) * 1.5,
    upload: ((750 * 1000) / 8) * 1.5,
    latency: 150,
    name: 'slow4g' as keyof Results,
  },
};

// Define CPU conditions for different test scenarios
const cpuConditions = {
  normal: {
    noThrottle: true,
    name: 'normal' as keyof Results['fast3g'],
    weight: 10,
  },
};

// Initialize results object to store performance metrics
const results: Results = {
  slow3g: {
    normal: {
      mobile: [],
      desktop: [],
    },
  },
  fast3g: {
    normal: {
      mobile: [],
      desktop: [],
    },
  },
  slow4g: {
    normal: {
      mobile: [],
      desktop: [],
    },
  },
};

// Describe the performance test suite
test.describe('Performance test', () => {
  // Skip tests for non-Chromium browsers and non-mobile devices
  test.skip(
    ({ browserName, isMobile }) => !isMobile || browserName !== 'chromium',
    'Chromium only!',
  );

  // Loop through network and CPU conditions to run performance tests
  for (const networkDetails of Object.values(networkConditions)) {
    for (const cpuDetails of Object.values(cpuConditions)) {
      test(`Get performance metrics for network: ${
        networkDetails.name
      } and cpu: ${`slow ${cpuDetails.name}`}`, async ({
        page,
      }, workerInfo) => {
        await basicTest(
          page,
          networkConditions[networkDetails.name],
          cpuConditions[cpuDetails.name],
          workerInfo,
        );
      });
    }
  }
});

Note: The results of this test only give you a vague idea of the user experience, and you should only compare it to previous results. It does not cover all cases: for example, networking will be faster and the processor will be slower for a hosted worker than for the average consumer device. But it should give you a baseline with other monitoring tools to understand where you stand.

Summary

Performance tests using Playwright are another tool in your performance monitoring toolkit. However, unlike many others, this tool has an advantage—it immediately reflects changes in the code. If you have previous results, you can determine whether the performance has improved or worsened.

The preferred environment to run this test is CI. Here's an article on testing with Playwright on CI.

Getting Started with Automated Testing

Sergey Labut — Wed, 07 Aug 2024 11:42:51 +0000

Testing is important to ensure that your web application works as expected. This becomes even more important as your application grows. Setting up and maintaining testing often seems like an unnecessary overhead. In this article, we'll look at the benefits of testing and learn how to quickly start testing your application.

What is Automated Testing?
Importance of Automated Testing in Web Development
Types of Automated Tests
Overview of Popular Testing Tools and Frameworks
How to choose Testing Tool
Setting Up Your Testing Environment
Writing Your First Test
Conclusion

What is Automated Testing?

Automated testing is the process of using software tools to run tests on code automatically, rather than manually. It involves creating scripts and test cases that simulate user interactions, validate functionality, and ensure that the software behaves as expected. The primary benefits of automated testing are speed, efficiency, and the ability to consistently repeat tests without human intervention.

Importance of Automated Testing in Web Development

We can highlight several reasons why automated testing is important:

Consistent and Full Coverage: tests run the same way every time, reducing human error and ensuring consistent results. They can cover a wide range of scenarios and edge cases that might be missed in manual testing.
Early Bug Detection and Rapid Feedback: tests can identify bugs early in the development process, providing immediate feedback to developers. This reduces the cost and effort required to fix issues later.
Faster Test Execution and Parallel Testing: tests run significantly faster than manual tests and multiple tests can run in parallel, saving time and resources.
Continuous Monitoring and 24/7 Testing: tests can continuously monitor the application’s performance and functionality, and can be scheduled to run at any time, including overnight, ensuring continuous testing.
Seamless CI/CD Integration and Continuous Deployment: tests can be integrated into CI/CD pipelines, ensuring that every code change is tested before deployment. This enables continuous deployment by ensuring that the application is always in a releasable state.
Refactoring Confidence and Regression Testing: tests provide confidence for developers to refactor code, knowing that tests will catch any regressions or new issues. Regression tests ensure that new code changes do not break existing functionality.

Types of Automated Tests

The three primary types of automated tests are unit tests, integration tests, and end-to-end (E2E) tests. Comparison and Use Cases:

Type of Test	Scope	Purpose	Example Use Case
Unit Test	Individual components	Ensure each unit performs correctly	Testing a utility function
Integration Test	Interactions between units	Ensure combined units work together	Testing service-to-database interaction
End-to-End Test	Complete application flow	Ensure the application works from user's perspective	Testing the entire user login process

Apart from this, we can also mention:
Regression Testing. It involves re-running previously conducted tests to ensure that recent code changes have not adversely affected existing functionality. This helps in maintaining the integrity of the application as it evolves.
Performance Testing. It assesses the speed, responsiveness, and stability of an application under various conditions. Includes load testing, stress testing, and scalability testing.

Overview of Popular Testing Tools and Frameworks

Unit Testing

Jest: Jest is a delightful JavaScript testing framework with a focus on simplicity. It works seamlessly with projects using Babel, TypeScript, Node, React, Angular, and Vue.
Mocha: Mocha is a feature-rich JavaScript test framework running on Node.js, making asynchronous testing simple and fun.
Jasmine: Jasmine is a behavior-driven development framework for testing JavaScript code. It has no dependencies and does not require a DOM.

Integration Testing

Jest: Jest can also be used for integration testing by grouping related units and testing their interactions.
Mocha: Mocha is flexible enough to handle integration tests, often paired with assertion libraries like Chai and HTTP request libraries like Supertest.
Ava: Ava is a test runner that helps you write tests in JavaScript with a minimalistic syntax, supporting asynchronous testing and providing clean error output.

End-to-End Testing

Cypress: Cypress is a fast, easy, and reliable testing framework for anything that runs in a browser. It is designed to make it easy to set up, write, and debug tests.
Puppeteer: Puppeteer is a Node library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol.
Playwright: Playwright is a Node.js library to automate Chromium, Firefox, and WebKit with a single API. It enables reliable end-to-end testing for modern web applications.

How to choose Testing Tool

Choosing the right testing tool for your project depends on several factors, including the type of application you're testing, your team's expertise, and the specific testing requirements. Here are some steps and considerations to help you choose a testing tool:

Understand Your Testing Needs: Determine whether you need to perform unit tests, integration tests, end-to-end tests, or a combination of these. Consider whether you need to test on different browsers, devices, or operating systems.
Evaluate Tool Capabilities: Review the features offered by each testing tool, such as assertions, test runners, mocking capabilities, and reporting. Ensure the tool supports the technologies and frameworks used in your project (e.g., React, Angular, Node.js).
Assess Learning Curve: Check the availability and quality of documentation, tutorials, and community support. Evaluate how easy it is to set up, write tests, and integrate with your existing development workflow.
Consider Performance and Scalability: Assess the execution speed of tests, especially for end-to-end tests that may involve UI interactions. Determine how well the tool handles large test suites and distributed testing.
Community and Support: Look for active communities and forums where you can get help and share experiences. Consider the availability of commercial support, if needed, for enterprise-level projects.
Integration with CI/CD: Check how easily the tool integrates with your continuous integration and deployment pipelines (e.g., Jenkins, GitHub Actions).
Cost and Licensing: Evaluate whether you prefer open-source tools with community support or commercial tools with additional features and support.

For this guide, we choose Playwright for several reasons:

Despite the fact that the main goal of Playwright is e2e testing, it is not prohibited to conduct other types of testing – same tool for all kind of tests,
Native TS support,
Some Playwright specific features: performance metrics, headless mode, device emulation, etc.

Setting Up Your Testing Environment

To install Playwright run npm init playwright@latest

After the previous command playwright.config.ts should be created in your repository.

Here is an example of playwright.config.ts.

import { defineConfig, devices } from '@playwright/test';
// Preview or production URL
const URL = process.env.PLAYWRIGHT_TEST_BASE_URL;
export default defineConfig({
  testDir: './tests',
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!URL,
  /* Retry on CI only */
  retries: URL ? 1 : 0,
  /* Opt out of parallel tests on CI. */
  workers: URL ? 1 : 4,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: 'html',
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    baseURL: URL || 'http://localhost:3000',
    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',
  },
  /* Stores an array of devices which represents real devices. Each device can have its own browser, width, height and other settings. Each test will run on every device you describe, which means the final number of tests is equal to the number of tests you wrote multiplied by the number of devices. 20 (final number of tests) = 5 (number of tests) x 4 (projects/devices) */
  projects: [
    {
      name: 'Desktop Chrome',
      use: {
        ...devices['Desktop Chrome'],
      },
    },
    {
      name: 'Moto G4',
      use: {
        ...devices['Moto G4'],
      },
    },
    {
      name: 'iPhone 13 Pro',
      use: {
        ...devices['iPhone 13 Pro'],
      },
    },
    {
      name: 'Desktop Safari',
      use: {
        ...devices['Desktop Safari'],
      },
    },
  ],
  expect: {
// Specifies the timeout (in milliseconds) for each test case
    timeout: 10000,
    toMatchSnapshot: {
// Configuration for snapshot testing with a maximum difference pixel ratio of
      maxDiffPixelRatio: 0.07,
    },
    toHaveScreenshot: {
//Configuration for screenshot comparison with a maximum difference pixel ratio of
      maxDiffPixelRatio: 0.07,
    },
  },
//Specifies the global timeout (in milliseconds) for test execution
  timeout: 30 * 1000,
});

Writing Your First Test

We will start with the simplest example – unit test.

import test, { expect } from "@playwright/test";

const sum = (a: number, b: number) => a + b;

test.describe("Test sum", () => {
  test("adds 1 + 2 to equal 3", () => {
    expect(sum(1, 2)).toBe(3);
  });
});

This test suite tests the result of a function for a specific set of arguments.

To run the test, run npx playwright test. This will run all your tests. To run a single test just add the name of the file after the command: npx playwright test yourTest. If your test file named sum.spec.ts command would be npx playwright test sum.

Now let's write an integration test.

import test, { expect } from "@playwright/test";

test.describe("GET /users", () => {
  test("responds with json", async ({ request }) => {
    const response = await request.get("/users");

    expect(response.ok()).toBeTruthy();
  });
});

Note: it may be slower than the simpler and lighter Jest/Mocha. But we simply want to show that the Playwright is capable of such tests too. The number of projects/devices is arbitrary in config but to skip it for a specific project/device you can use test.skip(skipCondition).

import test, { expect } from "@playwright/test";
test.describe("GET /users", () => {
  test.skip(({ browserName }) => browserName !== "chromium", "Chromium only!");
  test("responds with json", async ({ request }) => {
    const response = await request.get("/users");
    expect(response.ok()).toBeTruthy();
  });
});

And now – e2e testing.

To test e2e, you can have a scenario of user interaction with a web page or test the basic functionality of interactive components on the page.

Imagine we have a form. This form has multiple fields such as name, phone number, email address, etc. Each field is assumed to have an input validation and an error state when the input is invalid. So instead of checking it every time by hand we can write a test to check it automatically.

Our form has several fields:

Name,
Email,
Text,
Checkbox,
File uploader,
Date picker.

Note: in the code examples, we will limit our fields to only the following fields: name, file uploader, checkbox. Because all text fields are basically treated the same.

We can first try to submit a blank form to trigger validation errors and check if errors occur.

import test, { expect } from "@playwright/test";
import { join } from "path";

test.beforeEach(async ({ page }) => {
  await page.goto("https://example.com/contact");
});

test.describe("Contact form", () => {
   test("Check the validation", async ({ page }) => {
//First, before interacting, we must check that there are no errors on the page.
    const nameFieldError = "Name required";
    await expect(page.getByText(nameFieldError)).not.toBeAttached();
    const policyFieldError =
      "Please approve our privacy policy by ticking the box";
    await expect(page.getByText(policyFieldError)).not.toBeAttached();
// Here, when clicked, we launch a check, and errors should appear on the page.
    await page.getByRole("button").click();
//Check if errors have appeared
    await expect(page.getByText(nameFieldError)).toBeAttached();
    await expect(page.getByText(policyFieldError)).toBeAttached();
  });
});

Okay, the next step might be to populate the fields with data and then make sure there are no errors. Right at the end of the previous test we can add the following lines of code.

// fill out the fields one by one and check that the value is correct  
    const nameField = page.locator("#name");
    const nameValue = "Name";
    await nameField.fill(nameValue);
    await expect(nameField).toHaveValue(nameValue);
    const policyField = page.locator('input[id="important checkbox"]');
    await policyField.check();
    await expect(policyField).toBeChecked();
    const fileName = "test.pdf";
    await page
      .locator("#fileInput")
      .setInputFiles(join(process.cwd(), "/public/uploads/" + fileName));
//Here we expect to see no more validation errors.
    await expect(page.getByText(fileName)).toBeAttached();
    await expect(page.getByText(fileName)).toContainText(fileName);
    await expect(page.getByText(nameFieldError)).not.toBeAttached();
    await expect(page.getByText(policyFieldError)).not.toBeAttached();

We can also add a UI regression test: if you are using git or any other version control system, before pushing changes to master, you can check if there are any UI changes in the branch (master screenshots required for comparison).

We can also submit the form now and check how our API handles it. It can be noted that this is more like integration testing and it is beyond the topic of this article.

Here is the code with all the pieces. I think it's best to keep this in one test to avoid race conditions.

import test, { expect } from "@playwright/test";
import { join } from "path";
test.beforeEach(async ({ page }) => {
  await page.goto("https://example.com/contact");
});
test.describe("Contact form", () => {
  test("Check the form", async ({ page }) => {
    // first, before interacting, we must check that there are no errors on the page.
    const nameFieldError = "Name required";
    await expect(page.getByText(nameFieldError)).not.toBeAttached();
    const policyFieldError =
      "Please approve our privacy policy by ticking the box";
    await expect(page.getByText(policyFieldError)).not.toBeAttached();
    // here, when clicked, we launch a check, and errors should appear on the page.
    await page.getByRole("button").click();
    // check if errors have appeared
    await expect(page.getByText(nameFieldError)).toBeAttached();
    await expect(page.getByText(policyFieldError)).toBeAttached();
    // check error state UI
    expect(await page.screenshot({ fullPage: true })).toMatchSnapshot(
      "form-error-state.png"
    );
    // fill out the fields one by one and check that the value is correct
    const nameField = page.locator("#name");
    const nameValue = "Name";
    await nameField.fill(nameValue);
    await expect(nameField).toHaveValue(nameValue);
    const policyField = page.locator('input[id="important checkbox"]');
    await policyField.check();
    await expect(policyField).toBeChecked();
    const fileName = "test.pdf";
    await page
      .locator("#fileInput")
      .setInputFiles(join(process.cwd(), "/public/uploads/" + fileName));
    await expect(page.getByText(fileName)).toBeAttached();
    await expect(page.getByText(fileName)).toContainText(fileName);
    // here we expect to see no more validation errors.
    await expect(page.getByText(nameFieldError)).not.toBeAttached();
    await expect(page.getByText(policyFieldError)).not.toBeAttached();
    // check filled form UI
    expect(await page.screenshot({ fullPage: true })).toMatchSnapshot(
      "fille-from.png"
    );
    // Submit the form
    await page.getByRole("button").click();
  });
});

Since we have UI tests, we essentially have to run our test twice: first on master and then on branch. Or we can use some kind of cache for master screenshots.

Before running tests, you should start the development environment: yarn run dev.
To run your test locally you need to run the npx playwright test --update-snapshots command on the master and npx playwright test on your current branch. To easy up things a little bit we can automate it under one command:

  "scripts": {
    "test:local": "git checkout master && npx playwright test --update-snapshots && git checkout - && npx playwright test",
  },

Note: to make this work your screenshots should be ignored: /tests/*/*.png on .gitignore.

Conclusion

As a result, we can say that Playwright is suitable not only for e2e testing, but also for many other purposes: unit testing - you can test regular functions with expect matcher, snapshot/UI regression testing - you can test specific isolated components by removing the rest from the page, integration – by sending data to other API/services.

Playwright is also suitable for monitoring the performance of your site. We have a separate article on this topic.

In this article we consider only local testing but Playwright starts to shine on CI. We will consider CI with Playwright using Github actions in a separate article.

Critical CSS with NextJS

Sergey Labut — Fri, 31 May 2024 09:36:37 +0000

In this article, we will explore the possible approaches for implementing Critical CSS with different render modes, including, but not limited to, Next JS.

Limitations
Critical CSS for Static Websites
Critical CSS for Static Websites With CSS-in-JS
Fully Static With Both: Regular CSS and CSS-in-JS
NextJS Pages Router Critical CSS
Critical CSS for Dynamic Pages (SSR) in NextJS Pages Router with Custom Server
NextJS App Router Critical CSS
Critical CSS for Dynamic Pages (SSR) in NextJS App Router with Custom Server
Conclusion

There are different ways and tools to extract Critical CSS, but we will look at the most efficient and scalable tool — Beasties (forked from Critters, as Critters is now deprecated). This tool isn't perfect either, but at least it has the shortest extraction time among Critical CSS tools and is easy to integrate. A possible downside is that there may be scenarios where additional steps might be required before and after optimization. We will explore this in more detail below.

Limitations

While Critical CSS will boost performance (by performance, we mean the first render, First Contentful Paint (FCP), and in some cases, Largest Contentful Paint (LCP)), in almost all cases, we need to highlight that Critical CSS extraction might be tricky when executed during runtime. In a nutshell, to extract Critical CSS, we need the HTML that we plan to send to the user, along with the CSS attached to it. After extraction, we need to modify the original HTML to have the Critical CSS inlined and the original CSS lazy-loaded. While static site generation (SSG) is fairly straightforward and results in static files that can be easily analyzed and modified, server-side rendering (SSR) introduces nuances that can make Critical CSS optimizations less effective. So, we'll apply Critical CSS optimization to static sites only, but we will also explore some possible approaches for dynamic sites.

Critical CSS for Static Websites

Okay, let's look at the first simplest case - a completely static site. All pages are rendered at build time. In this case, you simply create the following script and run it at the end of the build pipeline.

Here are the basic steps:

Get all HTML files
Initialize Critters with settings
Read HTML file
Process HTML file with Beasties
Parse HTML after Beasties
Find all the <link/> tags in the <head/> and remove each <link/>.
Save HTML file

We will use the code from this snippet to get all the HTML files from a folder. (took it from here)

const fs = require("fs");

// Recursive function to get files
function getHTMLFiles(dir, files = []) {
  // Get an array of all files and directories in the passed directory using fs.readdirSync
  const fileList = fs.readdirSync(dir);
  // Create the full path of the file/directory by concatenating the passed directory and file/directory name
  for (const file of fileList) {
    const name = `${dir}/${file}`;
    // Check if the current file/directory is a directory using fs.statSync
    if (fs.statSync(name).isDirectory()) {
      // If it is a directory, recursively call the getFiles function with the directory path and the files array
      getHTMLFiles(name, files);
    } else {
      // If it is an HTML file, push the full path to the files array
      if (name.endsWith("html")) {
        files.push(name);
      }
    }
  }
  return files;
}

criticalcss.js module

const fs = require("fs");
const Beasties = require("beasties");
const { join } = require("path");
const { parse } = require("node-html-parser");

async function criticalCSS() {
  const currentFolder = join(process.cwd(), "build");
  const files = getHTMLFiles(currentFolder);

  for (const file of files) {
      const critters = new Beasties({ path: currentFolder });
      const html = fs.readFileSync(file, "utf-8");
      const inlined = await critters.process(html);
      // additional step: delete links in the <head/> that left after critters
      const DOMAfterBeasties = parse(inlined);
      const head = DOMAfterBeasties.querySelector("head");

      for (const linkInHead of head.querySelectorAll("link")) {
        if (
          linkInHead.attributes?.as === "style" ||
          linkInHead.attributes?.rel === "stylesheet"
        ) {
          linkInHead.remove();
        }
      }

      fs.writeFileSync(file, DOMAfterBeasties.toString());
    }  
}

criticalCSS();

Note: Beasties should postpone your styles – move from <head/> to <body/> or wrap it with <noscript/> fallback. If there are still <link/> in <head/> you can add an additional step to handle that.

Critical CSS for Static Websites With CSS-in-JS

If you are using any CSS-in-JS library, you usually don't have a CSS file. No worries, we can create it during optimization, save it and add <link/> with this CSS to the html file.

Here are the basic steps:

Get all HTML files
Initialize Beasties with settings
Read html file
Parse HTML
Read existing styles from <style/> tags in <head/> and store each one in a Set.
Combine styles in one string and create a hash and path for the resulting styles
Save the CSS file
Process HTML with Beasties
Parse HTML after Beasties
Add style sheets to <body/> for lazy loading
Save HTML file

const fs = require("fs");
const Beasties = require("beasties");
const { parse } = require("node-html-parser");
const CryptoJS = require("crypto-js");
const { join } = require("path");
const { minify } = require("csso");

async function criticalCSS() {
  const currentFolder = join(process.cwd(), "build");
  const files = getHTMLFiles(currentFolder);

  for (const file of files) {
    const critters = new Beasties({ path: currentFolder });
    const html = fs.readFileSync(file, "utf-8");
    const DOMBeforeBeasties = parse(html);
    const uniqueImportantStyles = new Set();

    // first find all inline styles and add them to Set
    for (const style of DOMBeforeBeasties.querySelectorAll("style")) {
      uniqueImportantStyles.add(style.innerHTML);
    }

    const inlined = await critters.process(html);
    const importantCSS = Array.from(uniqueImportantStyles).join("");
    const DOMAfterBeasties = parse(inlined);
    const body = DOMAfterBeasties.querySelector("body");

    // if there was inline styles before Beasties
    if (importantCSS.length > 0) {
      const hash = CryptoJS.MD5(CryptoJS.enc.Latin1.parse(importantCSS));
      const inlinedStylesPath = `/static/css/styles.${hash}.css`;

      fs.writeFileSync(
        join(currentFolder, inlinedStylesPath),
        // minification is optional here if you have gzip enabled
        minify(importantCSS).css
      );

      if (body) {
        // we should add <link/> with stylesheet
        body.insertAdjacentHTML(
          "beforeend",
          `<link rel="stylesheet" href="${inlinedStylesPath}" />`
        );
      }
    }

    fs.writeFileSync(file, DOMAfterBeasties.toString());
  }
}

criticalCSS();

Fully Static With Both: Regular CSS and CSS-in-JS

We can also imagine a case where we use CSS at the same time as CSS-in-JS. Doubtful, but ok.

Here are the basic steps:

Get all HTML files
Initialize Beasties with settings
Read html file
Parse HTML
Read existing styles from <style/> tags in <head/> and store each one in a Set.
Process HTML with Beasties
Parse HTML after Beasties
Get all <link/> stylesheets from all HTML, keep the href and remove the <link/>.
Read all the styles from the stylesheets, pass them through Set and combine them into one
Combine the styles from step 5 with the file from the previous step
Create a hash and path for the resulting styles
Save the CSS file
Add style sheets to <body/> for lazy loading
Save HTML file

const Beasties = require("beasties");
const { join } = require("path");
const fs = require("fs");
const { parse } = require("node-html-parser");
const CryptoJS = require("crypto-js");
const { minify } = require("csso");

async function criticalCSS() {
  const currentFolder = join(process.cwd(), ".next");
  const files = getHTMLFiles(currentFolder);

  const beasties = new Beasties({
    path: currentFolder,
    fonts: true, // inline critical font rules (may be better for performance)
  });

  for (const file of files) {
    try {
      const html = fs.readFileSync(file, "utf-8");
      const DOMBeforeBeasties = parse(html);
      const uniqueImportantStyles = new Set();

      // first find all inline styles and add them to Set
      for (const style of DOMBeforeBeasties.querySelectorAll("style")) {
        uniqueImportantStyles.add(style.innerHTML);
      }

      const pathPatterns = {
        real: "/static/css",
        original: "/_next/static/css",
      };

      const changedToRealPath = html.replaceAll(
        pathPatterns.original,
        pathPatterns.real
      );

      const inlined = await beasties.process(changedToRealPath);
      const DOMAfterBeasties = parse(inlined);

      // merge all styles form existing <style/> tags into one string
      const importantCSS = Array.from(uniqueImportantStyles).join("");
      const body = DOMAfterBeasties.querySelector("body");

      if (importantCSS.length > 0) {
        const attachedStylesheets = new Set();
        const stylesheets = [];

        // find all <link/> tags with styles, get href from them and remove them from HTML
        for (const link of DOMAfterBeasties.querySelectorAll("link")) {
          if (
            link.attributes?.as === "style" ||
            link.attributes?.rel === "stylesheet"
          ) {
            attachedStylesheets.add(link.getAttribute("href"));

            link.remove();
          }
        }

        // go through found stylesheets: read file with CSS and push CSS string to stylesheets array
        for (const stylesheet of Array.from(attachedStylesheets)) {
          const stylesheetStyles = fs.readFileSync(
            join(currentFolder, stylesheet)
          );

          stylesheets.push(stylesheetStyles);
        }

        // Merge all stylesheets in one, add importantCSS in the end to persist specificity
        const allInOne = stylesheets.join("") + importantCSS;
        // using the hash, we will only create a new file if a file with that content does not exist
        const hash = CryptoJS.MD5(CryptoJS.enc.Latin1.parse(allInOne));
        const inlinedStylesPath = `/static/css/styles.${hash}.css`;

        fs.writeFileSync(
          join(currentFolder, inlinedStylesPath),
          // minification is optional here, it doesn't affect performance -- it is a lazy loaded CSS stylesheet, it only affects payload
          minify(allInOne).css
        );

        if (body) {
          body.insertAdjacentHTML(
            "beforeend",
            `<link rel="stylesheet" href="/_next${inlinedStylesPath}" />`
          );
        }
      }

      fs.writeFileSync(file, DOMAfterBeasties.toString());
    } catch (error) {
      console.log(error);
    }
  }
}

criticalCSS();

Essentially, this is a workaround for preserving the specificity of applied styles in cases where both inline styles and regular CSS are used.

NextJS blog boilerplate with critical css (styled-components and regular css)

NextJS Pages Router Critical CSS

To start using critters with NextJS, you need to install it and add one line to next.config.js

const nextConfig = { experimental: { optimizeCss: true } };
module.exports = nextConfig;

We will get both: build time and runtime optimization. The potential downside here is that the critters will run on every request for SSR pages. Also it is not working with the App folder.

To get an idea of what you can get, check out some of the metrics below. Please note that these tests are synthetic and not for the actual production page, but for the slightly modified default nextJS page. But either way, I hope you'll get some thoughts on the usefulness (or otherwise) of critical CSS.

We won't get a performance boost for the default NextJS page with a logo because loading the image will take much longer than loading the CSS, but if we replace the logo with text, the CSS will slow down the LCP.

A custom page where the logo is replaced with text.

Red and Orange – without Critters for desktop and mobile devices respectively. Blue and Green – with Critters for desktop and mobile respectively. The X axis is the test number, starting from zero, the Y axis is the time in milliseconds. The higher, the worse.

Slow 3G

Fast 3G

Slow 4G

As you can see, on slow connections you can potentially improve performance with critical CSS.

Effective caching can help reduce the frequency of critters launches. But if we don't have control over the critters itself, we will need to cache the entire page. Another possible solution would be to use a custom server.

Critical CSS for Dynamic Pages (SSR) in NextJS Pages Router with Custom Server

Note: Because dynamic pages fetch content on request, the page layout can potentially change. If at some point we extracted the critical CSS and then the layout changed - a new element appeared - then we have a potential problem. In such cases it is no longer safe to use these styles as it can cause the layout to shift and flash of unstyled content. One possible solution would be to cache the entire page or styles for a short period of time - for example, 5 minutes.

We will look at the case where we have both static and dynamic pages. In this case we will have runtime optimization and possible additional build time optimization. Optimization at the build step can be one of those described above.

Note: custom server will not work on Vercel.

First, I'll describe the idea in simple terms: on every first visit to the dynamic route, we fallback to the default styles and run critical CSS optimization in the background. The next time the user visits the same dynamic route, we will inline the critical CSS and load the stylesheets asynchronously. The build step, if it exists, will process static pages and also create a file with the processed pages/routes to ignore at runtime. There are no special requirements for optimization at the build step other than those listed above.

However, there is a lot of overhead at runtime: it is better to use a special _document.js file to associate the css with the routes, rather than importing it where it is needed. Why? Nextjs has its own handling of CSS files, and once bound to a page/route, the CSS is difficult to detach. So it's better to inject CSS dynamically, and _document.js is the ideal place since it runs on the server. We will need to add links with stylesheets depending on the route. To serve static CSS files from the NextJS build folder, we will need to copy them to a static folder during the build phase.

Now in detail.

To mark routes that have already been processed, we can collect the routes during the build phase and save them to a file processedRoutes.json.

['/','/about', ...]

beasties.js module looks like this.

const Beasties = require("beasties");
const { join } = require("path");
const fs = require("fs");
const { parse } = require("node-html-parser");

// Recursive function to get files
function getHTMLFiles(dir, files = []) {
  // Get an array of all files and directories in the passed directory using fs.readdirSync
  const fileList = fs.readdirSync(dir);
  // Create the full path of the file/directory by concatenating the passed directory and file/directory name
  for (const file of fileList) {
    const name = `${dir}/${file}`;
    // Check if the current file/directory is a directory using fs.statSync
    if (fs.statSync(name).isDirectory()) {
      // If it is a directory, recursively call the getFiles function with the directory path and the files array
      getHTMLFiles(name, files);
    } else {
      // If it is an HTML file, push the full path to the files array
      if (name.endsWith("html")) {
        files.push(name);
      }
    }
  }

  return files;
}

async function processHTMLFile(file, htmlString, runtime) {
  try {
    const beasties = new Beasties();
    // we don't read file at runtime
    const html = htmlString || (file && fs.readFileSync(file, "utf-8"));

    // nextJS paths
    const pathPatterns = {
      real: "/.next/static/css",
      original: "/_next/static/css",
    };

    const changedToRealPath = html.replaceAll(
      pathPatterns.original,
      pathPatterns.real
    );

    const inlined = await beasties.process(changedToRealPath);

    const restoredNextJSPath = inlined.replaceAll(
      pathPatterns.real,
      pathPatterns.original
    );

    const DOMAfterBeasties = parse(restoredNextJSPath);
    const head = DOMAfterBeasties.querySelector("head");

    if (head) {
      // delete links in the <head/> that left after beasties
      for (const linkInHead of head.querySelectorAll("link")) {
        if (
          linkInHead.attributes?.as === "style" ||
          linkInHead.attributes?.rel === "stylesheet"
        ) {
          linkInHead.remove();
        }
      }
    }

    // save HTML file in runtime, only for ISR https://nextjs.org/docs/pages/building-your-application/data-fetching/incremental-static-regeneration
    if (runtime === "ISR") {
      const filePath = join(
        process.cwd(),
        ".next",
        "server",
        "pages",
        file + ".html"
      );

      fs.writeFile(filePath, DOMAfterBeasties.toString(), (err) => {
        if (err) {
          console.error("Error saving the HTML file:", err);
        } else {
          console.log("The HTML file has been saved: ", filePath);
        }
      });
      // we don't save file in SSR
    } else if (runtime !== "SSR") {
      fs.writeFileSync(file, DOMAfterBeasties.toString());
    }

    const inlinedStyles = DOMAfterBeasties.querySelector("style");

    // return critical css
    return inlinedStyles.text;
  } catch (error) {}
}

async function main() {
  const currentFolder = join(process.cwd(), ".next");
  const files = getHTMLFiles(currentFolder);
  const processedRoutes = [];

  for (const file of files) {
    const slug = file.split(".next/server/pages")[1];

    await processHTMLFile(file);

    processedRoutes.push(slug.replace(".html", "").replace("index", ""));
  }

  fs.writeFileSync(
    join(process.cwd(), "processedRoutes.json"),
    JSON.stringify(processedRoutes)
  );
}

module.exports = { processHTMLFile };

if (process.env.BEASTIES_BUILD) {
  console.time("Beasties: build job");
  main();
  console.timeEnd("Beasties: build job");
}

We need to place the server.js file at the same level as next.config.js.

const { createServer } = require("http");
const { parse } = require("url");
const next = require("next");
const fs = require("fs");
const zlib = require("zlib");
const { join } = require("path");
const { processHTMLFile } = require("./beasties");

const dev = process.env.NODE_ENV !== "production";
const hostname = dev ? "localhost" : "example.com";
const port = 3000;
const app = next({ dev, port, hostname });
const handle = app.getRequestHandler();
const DIR = "beasties";
let processedRoutes = new Set();
const routes = {};
const cachingTime = 5 * 60 * 1000; // 5 min

try {
  console.time("Beasties: runtime prepare");

  fs.rmSync(DIR, { recursive: true, force: true });

  fs.cpSync("pages", DIR, {
    recursive: true,
    overwrite: true,
    filter: function (source) {
      if (source.includes(".")) {
        return false;
      }

      return true;
    },
  });

  const processedHTMLFiles = fs.readFileSync(
    join(process.cwd(), "processedRoutes.json"),
    "utf-8"
  );

  JSON.parse(processedHTMLFiles).forEach((file) => processedRoutes.add(file));

  console.timeEnd("Beasties: runtime prepare");
} catch (error) {}

async function saveStylesToFile(html, path) {
  const folder = DIR + path;
  const styles = await processHTMLFile(path, html, "SSR");

  fs.mkdirSync(folder, { recursive: true });

  const filePath = join(folder, "styles.css");

  fs.writeFile(filePath, styles, (err) => {
    if (err) {
      console.error("Error saving styles to file:", err);
    } else {
      console.log("styles saved to file:", filePath);
    }
  });
  console.timeEnd("Beasties: runtime");
}

app.prepare().then(() => {
  createServer((req, res) => {
    const parsedUrl = parse(req.url, true);
    const pathname = parsedUrl.pathname;

    if (
      !processedRoutes.has(pathname) ||
      Date.now() - routes[pathname] > cachingTime
    ) {
      const originalWrite = res.write;
      const chunks = [];

      res.write = function (chunk) {
        if (
          res.statusCode === 200 &&
          res.getHeader("content-type")?.includes("text/html")
        ) {
          chunks.push(chunk);
        }

        originalWrite.apply(res, arguments);
      };

      res.on("finish", () => {
        if (
          res.statusCode === 200 &&
          res.getHeader("content-type")?.includes("text/html")
        ) {
          processedRoutes.add(pathname);

          setTimeout(() => {
            console.time("Beasties: runtime");

            const html = Buffer.concat(chunks);

            zlib.unzip(html, (err, decompressedData) => {
              if (err) {
                console.error("Error decompressing data:", err);
                return;
              }

              saveStylesToFile(decompressedData.toString(), pathname);

              routes[pathname] = Date.now();
            });
          }, 0);
        }
      });
    }

    handle(req, res, parsedUrl);
  }).listen(3000, (err) => {
    if (err) throw err;
    console.log(`> Ready on http://localhost:${port}`);
  });
});

Also we need to modify build and start scripts in package.json.

  "scripts": {
   "build": "next build && BEASTIES_BUILD=1 node beasties",
    "start": "BEASTIES_RUNTIME=1 NODE_ENV=production node server"
  }

_document.js file can look like this.

import React from "react";
import { Html, Head, Main, NextScript } from "next/document";
import { join } from "path";
import fs from "fs";

function getCSSPaths(page) {
  switch (true) {
    case page.startsWith("/dynamic"):
      return [
        join("/_next", "static", "css", "dynamic.css"),
        join("/_next", "static", "css", "global.css"),
      ];
    case page === "/":
      return [
        join("/_next", "static", "css", "home.css"),
        join("/_next", "static", "css", "global.css"),
      ];
    default:
      return [join("/_next", "static", "css", "global.css")];
  }
}

function getCriticalCSS(page) {
  const withoutQuery = page.split("?")[0];

  try {
    return (
      <style
        dangerouslySetInnerHTML={{
          __html: fs.readFileSync(
            join(process.cwd(), "beasties", withoutQuery, "styles.css"),
            "utf-8"
          ),
        }}
      />
    );
  } catch (error) {}

  return false;
}

export default function Document(props) {
  const criticalCSS = getCriticalCSS(props.dangerousAsPath);
  const isCriticalCSSMode = process.env.BEASTIES_RUNTIME && criticalCSS;

  return (
    <Html lang="en">
      <Head>
        {isCriticalCSSMode
          ? criticalCSS
          : getCSSPaths(props.dangerousAsPath).map((link) => (
              <link key={link} rel="stylesheet" href={link} />
            ))}
      </Head>
      <body>
        <Main />
        <NextScript />
        {isCriticalCSSMode &&
          getCSSPaths(props.dangerousAsPath).map((link) => (
            <link key={link} rel="stylesheet" href={link} />
          ))}
      </body>
    </Html>
  );
}

To copy styles to the NextJS static folder we need to install copy-webpack-plugin -- yarn add copy-webpack-plugin and add these lines to next.config.js.

const CopyWebpackPlugin = require('copy-webpack-plugin');
const config = {
    webpack: (config) => {
        config.plugins.push(
            new CopyWebpackPlugin({
                patterns: [
                patterns: [{ from: "styles", to: "static/css" }],
                ],
            }),
        );
        return config;
    },
};
module.exports = config;

NextJS runtime critical css optimization example repo

NextJS App Router Critical CSS

App Router now has an experimental inline CSS feature.

Critical CSS for Dynamic Pages (SSR) in NextJS App Router with Custom Server

NextJS App router runtime critical css optimization example repo

Conclusion

Even though critical css optimization comes with the cost it can greatly increase performance. But first, it's important to measure performance and see if you have performance issues with styles: whether they're actually blocking your metrics like FCP and LCP or not. So it might not be worth it if your LCP is an image and loads much later than the styles. In this case, you will not get a big improvement. On the other hand it doesn't hurt if you can overcome the extra overhead. It's not exactly low hanging fruit, but at least for static builds it's viable.

This approach is not limited to NextJS only. Looks like this critical CSS optimization is suitable for any frameworks that can generate HTML at build time.

Google Consent Mode v2

Sergey Labut — Thu, 11 Apr 2024 08:18:36 +0000

What is new in Google Consent Mode v2
How to set up Google Consent Mode
Summary

Why should this matter to you?

For those utilizing Google's offerings like Google Analytics, Google Ads Conversion Tracking, Google Ads Remarketing, and other marketing tools from Google, it's crucial to understand that data collection and processing regulations are tightening. Gaining explicit consent from users to gather and manage personal information is now a requirement. Adopting Google Consent Mode (GCM) represents a strategic approach to comply with these rules.

What is new in Google Consent Mode v2?

Google has updated its consent mode, introducing two new parameters. These parameters are to be used alongside ad_storage and analytics_storage
ad_user_data: sets consent for sending user data related to advertising to Google,
ad_personalization: sets consent for personalized advertising.

If you've been using Google services alongside Google Consent Mode, it's time for an update. Failing to integrate the new consent parameters could lead to your Google Ads and Google Remarketing ceasing to function.

How to set up Google Consent Mode?

There are several ways to implement Google Consent Mode: using gtag.js or Google Tag Manager (GTM), using basic or advanced mode, with or without Consent Management Platform (CMP). Gtag.js is only suitable for Google tags, not for all, while GTM provides a container suitable for any tags (eg gtag.js) where you can manage them all. Basic mode operates on an all-or-nothing principle, while Advanced mode offers more granular control.
Here is the visual difference between the modes.

Here is a visual representation of key implementation steps.

Now the same thing, but in text form and with additional information.
First of all you can set default values for consents. Regardless of the mode, gtag.js and GTM are almost the same.

<script>
  window.dataLayer = window.dataLayer || [];
  function gtag() { dataLayer.push(arguments); }
  gtag('consent', 'default', {
    'ad_user_data': 'denied',
    'ad_personalization': 'denied',
    'ad_storage': 'denied',
    'analytics_storage': 'denied',
    'wait_for_update': 500,
  });
 // for gtag.js
  gtag('js', new Date());
  gtag('config', 'Google tag ID');
 // for GTM 
  dataLayer.push({'gtm.start': new Date().getTime(), 'event': 'gtm.js'});
  </script>

Note: It's best not to overlook setting defaults. In the API, all of these options are set to "granted" but that's not what we usually want, we usually want them to be "denied".

Then a few words about modes. The basic mode works like a switch - accept all or deny all; only after consent is received, tags are loaded and data collection begins. To do this, you need to update all consents to "granted" when the user clicks "Accept" and then add a tag to the page. On the “Accept” button, add an onclick listener with the following function.

 localStorage.setItem('consentGranted', 'true');
  function gtag() {
    dataLayer.push(arguments);
  }
  gtag('consent', 'update', {
    ad_user_data: 'granted',
    ad_personalization: 'granted',
    ad_storage: 'granted',
    analytics_storage: 'granted',
  });

  // If you use gtag.js script
  var gtagScript = document.createElement('script');
  gtagScript.async = true;
  gtagScript.src = 'https://www.googletagmanager.com/gtag/js?id=Google tag ID';

  var firstScript = document.getElementsByTagName('script')[0];
  firstScript.parentNode.insertBefore(gtagScript,firstScript);

  // If you use Tag Manager script.
  var gtmScript = document.createElement('script');
  gtmScript.async = true;
  gtmScript.src = 'https://www.googletagmanager.com/gtm.js?id=Container ID';
  var firstScript = document.getElementsByTagName('script')[0];
  firstScript.parentNode.insertBefore(gtmScript, firstScript);

Advanced mode can collect some anonymous data – cookieless ping before consent and even on denial. It is assumed that you can update consents individually - each consent has its own control. For GTM it is recommended to use the Consent Mode Template (CMT) your own or from the gallery.

A few words about CMP. It is a software tool or service used by website owners to manage user consent preferences regarding the collection and processing of their personal data. So usually it has support for other services such as advertising networks, analytics platforms, and content management systems and has its own management system. Usually added separately to your codebase alongside GTM.

This is all? Looks simple

Not so fast.

If you use gtag.js, the configuration typically occurs in your website's codebase. When you integrate gtag.js into your website, you will include a JavaScript snippet provided by Google within the HTML of your web pages.

If you are using GTM, you can add a script to the page if special conditions are met. Google services are natively supported by GTM. If you are going to use third party scripts with GTM, you need to know about the Google Consent API. When setting up a tag, it will be checked to see if it uses the Google Consent API. If yes, a built-in consent setting will appear. If not, then you need to configure it yourself. And you can add additional consent requirements even if they are already built in.

Summary

Modern Internet users are demanding that technology companies respect their personal data, and governments are ready to help with this by publishing regulations. And the current trend is that the rules are becoming more stringent. Modern websites must first obtain users' permission before collecting and processing personal data, and consent mode is the way to do this. While GTM is a default option that provides Google’s consent mode and many other features, it is not the only option.

Deep Dive into Web Performance: Mastering LCP Optimization for SEO Success

Sergey Labut — Mon, 18 Mar 2024 10:11:04 +0000

In this article, we'll talk about performance and look at LCP optimization.

LCP breakdown
Ways to improve
Article Recap

Who is this article for: If you're just curious or think you have performance issues (in which case we need hard numbers first - real user data is best here.Google, hosting companies and online services provide tools known as real user monitoring (RUM) that allow you to collect and view user data such as performance. If you don't collect it you can use for example webpagetest or pagespeed)

LCP breakdown

What is it?
LCP stands for Largest Contentful Paint; it is one of the Core Web Vitals (CWV) metrics that is used to evaluate the performance of a web page. In short, it is a measure of how quickly you can deliver and show content to users. LCP score directly affects SEO: the better the LCP, the better the SEO. Please do not confuse it with FCP (First Contentful Paint) - the two mean different things: FCP indicates the first appearance of content (not necessarily main content) on the screen and has only one entry, whereas LCP measures until the end of loading (more precisely, before the first user interaction) and can have multiple entries. since larger elements may appear in the viewport while the page is loading, but only the last – the largest element will be taken as the metric value. But sometimes they can point to the same element.
Let's see some examples for better understanding. Regardless of special tools, we can extract performance metrics manually right in a browser console during a session.

  new PerformanceObserver((list) => console.log(list.getEntries())).observe({
    type: "paint",
    buffered: true,
  });

  new PerformanceObserver((list) => console.log(list.getEntries())).observe({
    type: "largest-contentful-paint",
    buffered: true,
  });

For example, the FCP and LCP of this web page have the same time. We also only have one entry for LCP. This means that FCP corresponds to LCP.

But it was an ideal scenario, usually we have a different picture. There are several LCP entries on this web page, none of which correspond to Larger items appearing during loading until the largest one appears, which will be the LCP.

According to httparchive, a significant number of websites have problems with LCP, so this may be a sign that getting a good score on LCP may not be easy.

The main reason for this is the LCP complexity: we need to think about content and speed at the same time. Even the default text element isn't necessarily fast, let alone images. What counts as content? – Since this is a user-centric metric we are interested in contentful blocks of information – only visible elements can be candidates for LCP. The contentful element is the largest and most significant piece of content within the user's viewport. This can be an image, a block of text, a video. Speed corresponds to time here – the time it takes to load and display an element to the user. Moving from the last event to the first: before rendering, we need to load a resource for the element - an image or a web font; to load the resource first we need to detect the element and its resource url and before that we need to load the HTML document first. Google defined these stages as follows: now from the first event to the last:

TTFB is the time to first byte. This is a measure of how quickly your server can send an HTML document to a user upon request,
Load delay is a measure of the time between the TTFB and the start of loading of the LCP element,
Load time the time it takes to load resource,
Render delay time is the time between states when the browser has everything to render your LCP, but has not yet rendered it.

Some details about which element can be an LCP candidate.
LCP has its own specification, according to which it can be image or text. No restrictions for text. An image can be provided by the following tags: <img/>, <video/>, <image/> within a <svg/>, but <svg/> itself can't be an LCP. A CSS pattern will not be treated as LCP, but a background with an image provided in the url() will. For animated images (GIFs) or videos, only the first frame (thumbnail) will be considered as a candidate for LCP. If an image fills the entire viewport, it will not be considered an LCP, but rather a background.

There are some limitations in measuring LCP.

Ways to improve

Should be noted that to improve performance you need to prioritize the resources that are critical to rendering everything in the user's viewport and put everything else aside.
It's useful to know the browser's critical rendering path (CRP).

What we will improve is the timings, but we also need to consider both text and images.

We will follow Google's timing model for LCP. The sum of all timings must be less than 2.5 seconds to be considered good. But each part has completely different reasons. Moreover, sometimes when you reduce one part, you simply increase the adjacent part to the same amount, leaving the overall time the same. So to make real improvement you need to improve all timings – each timing should be as short as possible.

TTFB. This performance metric refers to the point at which the user begins receiving data. Therefore, we must send data as soon as possible after user interaction. Each additional step will potentially increase this time. It can sometimes be difficult to improve and this metric deserves its own article, but we can limit ourselves to the following tips:

avoid redirects – they can impact TTFB because they introduce additional steps in the process of fulfilling a user's request for a web page,
optimize server response times by using efficient hosting, CDNs, and caching mechanisms.
if you use server side rendering (SSR) ensure it is fast, switch to static site generation (SSG) if possible, some frameworks support streaming from the server - which allows you to start sending HTML even if it not completely ready.

What is good: 0.8 seconds or less

Load delay. This is the time between TTFB and the start of loading. To start loading, the browser needs to scan (preload scanner) the page to understand what resources it needs to render in order to start loading them. And if you don't give the browser any hint, it will only find those resources when it encounters an img element or text node with a required web font. Sometimes it's a significant portion of the time. To reduce this time, you can add <link/> tags in the <head> for the web font and images that could potentially be LCP.
Let's consider fonts first. For Google fonts it could be something like this

<link rel="preload" href="https://fonts.googleapis.com/css?family=Roboto&display=swap" as="style"crossorigin="anonymous"/>
 <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto&display=swap"crossorigin="anonymous"/>

Since it's a separate https connection, serving those fonts from your server may be faster.

<link rel="preload" href="/fonts/Roboto/RobotoBold.woff2" as="font" type="font/woff2">
<link rel="stylesheet" href="styles/fonts.css"/>

But I suggest testing both approaches to see which is the fastest in your case.

For images it looks very similar.

<link rel="preload" as="image" href="https://imagecdn.com/some-image.webp">

The background image cannot be preloaded this way but you can embed the image in CSS using the dataURI and then inline styles in the <head/>. So instead of using an image link.

.cool-background {
background: url("image.png");
}

You can insert an encoded image string into the url.

.cool-background {
background: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAABmCAMAAADS3TdOAAACTFBMVEX/////TkIqef8Jzmv/pABmZmZDQ0MAAAD/89z/5+X/7c7d+OvQ9uL/3dpFiP+Ktf+ewv9+5bJra2t0vD36+voMBgGsyv/d3d2kpKT39/dtbW0EAwHEfgJ5eXkjFgPcjgL....")
}

There are a few pitfalls to this approach: the base64 string will weigh more in KB than the original image, not to mention increasing the overall weight of the page, and the browser will also waste more main thread resources decoding the string into the actual image and finally the image won't be cached.

If you are using svg, then instead of using <img/> with a src link to the svg file, you can include the svg in the html. So instead of

<img src="logo.svg" />

You can do something like this

<svg xmlns="http://www.w3.org/2000/svg" width="400" height="400" viewBox="0 0 124 124" fill="none">
<rect width="124" height="124" rx="24" fill="#F97316"/>
<path d="M19.375 36.7818V100.625C19.375 102.834 21.1659 104.625 23.375 104.625H87.2181C90.7818 104.625 92.5664 100.316 90.0466 97.7966L26.2034 33.9534C23.6836 31.4336 19.375 33.2182 19.375 36.7818Z" fill="white"/>
<circle cx="63.2109" cy="37.5391" r="18.1641" fill="black"/>
<rect opacity="0.4" x="81.1328" y="80.7198" width="17.5687" height="17.3876" rx="4" transform="rotate(-45 81.1328 80.7198)" fill="#FDBA74"/>
</svg>

Some general tips:

<link/> can also have fetchpriority attribute with ‘high’, ‘low’, ‘auto’(default) values.
By using this attribute, you can more precisely control resource loading.
Setting it to 'high' means the browser will give it priority over others, setting it to 'low' will reduce the priority of the resource, but the browser can still increase the priority if the browser needs it.

Even though you can use this attribute directly in the <img/> tag, it's better to add it to the head so that the browser can detect it earlier. Additionally, it is not widely supported yet – around 78% global.

<img src="image.jpg" fetchpriority="high">

Note: You can't just mark everything with fetchpriority="high". If everything is "high" priority, then nothing is "high" priority – important resources will be loaded among other resources in a large queue, and this can potentially lead to poor performance.

You also can try to preconnect to the hosting with required resources.

<link rel="preconnect" href="https://fonts.googleapis.com" crossorigin>

To get a feel for the numbers, let's look at an example.
I did two tests of the same page on different connections on webpagetest.

4G connection: 9000/9000 Kbps, latency 170ms, total connection time 353ms.

3G connection: 1600/768 Kbps, latency 300ms, total connection time 647ms.
The total connection time should give you an idea of the potential savings when using pre-connection optimization.

chrome.com improved Time To Interactive by almost 1 s by pre-connecting to important origins.

Note: It's important to understand that preconnect directives serve as optimization cues. The browser may not always act upon every directive, and it has the flexibility to adjust its approach, possibly opting for a partial handshake like falling back to DNS lookup alone or combining DNS with TCP for TLS connections

Best case = 0 – everything you need comes with an html document – LCP is a text node or image that is inlined.

Load time. We cannot speed up user connections, so we can only reduce the weight of resources. Here images optimization will work:

use lossless compression and resize images to reduce their file size without compromising quality (some modern framework can do it at a build step, also there are plenty of online services, CMSs usually have it is own image service),
use modern image formats like WebP that offer better compression efficiency (Typically the conversion to webp occurs during compression. WebP is widely supported, AVIF has even better compression and is widely supported too),
use browser caching to store static resources on the user's device, reducing the need to retrieve them on subsequent page loads (Cache-control header),
use a CDN for static assets to reduce latency and improve content delivery speed,
show images in different sizes depending on the size of the viewport — responsive images.

Let's talk about the latter in more detail. The main idea here is that you don't have to send a full-size image to a mobile device. Instead, you can specify multiple URLs depending on the viewport.

<img
  srcset="image-small.jpg 480w, image-big.jpg 800w"
  sizes="(max-width: 600px) 480px,800px" src="image-big.jpg" />

In this example, in srcset we list all image links by screen width (first goes image link then space and then width), the "w" here does not mean pixels, but a width descriptor (in short, to get the actual pixels – CSS pixels, the specified width is multiplied by the device pixel ratio (DPR)). In sizes you declare the rules on how images should be applied.

If you use the width descriptor, it is possible that even the mobile browser will select a large image due to the pixel ratio.

If your image has certain dimensions it is better to build the srcset depending on the pixel ratio.

<img
  srcset="image-small.jpg 1.5x, image-big.jpg 2x"  src="image-big.jpg" />

Here 1.5x and 2x are the device pixel ratios (DPR).

It should be noted that the majority of people do not see the difference between 2x and 3x, so there is no need to show larger images.

Even though modern frameworks provide us with an API to optimize images, we usually don't have the ability to declare rules based on the device pixel ratio. So to save bandwidth and improve performance, we can use <picture> with multiple sources to limit the image size based on DPR.

<picture>
// here should be rules for lower DPR
<source media="(-webkit-min-device-pixel-ratio: 3) and (-webkit-max-device-pixel-ratio: 3.9)" srcset="https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/427x0/filters:quality(75) 640w, https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/720x0/filters:quality(75) 1080w, https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/1280x0/filters:quality(75) 1920w">
// here should be rules for bigger DPR
<img alt="image" sizes="100vw" srcset="https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/640x0/filters:quality(75) 640w, https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/1920x0/filters:quality(75) 1920w" src="https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/3840x0/filters:quality(75)">
</picture>

The idea here is to show images no higher than 2x. This applies for DPRs above 2x.

Don’t forget to add links in the <head/>.

<link rel="preload" as="image" media="(-webkit-min-device-pixel-ratio: 3) and (-webkit-max-device-pixel-ratio: 3.9)" imagesrcset="https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/427x0/filters:quality(75) 640w, https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/720x0/filters:quality(75) 1080w, https://a.storyblok.com/f/167931/2812x1520/98fdb9db37/2812x1520px-5.jpg/m/1280x0/filters:quality(75) 1920w">
// same for other sources

For fonts, use modern formats like woff2, which are optimized for the web and widely supported.

Render delay. Something is interfering with rendering. Usually can be caused by one of the following:
Main thread is busy,
Last second DOM change,
Blocked by CSS

What it means is that the main thread is busy and doesn't have the resources to actually render. In this case, be sure to download and execute non-essential or third-party JS after loading the main document. Also the main thread may be busy decoding your images. All image decoding occurs on the main thread. Therefore, it is useful to have an idea of the image decoding speed.
Here's a comparison of decoding speeds for different image formats.
There is a "decoding" property for <img/> that has three possible values:

"sync": decode the image synchronously,
"async": decode the image asynchronously, allowing other content to be rendered before completion
"auto": no preference for decoding mode; the browser decides based on what's best for the user. Default values may vary across browsers.

For non-critical images it is better to use asynchronous decoding.

<img src="encoded-image.avig" decoding="async" />

By last second change I mean sudden changes to the DOM or styles that affect the LCP and therefore change the rendering latency. Adding an LCP element to the viewport: Dynamic DOM manipulation using JS or the element was initially hidden by CSS and then shown. In most cases, you can provide a completely finished HTML document, or at least with the main content. Check if you can do this server side/build time.

Can be blocked by CSS because stylesheets normally are blocking resources. You could try just inline all the CSS into the <head/> and see if that improves your FCP and LCP metrics. The best option here is to extract your critical CSS, inline it and postpone loads of other styles (a popular solution without a browser and a popular solution that uses a headless browser). Inlining CSS should noticeably improve your performance, especially if loading styles takes longer than loading the LCP - the LCP is ready and waiting for the styles.

Modern browsers can parse and render HTML by chunks. Browsers do not wait till full load to start parsing and rendering the document. This means you will not improve your LCP score by simply reducing overall document weight. Because you need to reduce the number of bytes/work before the LCP element, not after. By placing your LCP element at the top of your HTML document you can potentially decrease LCP timings.

Also should be noted that if the LCP element is not at the very top of the page – LCP elements can be different on different viewports (devices), depending on screen height and width.

Article Recap

In this article, we delved into web page performance, focusing on the intricacies of Largest Contentful Paint (LCP) optimization.

LCP's complexity arises from balancing content and speed considerations. Identifying contentful blocks within the viewport is crucial for LCP optimization.

Improving LCP involves prioritizing critical resources for rendering and optimizing load timings.

Despite the fact that the article is full of technical details, the topic itself is not fully covered. The article is longer than I expected, and adding more examples to it will only make it less readable. After reading this, you should have an idea of possible ways to improve your website's performance. Don't forget that performance is measurable and you should always run before and after tests to measure the impact of optimization.

Elevate Your Site's UX: Pro Tips on Minimizing Cumulative Layout Shift

Sergey Labut — Fri, 15 Dec 2023 10:58:08 +0000

In this article we will consider one of Core Web Vitals (CWV) – Cumulative Layout Shift (CLS). What is it? Why is it important? And how to improve it?

What is Cumulative Layout Shift
Why it is important
How to improve CLS
Recap

What is Cumulative Layout Shift

CLS was presented to a broad audience in Chrome Lighthouse by Google around 2020 as one of CWV metrics. After that, we'll learn more about what indicators Google uses to rank search results – CWV are measurable values used by Google to determine the overall performance of a site and this performance impacts search results. CLS specifically is a measure of layout stability. It reflects a largest (was sum previously) content shift – when an element in a viewport changes its starting position between two frames
To be more accurate CLS is a measure of the largest burst of layout shift scores.
A burst of layout shifts, known as a session window, occurs when one or more individual layout shifts happen rapidly, with intervals of less than 1 second between each shift, and within a total window duration of maximum 5 seconds.

Why it is important

Imagine you open a website and as soon as the first block of text appears you start reading it, but then the huge image above loads and pushes all the text down and you have to find the place where you read again. It's very annoying
Many users access the Internet from their smartphones and CLS for those users can be even more noticeable

Another classic example is the unintentional creation of orders.
When the user wants to click the return button but suddenly the advertisment above loads and pushes all the content down and the user clicks the create order button instead.

I am sure, I don't need to explain how bad this is.

Also it's important to mention that this is one of the three CWV, which means it has a big impact on SEO.

In any case, content jumps can lead to user disappointment in your site/product. According to the httparchive.org, most sites have good CLS - another reason to take care of it so as not to stand out for the worse.

Now you know that shift happens. Let's discuss the ways how to avoid it

How to improve CLS

To improve the metric we need to understand how it is measured. Essentially, this is multiplying the size of the content being moved by the distance it was moved:

layout shift score = impact fraction * distance fraction
In the example below, the largest viewport dimension is the height, and the unstable element has moved by 25% of the viewport height, which makes the distance fraction 0.25.
So, in this example the impact fraction is 0.75 and the distance fraction is 0.25, so the layout shift score is 0.75 * 0.25 = 0.1875.

Only unexpected shifts appear in the final score. By unexpected we understand that it was not requested by the user. CSS transform shifts are also ignored.

A good CLS is less than 0.1 but It's important to aim for lower values to enhance user experience, even if it falls under 0.1

Most of the time CLS happens during page load and Chrome Lighthouse can be a good tool to measure CLS in this case.

After the test you will receive some information on how to improve your performance.

Before we dive into this, let's introduce two new terms:

flash of invisible text (FOIT) - it is an issue where text remains invisible while webfont is loading – space for text is reserved, just empt;,
flash of unstyled text (FOUT) - it is an issue where text is changed because of a big difference of downloaded font and a fallback font, this usually affects the length of the text: the text may become larger or smaller and shifts the content on the page

But here are general causes of CLS:

Image/video suddenly downloaded and shifted all content down. It happens when the image/video has no dimensions: width and height. So browsers did not reserve a space for it.
Dynamic insertion in DOM – by JS script execution
FOIT
FOUT

Ways to reduce CLS:

In the case of images/videos you just need to make sure you added width and height attributes.
In case of dynamic DOM update – you should restrict the size of it and use placeholder maybe, also try to add it below the viewport.
In case of FOIT you should use font-display property The font-display property accepts five values: auto (default): The browser uses its default method for loading, which is most often similar to the block value. block: The browser will hide the text until the web font is fully loaded. But it is hidden in a very specific way - the text seems to be there, but it is not visible. Also known as Flash of invisible text or FOIT. swap: The browser will use local fallback font to display the text until the web font is fully downloaded. Can cause Flash of unstyled text or FOUT. fallback: The browser will hide the text for 100ms, then if the web font is not ready, uses a fallback font, and finally switches to the web font but swap period is less than 3 seconds. optional: The browser will use fallback font and will decide to switch or not to web font considering user's connection speed -- if connection is very slow it can even ignore web font Most of the time to eliminate FOIT you should use swap but swap can possibly lead to FOUT. We will handle it next Here is another chart from httparchive.org to show how many sites prevents FOIT by using font-display

In case of shifts due to font differences. You can change the local font (available by default for the user) to be very similar to your web font. For this example we are using this tool https://screenspan.net/fallback We will try to use Arial as fallback for Roboto regular We can adjust font properties manually and see a live preview.

After adjustments, we got a good match between the fonts.The same text, written in Arial and Roboto in different colors for clarity.

Here are the CSS rules we get

@font-face {
    font-family: 'Adjusted Arial Fallback';
    src: local(Arial);
    size-adjust: 98%;
    ascent-override: 96%;
    descent-override: normal;
    line-gap-override: normal;
}

Then, when listing suitable fonts, use the fallback for Roboto-Regular instead of the local font (Arial in our case).

h1 {
    font-family: 'Roboto-Regular', 'Adjusted Arial Fallback';
}

This should reduce the difference between fonts and reduce CLS or even eliminate it.

Recap

In this article we went through some tips on CLS performance. In conclusion we can note that CLS is a pretty important metric and by paying attention to it we can increase overall user experience on our site and get higher ranking in search results. It should be noted that CLS is measured during the lifespan of the page but it focuses on individual session windows within that lifespan, not the cumulative layout shift across the entire lifespan directly