Forem: David Peng

Test Svelte Component Using Vitest & Playwright

David Peng — Wed, 27 Jul 2022 15:09:42 +0000

Hi 👋, I'm David Peng.
It's been a while since my last blog post.

In the last two months, I added 100+ unit/ component/ e2e tests in my Svelte project (yeah, I didn't do TDD because I wasn't familiar with testing enough 😅).
I experimented with different test runners, kept measuring DX, and tried to find my ideal toolset for testing. I'd love to share some of my learnings and thoughts here.

In this article, I'll focus on the basic setup of Vitest & Playwright to test our Svelte components. You can check the demo repo here: Svelte Component Test Demo Repo

In my next blog post, I'll also write about the advanced component test and mocking (Svelte runtime module & networking). Stay tuned!

Different types of test

Here's a brief introduction of each type (my point of view):

Unit Test: Test the smallest piece of code, e.g., a simple function.
Integration Test: Test group of functions, e.g., fetching and transforming data
Component Test: Test the behavior/ transition of a UI component, e.g., form interaction, open a modal
End-to-end (E2E) test: Complete user journey, e.g., login to a website and a series of operations

Test Runners

Vitest: A Vite-native unit test framework. (alternative: Jest, uvu)

You might hear more Svelte Developers migrating their tests from Jest to Vitest in the past few months.

Vitest is way faster than Jest because of Vite goodness, and it's a great fit with SvelteKit (use vite under the hood). Another plus is you can remove all your jest/ babel config 🎊.

A great read here: Testing a Svelte app with Vitest

Playwright: A framework for Web Testing and Automation. (alternative: Cypress)

Playwright is a recommended E2E test runner in the Svelte community. You can also find Playwright as an option in create-svelte.

When to use each test runner?

Even though Vitest & Playwright are both test runners, they have a different focus and testing scenarios.

Vitest is incredibly fast because of its instant Hot Module Reload; it only reruns the test whenever the test, source code, or dependencies are changed, so it's suitable for the unit, integration tests.

While Playwright was created specifically for E2E testing, you need to build your Svelte app and start the server to test your app, which is much similar to the production.

But when it comes to the component test, we have a couple of choices:

Unit testing runners like Vitest, Jest, or uvu + Testing Library
Playwright/ Cypress component test (both powered by Vite)
Storybook (I haven't tried it 😁)

Unit Test	✨ Component Test ✨	E2E Test
Vitest	`vitest` + `@testing-library/svelte` or `@playwright/experimental-ct-svelte`	Playwright

Let's see how to test the Svelte component using Vitest & Playwright.

Since we're going to talk about the component test, I like to share this clip from JS Party, you can also listen to the episode #233
// Detect dark theme var iframe = document.getElementById('tweet-1551932536493953029-467'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1551932536493953029&theme=dark" }

Vitest + `@testing-library/svelte`

You can follow the below steps to setup Vitest or use the Svelte Adder: svelte-add-vitest

Take Svelte Demo App (TypeScript) for example

npm install -D vitest jsdom @testing-library/svelte
npm install -D @testing-library/jest-dom @types/testing-library__jest-dom (optional but recommended)
Configure Vitest in vite.config.js:

import { sveltekit } from '@sveltejs/kit/vite';

/** @type {import('vite').UserConfig} */
const config = {
  plugins: [sveltekit()],
  /** Add below settings */
  test: {
    // Jest like globals
    globals: true,
    environment: 'jsdom',
    include: ['src/**/*.{test,spec}.ts'],
    // Extend jest-dom matchers
    setupFiles: ['./setupTest.js']
  }
};

export default config;

Create setupTest.js and extend jest-dom assertions/ matchers in Vitest:

import matchers from '@testing-library/jest-dom/matchers';
import { expect, vi } from 'vitest';

expect.extend(matchers);

The reason to use jest-dom assertions can be found here: Common mistakes with React Testing Library #Using the wrong assertion

Add types in tsconfig.json:

{
  "compilerOptions": {
    "types": ["vitest/globals", "@testing-library/jest-dom"]
  }
}

Create src/lib/Counter.test.ts:

import { render, fireEvent, screen } from '@testing-library/svelte';
import Counter from './Counter.svelte';

describe('Test Counter.svelte', async () => {
  it('Initial counter should be 0', async () => {
    render(Counter);
    expect(screen.getByText('0')).toBeInTheDocument();
  });
  it('Test decrease', async () => {
    render(Counter);
    const decreaseButton = screen.getByLabelText('Decrease the counter by one');
    // Decrease by two
    await fireEvent.click(decreaseButton);
    await fireEvent.click(decreaseButton);
    // Wait for animation
    const counter = await screen.findByText('-2');
    expect(counter).toBeInTheDocument();
  });
  it('Test increase', async () => {
    render(Counter);
    const increaseButton = screen.getByLabelText('Increase the counter by one');
    // Increase by one
    await fireEvent.click(increaseButton);
    // Wait for animation
    const counter = await screen.findByText('1');
    expect(counter).toBeInTheDocument();
  });
});

The benefit of using screen is you no longer need to keep the render call destructure up-to-date as you add/remove the queries you need. You only need to type screen. And let your editor's magic autocomplete take care of the rest. (ref: Common mistakes with React Testing Library #Not using screen)

Add test script in package.json:

{
  "scripts": {
    "test": "vitest"
  }
}

Playwright Experimental Component Test

This experimental component test is powered by Vite. For more details, please read the doc and the overview video from Playwright:

Experimental: components

Take Svelte Demo App (TypeScript) for example

npm create svelte my-app and choose the Demo Project
npm install -D @playwright/experimental-ct-svelte
Create playwright-ct.config.ts

import type { PlaywrightTestConfig } from '@playwright/experimental-ct-svelte';
import { resolve } from 'node:path';

const config: PlaywrightTestConfig = {
  testDir: 'tests/component',
  use: {
    ctViteConfig: {
      resolve: {
        alias: {
          // Setup the built-in $lib alias in SvelteKit
          $lib: resolve('src/lib')
        }
      }
    }
  }
};

export default config;

Create several files in your Svelte project workspace:

playwright/index.html

<html lang="en">
  <body>
    <div id="root"></div>
    <script type="module" src="/playwright/index.js"></script>
  </body>
</html>

playwright/index.js

// Apply theme here, add anything your component needs at runtime here.
// Here, we import the demo project's css file
import '../src/app.css';

Create tests/component/Counter.test.ts

@playwright/experimental-ct-svelte wrap @playwright/test to provide an additional built-in component-testing specific fixture called mount

import { test, expect } from '@playwright/experimental-ct-svelte';
import Counter from '$lib/Counter.svelte';

test('Test Counter.svelte', async ({ mount }) => {
  const component = await mount(Counter);
  // Initial counter is "0"
  await expect(component).toContainText("0");
  // Decrease the counter
  await component.locator('[aria-label="Decrease the counter by one"]').dblclick();
  await expect(component).toContainText('-2');
  // Increase the counter
  await component.locator('[aria-label="Increase the counter by one"]').click();
  await expect(component).toContainText('-1');
});

Your VS Code may show an error like this:

We can solve this by adding include in your tsconfig.json:

{
  "include": ["tests/**/*.ts"]
}

Add test:com script in package.json

{
  "scripts": {
    "test:com": "playwright test -c playwright-ct.config.ts"
  }
}

Vitest (as test runner) + Playwright

I haven't tried this combination in my projects, but you can check the example from the Vitest GitHub repo: Use Playwright with Vitest as test runner

Wrapping Up

I've spent a considerable amount of time writing component tests using both Vitest & Playwright. Here are some of my thoughts and a reflection of DX (developer experience):

Vitest + Testing Library: Setting up your testing environment takes more steps and dependencies. It also took me a while to grasp the concept and best practices of Testing Library and use query/ assertions properly, but once you have more confidence, you'd have a pleasant DX.
Playwright Experimental Component Test: You can directly use most of Playwright's goodness like fixtures, network intercepting, in your component test, and its syntax are intuitive and pretty handy. While it's still in the experimental phase, getting more stable may take a while.

In my opinion, Vitest + Testing Library would probably be a better choice at this moment. Despite that Vitest hasn't hit 1.0, my personal experience of migrating from Jest is pretty smooth. Also, Testing Library is stable and used by many companies.

Thank you for your reading.
You can follow me on Twitter @davipon

Please leave your thoughts and experience below. Love to hear your feedback!

Resources

Discussion

sveltejs/kit: Vitest for unit testing #5285

Issues

Shim SvelteKit runtime import aliases / Importing $app/* fails #1485

Articles

Videos

Build a Svelte Component To Draw SVG On Hover

David Peng — Fri, 03 Jun 2022 07:01:51 +0000

In this post, I'll share my journey of building a reusable Svelte component to draw SVG on hover. Begin by trying to reduce boilerplate in my work project and publishing my first npm package.

It's great if you've already known below things:

Svelte Transition
- transition:draw
Svelte Component Composition
- let: directive
- --style-props
SVG animation in CSS

This is my npm package: Svelte Hover Draw SVG

Any feedbacks are welcomed!

Let's talk about motivation

In my work project, I have a Tab component with inline SVG in each tab:

I like to add SVG animation on hover, and the first that came to my mind was transition:draw in Svelte:

It works, but here are the problems:

Boilerplates like on:mouseenter, {#if cursorOn === 'tab'}, and transition:draw
SVG might have multiple <path>, line, rect, polyline, polygon. We need to add transition to each of them manually
We can't use it when the SVG is in a component like <SVGInComponent/> (draw can only be used on SVGElement or element which has getTotalLength() method.)

Can we DRY them up?

DRY = don't repeat yourself

I don't want to write boilerplate again and again if I just want this simple effect.
So I started to find a way to abstract the hovering event & effect.

First thing we need to know is the draw animation is actually a CSS transition:

// Please refer to https://svelte.dev/tutorial/custom-css-transitions
function draw(node, { delay = 0, speed, duration, easing: easing$1 = easing.cubicInOut } = {}) {
  let len = node.getTotalLength();
  const style = getComputedStyle(node);
  if (style.strokeLinecap !== 'butt') {
    len += parseInt(style.strokeWidth);
  }
  if (duration === undefined) {
    if (speed === undefined) {
      duration = 800;
    }
    else {
      duration = len / speed;
    }
  }
  else if (typeof duration === 'function') {
    duration = duration(len);
  }
  return {
    delay,
    duration,
    easing: easing$1,
    css: (t, u) => `stroke-dasharray: ${t * len} ${u * len}`
  };
}

This is the source code of draw, the key part is:

 css: (t, u) => `stroke-dasharray: ${t * len} ${u * len}

When the transition is triggered, a CSS animation with custom @keyframes would be added to that element:

So what we're trying to do is quite simple: toggling a CSS animation on hover!

A Trick That Makes Drawing SVG Lines Way Easier

I was looking for a way to simplify the complex @keyframes, and I found this article: A Trick That Makes Drawing SVG Lines Way Easier by Chris Coyier.

By setting the pathLength="1" to SVGElement (like <path>, line, rect, polyline, and polygon), set the stroke-dasharray to 1, and animate the offset in CSS!

<path class="path" d="M66.039,133.545 ... " pathLength="1" />

.path {
  stroke-dasharray: 1;
  stroke-dashoffset: 1;
  animation: dash 5s linear alternate infinite;
}

@keyframes dash {
  from {
    stroke-dashoffset: 1;
  }
  to {
    stroke-dashoffset: 0;
  }
}

I also found that I can put this path class to elements that have SVG inside, which means we don't need to add the animation to SVGElement programmatically:

<div class="tabs">
  <a href="#tab1" class="tab">
    <svg stroke-width="2">
      <!-- This works -->
      <path class="path" pathLength="1" d="..." />
    </svg>
    <span>Tab 1</span>
  </a>
  <!-- This also works -->
  <a href="#tab2" class="tab path">
    <svg stroke-width="2">
      <path pathLength="1" d="..."/>
    </svg>
    <span>Tab 2</span>
  </a>
</div>

Make a wrapper component for abstraction

The goal of creating a wrapper component:

Let the wrapper component handle the hover event & effect.
Trigger animation on hover (component level, not just SVG).
User can just put their components/ SVGs in the wrapper component.

Here is the final approach:

You can try it out here: Svelte REPL

Easy to use + less code to write FTW 💪

Inline SVG:

SVG Component (set draw time to 2s, default is 1s):

Nested elements (expose hovering status):

Wrapping up

There were some technical considerations I didn't mention in this article and finally gave up on, like designing a custom transition and using named slots to validate SVGElement, ...etc.

It'd be great to let users customize the transition using the native parameters like duration and easing, but it's overkill if this simple effect can be achieved by CSS.

That's all. Thank you for your reading! 🙌

You can find me on Twitter: @davipon

Better Backend DX: JSON Schema + TypeScript + Swagger = ✨ Vol. 2

David Peng — Fri, 20 May 2022 16:18:14 +0000

👋 Hi! I'm David Peng. You can find me on Twitter: @davipon.

This post is Vol. 2 of the Better Backend DX: JSON Schema + TypeScript + Swagger = ✨, and I'll cover following topics by building a simple Fastify CRUD Posts API:

Improve code readability & testability by separating options and handler of route method
Use of JSON Schema $ref keyword
Swagger UI and OpenAPI specification
Use Thunder Client (VS Code extension) to test APIs

Complete repo on GitHub: github.com/davipon/fastify-esbuild

Improve Code Readability & Testability

This is a general fastify shorthand route:

// src/routes/examples.ts
/*
  Route structure:
  fastify.get(path, [options], handler)
*/
fastify.get('/',
  {
    schema: {
      querystring: {
        name: { type: 'string' },
        excitement: { type: 'integer' }
      },
      response: {
        200: {
          type: 'object',
          properties: {
            hello: { type: 'string' }
          }
        }
      }
    }
  },
  (request, reply) {
    reply.send({ hello: 'world' })
  }
)

We can refactor it and break it into chunks with the notion of Separation of Concerns (SoC). It would be much easier for us to maintain and test our code.

// src/routes/examples/schema.ts
export const schema = {
  querystring: {
    name: { type: 'string' },
    excitement: { type: 'integer' }
  },
  response: {
    200: {
      type: 'object',
      properties: {
        hello: { type: 'string' }
      }
    }
  }
}

// src/routes/examples/handler.ts
export const handler = function (request, reply) {
  reply.send({ hello: 'world' })
}

// src/routes/examples/index.ts
import { schema } from './schema'
import { handler } from './handler'

...

fastify.get('/', { schema }, handler)

Since we're using TypeScript, we need to type schemas and handler functions.

Build a Simple Blog Post CRUD API

Here is the specification of our API:

GET
- '/posts': Return all posts
- '/posts?deleted=[boolean]'(querystring): Filter posts that are deleted or not
- '/posts/[postid]'(params): Find specific post
- Status code 200: Successful request
- Status code 404: Specific post not found
POST
- '/posts': Create a new post
- Status code 201: Create post successfully
PUT
- '/posts/[postid]'(params): Update specific post
- Status code 204: Update specific post successfully
- Status code 404: Specific post not found
DELETE
- '/posts/[postid]'(params): Delete specific post
- Status code 204: Delete specific post successfully
- Status code 404: Specific post not found

Recommend: Best Practices for Designing a Pragmatic RESTful API

First, create a sample data posts:

I plan to write about MongoDB's official driver & containerization in my next post, so I use a sample object as data here.

// src/routes/posts/posts.ts

// Sample data
export const posts = [
  {
    id: 1,
    title: 'Good Post!',
    published: true,
    content: 'This is a good post',
    tags: ['featured'],
    deleted: false
  },
  {
    id: 2,
    title: 'Better Post!',
    published: true,
    content: 'This is an even better post',
    tags: ['featured', 'popular'],
    deleted: false
  },
  {
    id: 3,
    title: 'Great Post!',
    published: true,
    content: 'This is a great post',
    tags: ['featured', 'popular', 'trending'],
    deleted: false
  }
]

Request & Response Schemas

Let's create JSON Schema for Params, Querystring, Body, Reply:

The shorthand route methods (i.e., .get) accept a generic object "RouteGenericInterface" containing five named properties: Body, Querystring, Params, Headers, and Reply.

// src/routes/posts/schema.ts

import { FastifySchema } from 'fastify'
import { FromSchema } from 'json-schema-to-ts'

// Params Schema
const paramsSchema = {
  type: 'object',
  require: ['postid'],
  properties: {
    postid: { type: 'number' }
  },
  additionalProperties: false
} as const

export type Params = FromSchema<typeof paramsSchema>

// Querystring Schema
const querystringSchema = {
  type: 'object',
  properties: {
    deleted: { type: 'boolean' }
  },
  additionalProperties: false
} as const

export type Querystring = FromSchema<typeof querystringSchema>

// Body Schema
export const bodySchema = {
  type: 'object',
  properties: {
    id: { type: 'number' },
    title: { type: 'string' },
    published: { type: 'boolean' },
    content: { type: 'string' },
    tags: { type: 'array', items: { type: 'string' } },
    deleted: { type: 'boolean' }
  },
  required: ['title', 'published', 'content', 'tags', 'deleted']
} as const

export type Body = FromSchema<typeof bodySchema>

// Reply Schema
const replySchema = {
  type: 'object',
  properties: {
    // Return array of "post" object
    posts: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          id: { type: 'number' },
          title: { type: 'string' },
          published: { type: 'boolean' },
          content: { type: 'string' },
          tags: { type: 'array', items: { type: 'string' } },
          deleted: { type: 'boolean' }
        },
        required: ['title', 'published', 'content', 'tags', 'deleted']
      }
    }
  },
  additionalProperties: false
} as const

export type Reply = FromSchema<typeof replySchema>

// ReplyNotFound Schema
export const postNotFoundSchema = {
  type: 'object',
  required: ['error'],
  properties: {
    error: { type: 'string' }
  },
  additionalProperties: false
} as const

export type ReplyNotFound = FromSchema<typeof postNotFoundSchema>

We also need to create a schema for each route method so @fastify/swagger can generate documents automatically. While before that, let's take a look at the above schemas.

You may notice a duplication in bodySchema and replySchema. We can reduce this by using the $ref keyword in JSON Schema.

JSON Schema `$ref` Keyword

Let's refactor the code and make it reusable:

// First create a general "post" schema

// Shared Schema
export const postSchema = {
  $id: 'post',
  type: 'object',
  properties: {
    id: { type: 'number' },
    title: { type: 'string' },
    published: { type: 'boolean' },
    content: { type: 'string' },
    tags: { type: 'array', items: { type: 'string' } },
    deleted: { type: 'boolean' }
  },
  required: ['title', 'published', 'content', 'tags', 'deleted']
} as const

// We don't need to create a separate "bodySchema".
// But directly infer type from postSchema
export type Body = FromSchema<typeof postSchema>

// Reply Schema
// Check https://www.fastify.io/docs/latest/Reference/Validation-and-Serialization/#adding-a-shared-schema
const replySchema = {
  type: 'object',
  properties: {
    posts: {
      type: 'array',
      items: { $ref: 'post#' }
    }
  },
  additionalProperties: false
} as const

// Check https://github.com/ThomasAribart/json-schema-to-ts#references
export type Reply = FromSchema<
  typeof replySchema,
  { references: [typeof postSchema] }
>

// Also make ReplyNotFound reusable for future use
export const postNotFoundSchema = {
  $id: 'postNotFound', // add $id here
  type: 'object',
  required: ['error'],
  properties: {
    error: { type: 'string' }
  },
  additionalProperties: false
} as const

export type PostNotFound = FromSchema<typeof postNotFoundSchema>

But to create a shared schema, we also need to add it to the Fastify instance.

// src/routes/posts/index.ts
import { type FastifyInstance } from 'fastify'
import { postSchema, postNotFoundSchema } from './schema'

export default async (fastify: FastifyInstance) => {
  fastify.addSchema(postSchema)
  fastify.addSchema(postNotFoundSchema)
  // shorthand route method will add later
}

Route Schemas

Route schemas are composed of request, response schemas, and extra property so that @fastify/swagger can automatically generate OpenAPI spec & Swagger UI!

Let's create route schemas based on our specifications:

// src/routes/posts/schema.ts

// Add route schemas right after request & respoonse schemas

/* Get */
export const getPostsSchema: FastifySchema = {
  // Routes with same tags will be grouped in Swagger UI
  tags: ['Posts'],
  description: 'Get posts',
  querystring: querystringSchema,
  response: {
    200: {
      // Return array of post
      ...replySchema
    }
  }
}

export const getOnePostSchema: FastifySchema = {
  tags: ['Posts'],
  description: 'Get a post by id',
  params: paramsSchema,
  response: {
    200: {
      ...replySchema
    },
    404: {
      description: 'The post was not found',
      // refer to postNotFound whenever a route use params
      $ref: 'postNotFound#'
    }
  }
}

/* Post */
export const postPostsSchema: FastifySchema = {
  tags: ['Posts'],
  description: 'Create a new post',
  body: postSchema,
  response: {
    201: {
      description: 'The post was created',
      // include a Location header that points to the URL of the new resource
      headers: {
        Location: {
          type: 'string',
          description: 'URL of the new resource'
        }
      },
      // Return newly created resource as the body of the response
      ...postSchema
    }
  }
}

/* Put */
export const putPostsSchema: FastifySchema = {
  tags: ['Posts'],
  description: 'Update a post',
  params: paramsSchema,
  body: postSchema,
  response: {
    204: {
      description: 'The post was updated',
      type: 'null'
    },
    404: {
      description: 'The post was not found',
      $ref: 'postNotFound#'
    }
  }
}

/* Delete */
export const deletePostsSchema: FastifySchema = {
  tags: ['Posts'],
  description: 'Delete a post',
  params: paramsSchema,
  response: {
    204: {
      description: 'The post was deleted',
      type: 'null'
    },
    404: {
      description: 'The post was not found',
      $ref: 'postNotFound#'
    }
  }
}

Now we have created schemas. Let's work on handler functions.

Handler Functions

The key in a separate handler.ts is the TYPE.

Since we no longer write the handler function in a fastify route method, we need to type the request and response explicitly.

// src/routes/posts/handler.ts

import { type RouteHandler } from 'fastify'
import {
  type Params,
  type Querystring,
  type Body,
  type Reply,
  type PostNotFound
} from './schema'
import { posts } from './posts'

/*
  We can easily type req & reply by assigning inferred types from schemas to
  Body, Querystring, Params, Headers, and Reply
  👆 properties of RouteGenericInterface
*/

export const getPostsHandler: RouteHandler<{
  Querystring: Querystring
  Reply: Reply
}> = async function (req, reply) {
  const { deleted } = req.query
  if (deleted !== undefined) {
    const filteredPosts = posts.filter((post) => post.deleted === deleted)
    reply.send({ posts: filteredPosts })
  } else reply.send({ posts })
}

export const getOnePostHandler: RouteHandler<{
  Params: Params
  Reply: Reply | PostNotFound
}> = async function (req, reply) {
  const { postid } = req.params
  const post = posts.find((p) => p.id == postid)
  if (post) reply.send({ posts: [post] })
  else reply.code(404).send({ error: 'Post not found' })
}

export const postPostsHandler: RouteHandler<{
  Body: Body
  Reply: Body
}> = async function (req, reply) {
  const newPostID = posts.length + 1
  const newPost = {
    id: newPostID,
    ...req.body
  }
  posts.push(newPost)
  console.log(posts)
  reply.code(201).header('Location', `/posts/${newPostID}`).send(newPost)
}

export const putPostsHandler: RouteHandler<{
  Params: Params
  Body: Body
  Reply: PostNotFound
}> = async function (req, reply) {
  const { postid } = req.params
  const post = posts.find((p) => p.id == postid)
  if (post) {
    post.title = req.body.title
    post.content = req.body.content
    post.tags = req.body.tags
    reply.code(204)
  } else {
    reply.code(404).send({ error: 'Post not found' })
  }
}

export const deletePostsHandler: RouteHandler<{
  Params: Params
  Reply: PostNotFound
}> = async function (req, reply) {
  const { postid } = req.params
  const post = posts.find((p) => p.id == postid)
  if (post) {
    post.deleted = true
    reply.code(204)
  } else {
    reply.code(404).send({ error: 'Post not found' })
  }
}

Fully typed req and reply can boost our productivity with real-time type checking and code completion in VS Code. 🥳

OK, let's finish the last part: fastify route method.

Fastify Route Method

Since we'd finished schema.ts and handler.ts, it's pretty easy to put them together:

// src/routes/posts/index.ts

import { type FastifyInstance } from 'fastify'
import {
  postSchema,
  postNotFoundSchema,
  getPostsSchema,
  getOnePostSchema,
  postPostsSchema,
  putPostsSchema,
  deletePostsSchema
} from './schema'
import {
  getPostsHandler,
  getOnePostHandler,
  postPostsHandler,
  putPostsHandler,
  deletePostsHandler
} from './handler'

export default async (fastify: FastifyInstance) => {
  // Add schema so they can be shared and referred
  fastify.addSchema(postSchema)
  fastify.addSchema(postNotFoundSchema)
  fastify.get('/', { schema: getPostsSchema }, getPostsHandler)
  fastify.get('/:postid', { schema: getOnePostSchema }, getOnePostHandler)
  fastify.post('/', { schema: postPostsSchema }, postPostsHandler)
  fastify.put('/:postid', { schema: putPostsSchema }, putPostsHandler)
  fastify.delete('/:postid', { schema: deletePostsSchema }, deletePostsHandler)
}

Now your folder structure should look like this:

Swagger UI & OpenAPI Specification

Please check how to set up @fastify/swagger in my last post.

After you start the dev server, go to 127.0.0.1:3000/documentation and you'll see the Swagger UI:

URL	Description
'/documentation/json'	The JSON object representing the API
'/documentation/yaml'	The YAML object representing the API
'/documentation/'	The swagger UI
'/documentation/*'	External files that you may use in $ref

Test API Using Thunder Client

Thunder Client is my go-to extension in VS Code for API testing.

I've exported the test suite to thunder-collection_CRUD demo.json. You can find it at my repo root folder and import it into your VS Code:

Let's test our API:

🙌 Wrapping up

Thank you for your reading!

In the 2nd part of the Better Backend DX series, we learned the goodness of using JSON Schema to validate routes and serialize outputs in Fastify.

By using json-schema-to-ts, we no longer need to type twice if we use TypeScript, and we also increase our productivity thanks to type checking and code completion in VS Code. Shorter feedback loop for the win! 💪

Since we'd declared route schemas, we can automatically generate Swagger UI & OpenAPI specifications by leveraging @fastify/swagger. Don't forget that good API documentation can improve your co-workers and end consumers' DX.

Kindly leave your thoughts below, and I'll see you in the next one. 👋

Better Backend DX: JSON Schema + TypeScript + Swagger = ✨ Vol. 1

David Peng — Tue, 17 May 2022 14:48:03 +0000

Hi! I'm David Peng👋. You can find me on Twitter: @davipon.

Welcome to Part 2 of the Better Backend DX series.

This time we're going to build a REST API (yet another REST API tutorial 😅) using the starter template from the previous post: Better Backend DX: Fastify + ESBuild = ⚡️, and leveraging JSON Schema & TypeScript to:

Validate routes & serialize outputs
Enable VS Code IntelliSense for route's Request & Reply
Automatically generate OpenAPI schemas and serve a Swagger UI
Improve code reusability & testability
Improve DX, for sure 😉

Before we start, I like to share a tweet about DX that resonated with me:

// Detect dark theme var iframe = document.getElementById('tweet-1524384220776484864-743'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1524384220776484864&theme=dark" }

This series might cover some, but it's great to keep in mind and review your DX metrics regularly.

What is JSON Schema?

JSON Schema is a vocabulary that allows you to annotate and validate JSON documents.

Here's an example, let's say we have a JSON object like this:

{
  "hello": "world"
}

We can ensure the value of "hello" is a string type by declaring a JSON Schema:

{
  "type": "object",
  "properties": {
    "hello": { "type": "string" }
  }
}

In Fastify, it's recommended to use JSON Schema to validate your routes and serialize your outputs.

Why do we validate routes?

Assume we're developing a blog post REST API. We want to get posts that haven't been deleted:

# General GET method
GET http://localhost:3000/posts HTTP/1.1

# Get method with a filter in the query string (/posts?deleted=[boolean])
GET http://localhost:3000/posts?deleted=false HTTP/1.1

It should return a JSON with an array of posts:

{
  "posts": [
    {
      "title": "...",
      ...
      "deleted": false
    },
    ...
  ]
}

In this case, query string deleted should only accept boolean, but what if the API user accidentally used string?

# The Wrong type of query string "deleted"
GET http://localhost:3000/posts?deleted=abcde HTTP/1.1

If we don't validate our route, the user may still receive a response, but it's a false-positive:

{
  "posts": []
}

As an API designer, we need to provide meaningful insights to the downstream consumer if schema validation fails for a request, like a proper error message:

{
  "statusCode": 400,
  "error": "Bad Request",
  "message": "querystring.deleted should be boolean"
}

Why do we serialize outputs?

Fastify encourages users to use an output schema, as it can drastically increase throughput and help prevent accidental disclosure of sensitive information.

I recommend reading the Fastify documentation Validation and Serialization for more details.

How to use JSON Schema in Fastify?

Take the blog post API for instance:

// src/routes/posts.ts
import { FastifyInstance } from 'fastify'
import { posts } from './posts'

export default async (fastify: FastifyInstance) => {

  const querySchema = {
    type: 'object',
    properties: {
      deleted: { type: 'boolean' }
    }
  }
  /*
    fastify.get(path, [options], handler)
  */
  fastify.get(
    '/', // GET http://localhost:3000/posts HTTP/1.1
    {
      // Validate Query String Schema: /posts?deleted=[boolean]
      schema: {
        querystring: querySchema
      }
    },
    async function (req, reply) {
      /*
        req.query would look like this:
        {
          "deleted": true | false
        }
      */
     const { deleted } = req.query
      if (deleted !== undefined) {
        const filteredPosts = posts.filter((post) => post.deleted === deleted)
        reply.send({ posts: filteredPosts })
      } else reply.send({ posts }) // return all posts if no "deleted" query string
    }
  )
}

Fastify can now validate the querystring by passing schema to routes options.

But there is one problem if you're using TypeScript: VS Code would shout at you that deleted doesn't exist on type '{}':

Emm 🧐, maybe just create a new type for req.query?

type postQuery = {
  deleted: boolean
}
...
// Works like a charm 🥳
const { deleted } = req.query as postQuery

Yeah, it works. We declare a JSON Schema to validate routes and then declare a type for static type checking.

const querySchema = {
  type: 'object',
  properties: {
    deleted: { type: 'boolean' }
  }
}

type postQuery = {
  deleted: boolean
}

Both objects carry similar if not exactly the same information. This code duplication can annoy developers and introduce bugs if not properly maintained.

Keep typing twice is just like forcing Uncle Roger to watch more of Jamie Oliver's video.

Stop typing twice 🙅‍♂️

json-schema-to-ts comes to the rescue. 💪

It's a library that helps you infer TS types directly from JSON schemas:

import { FromSchema } from 'json-schema-to-ts'

const querySchema = {
  type: 'object',
  properties: {
    deleted: { type: 'boolean' }
  }
} as const

type postQuery = FromSchema<typeof querySchema>
// => Will infer the same type as above

Here are some benefits to use this library:

✅ Schema validation: FromSchema raises TS errors on invalid schemas, based on DefinitelyTyped's definitions
✨ No impact on compiled code: JSON-schema-to-ts only operates in type space. And after all, what's lighter than a dev-dependency?
🍸 DRYness: Less code means less embarrassing typos
🤝 Real-time consistency: See that string that you used instead of an enum? Or this additionalProperties you confused with additionalItems? Or forgot entirely? Well, JSON-schema-to-ts does!
and more!

Let's update the route:

// src/routes/posts.ts
import { FastifyInstance } from 'fastify'
import { FromSchema } from 'json-schema-to-ts'
import { posts } from './posts'

export default async (fastify: FastifyInstance) => {

  const querySchema = {
    type: 'object',
    properties: {
      deleted: { type: 'boolean' }
    }
  } as const

  type postQuery = FromSchema<typeof querySchema>

  /*
    The shorthand route methods (i.e., .get) accept a generic object
    "RouteGenericInterface" containing five named properties:
    Body, Querystring, Params, Headers, and Reply.
    The interfaces Body, Querystring, Params, and Headers will be passed down
    through the route method into the route method handler request instance and
    the Reply interface to the reply instance.
  */

  fastify.get<{ Querystring: postQuery }>(
    '/',
    {
      schema: {
        querystring: querySchema
      }
    },
    async function (req, reply) {
     const { deleted } = req.query
      if (deleted !== undefined) {
        const filteredPosts = posts.filter((post) => post.deleted === deleted)
        reply.send({ posts: filteredPosts })
      } else reply.send({ posts })
    }
  )
}

Great! Now we can declare JSON Schemas to validate routes and infer types from them. JSON-schema-to-ts can raise TS error on invalid schemas and enable intelligent code completion of the request or reply instance in the handler function.

Most of these goodnesses happen during development so that we can catch bugs or problems as early as possible. The shorter feedback loop results in a much better DX.

API Documentation: Swagger UI

Swagger UI allows anyone — be it your development team or your end consumers — to visualize and interact with the API's resources without having any of the implementation logic in place.

When it comes to DX, it's not just my and my co-workers' experience but also our end consumers' DX. API documentation and standard OpenAPI specification can help your user in many ways.

I'd recommend watching this webinar by Swagger: API Developer Experience: Why it Matters, and How Documenting Your API with Swagger Can Help

While documenting can be a tedious and time-consuming task, and that's why we need @fastify/swagger to generate docs automatically.

OpenAPI Documentation Generator

@fastify/swagger is a fastify plugin to serve a Swagger UI, using Swagger (OpenAPI v2) or OpenAPI v3 schemas automatically generated from your route schemas, or from an existing Swagger/OpenAPI schema.

In other words, we can easily generate the doc and serve a Swagger UI because we'd already had route schemas! Let's do it:

pnpm add @fastify/swagger

Configure the plugin (I'm using OpenAPI v3):

// src/plugins/swagger.ts
import fp from 'fastify-plugin'
import swagger, { FastifyDynamicSwaggerOptions } from '@fastify/swagger'

export default fp<FastifyDynamicSwaggerOptions>(async (fastify, opts) => {
  fastify.register(swagger, {
    openapi: {
      info: {
        title: 'Fastify REST API',
        description: 'Use JSON Schema & TypeScript for better DX',
        version: '0.1.0'
      },
      servers: [
        {
          URL: 'http://localhost'
        }
      ]
    }
    exposeRoute: true
  })
})

Once you start the server, go to http://127.0.0.1:3000/documentation, and you'll see the Swagger UI:

Fantastic! By adding @fastify/swagger, we don't need to hand-code API specifications in YAML or JSON. Simply declare route schemas, and the doc will be generated automatically.

That's the Vol.1 of JSON Schema + Typescript = ✨

OK. I know we haven't touched the real coding part 😅. They'll be in Vol. 2.

The goal of the Vol. 1 is to give you a taste of the goodness of JSON Schema and how to cope with always-shouting TS errors properly.

I'm going to cover these in the Vol. 2:

Improve code reusability & testability by separating options and handler of route method
Use of JSON Schema $ref keyword
Use Thunder Client (VS Code extension) to test APIs

Kindly leave your comment and thoughts below!

See you in Vol. 2!

Better Backend DX: Fastify + ESBuild = ⚡️

David Peng — Mon, 09 May 2022 14:50:39 +0000

Hi! I'm David Peng👋, you can find me on Twitter: @davipon.

I'm currently working on an internal legacy project which I mentioned in my old blog post: "Supercharge Web DX in Svelte Way".

I built a back-end service from scratch in the past few months, part of the monolithic to microservice migration. It was also my first time making all technical decisions like:

Which framework to choose? Express, Koa, Fastify, or NestJS?
DB choice? SQL or NoSQL? Use ORM/ ODM or not?
Understand networking like transport protocols (UDP, TCP, and TLS), application protocols (HTTP/1.1, HTTP/2)
Do we need a load balancer/ reverse proxy? Apache, Nginx, HA Proxy, or Envoy?
Containerize the application or not? Will we need orchestration in the future?
How to build & test in the development, stage, and production environment?
What about the CI/ CD pipeline? DevSecOps?
🤯

It's intimidating😅, but I learned so much by getting my hands dirty. Even though we haven't finished the migration yet, I still want to write them down and start a new blog series.

I'll share my experience and thought process of building a production-ready backend service with a better DX.

There will be four parts (I might change the title in the future 🙃):

Better Backend DX using Fastify & ESBuild
Build a REST API using MongoDB
Build a minimal docker image
Add Envoy as a sidecar proxy

Let's start with the first one!

Better Backend DX (Developer Experience): Fastify + ESBuild = ⚡️

Below are my two beliefs about solving a real problem:

Build things that just work.

Better DX leads to better UX and better products.

When building a SvelteKit app, it's full of joy with a great DX. (Kudos👏 to the community and modern toolings.) But when it comes to JavaScript backend development, the DX is still like in the stone age.

How is the general backend DX?

Just my two cents:

Lacks modern toolings like hot module replacement
Restrictions of Node.js - immature ESM support leads to extra setup or compilation to cjs
tsc/ ts-node + nodemon still slow as hell
No interest in bundling production code (hard to debug? we don't care because it doesn't ship to clients? 🤷)

I couldn't stop thinking about these when I started to develop a pure backend. It seems to me that there is a massive gap between the frontend & backend world in terms of toolings and DX.

Let's say you want to build a node app using TypeScript, and you'd probably use tsc, ts-node, concurrently, or nodemon to:

Watch and compile your project from ts to js
Start the server

It works, but the problem is that you'll have a broken/ long feedback loop. It recompiles the whole project whenever a file changes, and tsc is way too slow:

ref: Benchmarking esbuild, swc, tsc, and babel for React/JSX projects

That's how I feel! I missed Hot Module Replacement and Vite goodness... wait, can we use it?

I used Vite in my React project, and SvelteKit also uses Vite under the hood. So I wondered if it's possible to use Vite in the backend development?

Vite for Node apps?

Yes, there is a Vite plugin: vite-plugin-node which leverages Vite's SSR mode to bring HMR to the Node dev server and also provides adapters for Express, Koa, and Fastify.

I also found a nice vite-fastify-boilerplate which uses the plugin & Vitest.

After trying them out, the feedback loop was much shorter than before thanks to HMR & esbuild, but there are two problems:

Websockets don't seem to work, at least with Fastify #22
I Haven't found a way to bundle production code using vite-plugin-node. (Vite SSR mode doesn't support bundling built-in Node.js lib

I like to bundle/ minify the production code because I want to build a docker image that doesn't need to copy the huge node_modules to the final image. I'll explain in a future post 😉.

Let's step back and compare different approaches:

	`tsc`+`nodemon`	`vite-plugin-node`
HMR	Recompile the entire app every time.	Yes
Compiler	`tsc`	`esbuild` or `swc`
Problem	Too slow	WebSockets issue, bundling options

Hmm🧐, it seems I need to find an alternative approach, so I started to experiment with ESBuild.

Use esbuild as a compiler and backend bundler

If we don't use Vite, we can't have HMR in the dev server.
But we can still use esbuild, the compiler that Vite uses, to replace tsc.

esbuild is so fast that even if we use nodemon to monitor and recompile the entire app, we can still have a short feedback loop.

esbuild-node-tsc is one of the packages that use this approach.

You can also try ts-node + swc + nodemon. ref: https://typestrong.org/ts-node/docs/swc

To have a better backend DX, there are more factors to consider besides dev server and production bundling. Before we start to set up the fastiy + esbuild project, let's talk about Fastify.

Why did I choose Fastify?

Here are the reasons:

Support TypeScript out of the box
Baked-in validation using JSON-Schema
Extensibility - hooks, plugins, and decorators
Good DX, e.g., great logger using pino, rich plugin ecosystem.
@fastify/autoload enables filesystem-based routes & plugins

Performance was not my primary consideration.

Both Express & Koa are fast enough for a small project. IMHO, It would be better to spend dev time optimizing and parallelizing backend calls and DB queries than speeding up a router by 2ms.

Let's build a simple `fastify` + `esbuild` app!

This section aims to build a scalable starter template with a better DX.

Here is the GitHub repo: github.com/davipon/fastify-esbuild

# Create a new project
mkdir fastify-esbuild
cd fastify-esbuild
pnpm init
# Install dependencies
pnpm add fastify fastify-plugin @fastify/autoload
# Install devDependencies
pnpm add -D typescript @types/node nodemon esbuild

Create a src folder and index.ts under src:

// src/index.ts
import Fastify from 'fastify'

const fastify = Fastify({
  logger: true
})
// Declare a route
fastify.get('/', function (request, reply) {
  reply.send({ hello: 'world' })
})
// Start the server
const start = async () => {
  try {
    await fastify.listen(3000)
  } catch (err) {
    fastify.log.error(err)
    process.exit(1)
  }
}
start()

We use nodemon to monitor & restart the dev server.
Let's create nodemon.json under your project root folder:

// nodemon.json
{
  "watch": ["src"],
  "ignore": ["src/**/*.test.ts"],
  "ext": "ts,mjs,js,json,graphql",
  "exec": "esbuild `find src \\( -name '*.ts' \\)` --platform=node --outdir=dist --format=cjs && node dist",
  "legacyWatch": true
}

The "exec" script does following:

Compile all ts files under src folder to cjs (CommonJS) format and output to the folder dist
Start the dev server with node dist (or node dist/index.js)

Caveat

find src \$ -name '*.ts' \$ is only available in Linux/ macOS.

Will provide an alternative solution for Windows later. (ref: https://github.com/evanw/esbuild/issues/976)

The reason to assign all files but not just the entry like src/index.ts in the esbuild script is that we need to use @fastify/autoload later.

Then add scripts in package.json:

...
"scripts": {
    "dev": "nodemon",
    "build": "rm -rf build && esbuild `find src \\( -name '*.ts' \\)` --platform=node --outdir=build --bundle",
    "start": "pnpm build && node build"
  },

Let's try pnpm dev. You should see something like this:

Enables filesystem-based routes & plugins using `@fastify/autoload`

First create a routes folder under src, and then create a root.ts under routes:

// src/routes/root.ts
import { FastifyPluginAsync } from 'fastify'

const root: FastifyPluginAsync = async (fastify, opts): Promise<void> => {
  fastify.get('/', async function (request, reply) {
    return { root: true }
  })
}

export default root

Then in your src/index.ts:

import Fastify from 'fastify'
import { join } from 'path'
import autoLoad from '@fastify/autoload'

const fastify = Fastify({
  logger: true
})

// Will load all routes under src/routes
fastify.register(autoLoad, {
  dir: join(__dirname, 'routes')
})

const start = async () => {
  try {
    await fastify.listen(3000)
  } catch (err) {
    fastify.log.error(err)
    process.exit(1)
  }
}
start()

Let's give it a try!

Awesome🥳! Now let's add more routes:

Create a examples folder under routes and create index.ts in it:

// src/routes/examples/index.ts
import { FastifyPluginAsync } from 'fastify'

const examples: FastifyPluginAsync = async (fastify, opts): Promise<void> => {
  fastify.get('/', async function (request, reply) {
    return 'Hi there!'
  })
}

export default examples

Here is the demo:

With @fastify/autoload, we can easily add plugins and routes. In the future, you might want to deploy some of those independently. It also provides an easy path to a microservice architecture.

You can see a basic structure with routes and plugins like this:

⚠️ Windows' solution for the `esbuild` script

Create esbuild.js under the project root folder.

const env = process.argv[2]
/* eslint-disable @typescript-eslint/no-var-requires */
/* eslint-disable no-undef */
const fs = require('fs')
const path = require('path')
const esbuild = require('esbuild')

let fileArray = []
const getFilesRecursively = (dir) => {
  const files = fs.readdirSync(dir)
  files.forEach((file) => {
    const filePath = path.join(dir, file)
    if (fs.statSync(filePath).isDirectory()) {
      getFilesRecursively(filePath)
    } else {
      fileArray.push(filePath)
    }
  })
}
getFilesRecursively('src')

const entryPoints = fileArray.filter((file) => file.endsWith('.ts'))

esbuild.build({
  entryPoints,
  logLevel: 'info',
  outdir: env === 'dev' ? 'dist' : 'build',
  bundle: env === 'dev' ? false : true,
  platform: 'node',
  format: 'cjs'
})

Replace esbuild script in both nodemon.json and package.json:

// nodemon.json
{
  ...
  "exec": "node esbuild.js dev && node dist",
}

// package.json
{
  ...
  "script": {
    "build": "rm -rf build && node esbuild.js",
  }
}

That's it! 🎉 This is Part 1.

Thank you for your reading!

You may notice that this fastify + esbuild is heavily inspired by modern web frameworks and toolings.

What motivated me to write this series was the frustration. You can find many resources and articles about modern tools and how they improve frontend DX, but only a few in the backend world talk about it.

In the frontend world, Vite's approach to leveraging ESM & HMR to tackle slow server start and bundling is fantastic. I hope we can have native backend/ Node.js support in the future.

In the next blog post, I'll share how we can use this fastify + esbuild template to:

Build REST API using MongoDB
Use json-schema-to-ts to validate data and stop typing twice
Use native MongoDB driver instead of ODM like Mongoose

Kindly leave your comment and thoughts below!

See you in the next one!

Add lint-staged To SvelteKit Project

David Peng — Sun, 17 Apr 2022 07:46:32 +0000

In my previous post: Add Commitint, Commitizen, Standard Version, and Husky to SvelteKit Project, I'd introduced how to add Conventional Commits and SemVer to your SvelteKit Project.

We added a pre-commit hook to format and lint our code before committing:

# Add pre-commit hook
npx husky add .husky/pre-commit 'pnpm format && pnpm lint && git add .'

While there is a problem that we format and lint our entire codebase every time, it's not ideal if we make a small commit.

Can we ONLY format and lint files we've changed? 🙋

The answer is "Yes", and this is where lint-staged comes into play.

What is 🚫💩 lint-staged?

Run linters against staged git files, and don't let 💩 slip into your codebase!

As the name suggests, lint-staged enables us to lint staged files. But we can leverage it to do more like formatting (prettier), type checking (tsc), ... etc.

Let's see how to add lint-staged and improve the workflow.

Install

Please refer to my previous post before we start or directly check the final repo.

# Install lint-staged
pnpm add -D lint-staged

Setup

Create a new file called .lintstagedrc under the project root folder:

{
  "*.{js,ts,svelte,css,scss,postcss,md,json}": [
    "prettier --write --plugin-search-dir=.",
    "prettier --check --plugin-search-dir=."
  ],
    "*.{js,ts,svelte}": "eslint"
}

Update .husky/pre-commit:

#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

# npm
# npx lint-staged

# pnpm
pnpm exec lint-staged

You might be curious "Why not just use pnpm format, pnpm lint directly?". To explain that, let's take a look at our script tag in package.json:

// package.json
// Default SvelteKit skeleton project with TypeScript, ESLint, Prettier
{
  ...
  "script": {
    ...
    "lint": "prettier --ignore-path .gitignore --check --plugin-search-dir=. . && eslint --ignore-path .gitignore .",
    "format": "prettier --ignore-path .gitignore --write --plugin-search-dir=. .",
  }
}

Put simply, these two scripts would:

Ignore files listed in .gitignore
Run eslint & prettier against the whole project because we had . at the end of the each script

Remember we ONLY want to lint and format files that changed?

We need to customize the script so lint-staged can perform correctly.

Anatomy of the `.lintstagedrc`

{
  "*.{js,ts,svelte,css,scss,postcss,md,json}": [ 
    "prettier --write --plugin-search-dir=.",
    "prettier --check --plugin-search-dir=."
  ],
    "*.{js,ts,svelte}": "eslint"
}

Filtering Files: Which files do you want to lint or format? Ex: *.{js,ts,svelte}
Ignoring Files: Project uses .gitignore as its ignore path for each command. Since lint-staged only runs against staged files, we don't need to ignore files that have been already ignored by .gitignore.
Running multiple commands in a sequence: We can run multiple commands in a sequence, just put them in an array.
--plugin-search-dir=.: Tell prettier to use plugins. We have prettier-plugin-svelte installed in the SvelteKit project.
prettier --write & prettier --check runs sequentially; prettier & eslint runs against filtering files concurrently.

You can remove prettier --check, use eslint --fix, or customize it depending on your need.

What about `svelte-check`?

In my previous post, I put svelte-check in my pre-push but not pre-commit hook.
The reason is that svelte-check needs to know the whole project to do valid checks.
Refs:

Wrapping Up

I have been experimenting with different configurations and toolings to optimize our workflow in the last few days.

Conventional Commits & SemVer help us forge a better code review process, but we also want to ensure the workflow is robust enough when the project scales.

Using lint-staged in pre-commit is one of the improvements to reduce the time and improve our DX.

That's all. Thank you for your reading! 🙌

Please leave a comment if you have any questions or share your experience. Love to connect with you and learn from you.

You can find me on Twitter: @davipon

Add Commitlint, Commitizen, Standard Version, and Husky to SvelteKit Project

David Peng — Sun, 10 Apr 2022 06:39:22 +0000

4/19/2022 Updates. I published a second post according to this one: Add lint-staged to SvelteKit Project

Recently I spent a lot of time improving our internal SvelteKit project by aligning coding style & repo quality, especially the commit message and versioning.

In this post, I will share how I added Conventional Commits and SemVer standards to our project and walk you through the setup of commitlint, commitlizen, standard-version, and husky.

What are Conventional Commits & SemVer?

Here, I copy the summary from each website. You can find more via the link.

Conventional Commits

The Conventional Commits specification is a lightweight convention on top of commit messages. It provides an easy set of rules for creating an explicit commit history, which makes it easier to write automated tools on top of. This convention dovetails with SemVer, by describing the features, fixes, and breaking changes made in commit messages.

The commit message should be structured as follows:



<type>[optional scope]: <description>
[optional body]
[optional footer(s)]

/* Example */
feat(landing): add register button and authentication

SemVer - Semantic Versioning

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes

MINOR version when you add functionality in a backward-compatible manner

PATCH version when you make backward-compatible bug fixes

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

How to adopt in your existing SvelteKit (or any other JavaScript) Project?

You can find the demo repo here: conventional-commits-sveltekit.

commitlint

First, install commitlint, a linting tool to check if your commit messages meet the conventional commit format:



# Install commitlint cli and conventional config
pnpm add -D @commitlint/config-conventional @commitlint/cli

# Configure commitlint to use conventional config
echo "module.exports = {extends: ['@commitlint/config-conventional']}" > commitlint.config.cjs

Use .cjs instead of .js because "type": "module" is set in package.json in SvelteKit project.

Now let's test if commitlint works (no prompt if linting passed):

🐶 husky

It's recommended to use husky with commitlint, a handy git hook helper, to check your commit message before making any git commit.

Will add more git hooks later.



# Install husky
pnpm add -D husky

# Active hooks
pnpx husky install

# Add hook
npx husky add .husky/commit-msg 'pnpx --no -- commitlint --edit $1'

Let's make a git commit:

husky would abort the commit if it didn't pass the commitlint

commitizen

It isn't very pleasant if we need to type the valid commit message every time.

commitizen is a CLI tool to help us fill out any required commit fields at commit time.



# Install commitizen, 
# @commitlint/cz-commitlint (adapter), and
# inquirer (peer dependency of @commitlint/cz-commitlint)
pnpm add -D commitizen @commitlint/cz-commitlint inquirer

Simply use pnpm cz instead of git commit when committing. Or add a pnpm script to your package.json:



...
  "scripts": {
    "commit": "cz"
  }

Let's give it a try:

We use @commitlint/cz-commitlint as a commitizen adapter to ensure the commit adheres to the commit convention configured in commitlint.config.cjs.

standard-version

We've configured commitlint, husky, and commitizen to easily create and validate our commit message.

Now, let's see how we can automate versioning and changelog generation using standard-version.



# Install standard-version
pnpm add -D standard-version

Add pnpm script to your package.json.



...
  "scripts": {
    "release": "standard-version --no-verify" 
    // use --no-verify to skip git hooks we'll introduce later
  }

Here is how standard-version works:

Retrieve the current version of your repository by looking at packageFiles, falling back to the last git tag.

bump the version in bumpFiles based on your commits.

Generates a changelog based on your commits (uses conventional-changelog under the hood).

Creates a new commit including your bumpFiles and updated CHANGELOG.md.

Creates a new tag with the new version number.

Let's try:

By default, CHANGELOG.md would only show commits belonging to feat or fix, but I like to have every commit in my changelog and add emojis.

We can customize it by creating .versionrc under your project root folder. Here is my setting:



{
  "types": [
    {
      "type": "feat",
      "section": "✨ Features"
    },
    {
      "type": "fix",
      "section": "🐛 Bug Fixes"
    },
    {
      "type": "chore",
      "hidden": false,
      "section": "🚚 Chores"
    },
    {
      "type": "docs",
      "hidden": false,
      "section": "📝 Documentation"
    },
    {
      "type": "style",
      "hidden": false,
      "section": "💄 Styling"
    },
    {
      "type": "refactor",
      "hidden": false,
      "section": "♻️ Code Refactoring"
    },
    {
      "type": "perf",
      "hidden": false,
      "section": "⚡️ Performance Improvements"
    },
    {
      "type": "test",
      "hidden": false,
      "section": "✅ Testing"
    }
  ]
}

You can see more options in Conventional Changelog Configuration Spec.

🐶 husky - more git hooks

Before we push commits and tags, we can consider adding more git hooks to improve our code quality.
Like formatting, linting, and testing.

Maybe you've already set "Format On Save", run linting, or unit testing in watch mode, but it's a personal preference. To ensure a better code quality and efficient collaboration, we can add git hooks like pre-commit, and pre-push to format, lint, and test before opening a PR.

Below we'll take the SvelteKit skeleton project as an example. I choose Typescript, Prettier, ESLint, and Playwright.

First, add formatting and linting at pre-commit stage:



# Add pre-commit hook
npx husky add .husky/pre-commit 'pnpm format && pnpm lint && git add .'

Second (optional), add svelte-check at pre-push stage:



# Add pre-push hook
npx husky add .husky/pre-push 'pnpm check'

I didn't put playwright test here because I think it's better to do the e2e test in CI.
If you're using a test runner like jest, uvu, or vitest, you may consider running unit/ component tests in the pre-commit or pre-push stage.

Caveats

git hooks is not a silver bullet. Your colleague can still "bypass" it in specific ways, but it'd be good to use and align with your team.

Some argue that "Run linting and testing in git hooks is just a drag on productivity and can't force anything. Why do this if we've already had them in CI? It's even useless if your tests didn't run in a production-like environment."

That might be true and depends on your team and your focus.

We use it anyway while thinking about the "Shift-Left" approach in DevOps.

Wrap up

Thank you for your reading! 🙌

We've walked through the steps to adopt Conventional Commits and SemVer in your project.
Also, there are lots of different commitizen adapters and commitlint configurations you can try.

I tried gitmoji and followed Make everyone in your project write beautiful commit messages using commitlint and commitizen by @sohandutta, but end up using the setting in this article because it's not compatible to changelog generation of standard-version (Ref: Issue #859)

My team and I are still working on our first CI/ CD pipeline, and this article is our first step to improve our workflow.

Please kindly share your thoughts and experience. Love to learn from you!

You can find me on Twitter: @davipon

Useful resources:

Continuous Delivery by Dave Farley (Full of wisdom)

Master the Code Review by Curtis Einsmann (I haven't finished yet, but the content is so great that I need to share it here!)

Svelte Starter Kit - Helpful Learning Resources & Packages

David Peng — Sat, 12 Mar 2022 16:14:01 +0000

Hi 👋! I'm David Peng.

In this article, I'll share some helpful Svelte learning resources & packages that I consumed and used in the past few months to migrate a legacy web app in my work.

You can check my last article about my journey here 👉 Supercharge Web DX in Svelte way

I hope this article can help if you're new to Svelte or still on the way to learning it.

Learning Resources

Official Tutorial

Svelte Official Tutorial
I'd strongly recommend finishing this tutorial before watching any "How to build a To-do app in Svelte" videos.
By following this interactive tutorial, you'll have an overall concept of how Svelte works and a simple taste of reactivity in Svelte.

Crash Course

If you've just finished the official tutorial and looking for a video one, here are my collections for Svelte & SvelteKit:

The Net Ninja is one of my favorite YouTube channels with high-quality tutorials. These two great series can give you the confidence to build Svelte apps in real life.

I also watched and coded along with James Q Quick(@jamesqquick)'s SvelteKit Crash Course:

Advanced Svelte from the core team

I found Lihau's blog and YouTube channel are super helpful for those who like to dive deeper into Svelte, from compiler to advanced tips of using store or actions.

Li Hau's Blog

Li Hau's YouTube channel

I also learned a lot from Geoff's articles.

Weekly Svelte by LevelUpTuts

I discovered Svelte in an episode of Syntax.fm, and Scott (host of Syntax.fm and founder of LevelUpTuts) shared his experience of migrating from React to Svelte. Then I found his Weekly Svelte playlist. It helped me a ton in my first few weeks in the Svelte world.

Community

Svelte Society

We are a volunteer global network of Svelte fans that strive to promote Svelte and its ecosystem.

A go-to community for nearly everything you would need. You can find best practices and hundreds of Svelte packages here:

Svelte Society

Svelte Society YouTube channel

And don't forget to join the Svelte Discord channel!

Svelte Sirens

A Svelte Society for women, non-binary people, & allies.

It's fantastic to see such a supportive Svelte community grow! You can find great talks on the Svelte Society YouTube channel.

Svelte Summit

Svelte Summit is dedicated to Svelte and everything happening in the community.

I found informative talks and valuable packages in the last Svelte Full Summit. You can also find them on
the Svelte Society YouTube channel.

Svelte Packages & Utilities

Made with Svelte

It's a collection of projects made with Svelte, from UI libraries to components.

➕ Svelte Add

Svelte Add

This is a community project to easily add integrations and other functionality to Svelte apps.

IMHO this is one of the most critical parts when building a Svelte app.

The SvelteKit template has already covered ESLint, Prettier, TypeScript & Playwright setup.
But what if you like to use Tailwind or Jest in your project as well? (yeah, it's me.)
You might go down a massive rabbit hole of configuration hell.

So use svelte-add instead of reinventing the wheel!

Packages I used in internal business apps

Svelte Actions: A Prototype of Svelte actions for inclusion into official actions in the future.

Svelte MultiSelect: Keyboard-friendly, zero-dependency multi-select Svelte component.

svelte-tiny-virtual-list: A tiny but mighty list virtualization library, with zero dependencies 💪

Svelte notifications: Extremely simple and flexible notifications for Svelte

mdsvex: A Markdown preprocessor for Svelte. Markdown in Svelte.

Vest: Declarative validations framework inspired by unit testing libraries.

vitest-svelte-kit: Automatically configure Vitest from your SvelteKit configuration.

Thanks for reading!

The Svelte community and ecosystem are getting stronger. We can see more and more content creators now working on their Svelte & SvelteKit tutorials, and that's a good thing!

But instead of creating tutorials, I came up with an idea to list all helpful resources that may lower the barrier to introducing Svelte to others.

That's all for this time! I hope you have an incredible journey in the Svelte world 🙌.

You can follow me on Twitter: @davipon

Supercharge Web DX in Svelte way

David Peng — Sun, 20 Feb 2022 08:37:31 +0000

How Svelte saves my mental fatigue from React and JavaScript tooling hell

Hi! I'm David Peng 👋.

In this blog post, I'm going to share my experience of how Svelte improves my DX (Developer Experience) and enables me to build better products.

About Me

I'm a software engineer in semiconductor industry who focus on web technologies, virtualization, and data visualization. I build products which enable internal users to be more creative & productive at their work.

I used to support our customers with their software issues, during that time, I realized that the tool box we had was inefficient & insufficient. Our troubleshooting process was slow, it even took like a week to build a sandbox so we could start "duplicating" our customer's issues.

Later, I joined a small project team and worked on UI/ UX design (I only got some shxx done when I was in college), to reshape an internal platform which aims to solve problems I mentioned above. Few years later, we've built several great products and services that me and my colleagues would use every single day!

I spent quite amount of time in these different areas:

UI/ UX design -> Data Visualization -> Front End -> Back End/ DevOps

I'm not an expert of any of these areas, but what I learned as a designer, developer, and a user is:

Never give in to poor UX, build things that just work.

This is why I like to start my first blog post with this title, but enough talk about me, let's get started!

Background - Migrate Legacy App With Small Team

Our project is basically a website which runs on XAMPP, a web server solution which use Apache http server, PHP and MariaDB. It was a decent choice a couple of years ago because we were asked to make a website in an extremely short time.

As time goes by, it's getting harder to fix bugs or add features, we spent too much time figuring out why we coded that way instead of creating values.

There were two major technical debts:

Messy PHP and Javascript in php file
Ship lots of JS/ CSS libraries to EVERY route

I started to refactor the Javascript part, modularize chunk of codes, use ESM for coding splitting and minimal tree shaking, replace jQuery with vanilla Javascript beacuse we only used it for few Bootstrap components. Codebase was getting better, but far from good.

So I thought it was time to pick a front end framework which is:

Easy for me and my team to learn
Able to migrate gradually and work with php (SPA for each service)
Stable and strong community support

Journey of Web Frameworks And Toolings

We choose React (at first).

Large ecosystem, lots of enterprises are using it, maintained by facebook. Nothing can be even promising right? So I picked a course on Udemy and jumped into React world.

Steep Learning Curve

Understand what is JSX, how virtual DOM works. Learn lifecycle method in class component, proper use of 'this'. Oh, you need state management and the power of immutability, so here is redux, redux saga and redux thunk. Don't forget CSS-in-JS because you'll need to scope your styles to the component level.

Prefer FP over OOP? We have functional components! Learn how to use Hooks like useState, useEffect. Understand HOC to make your component reusable. Redux alternative? You can try useContext & useReducer. Memoization? Use useMemo & useCallback.

Couple of months later, I finally finished the course, I coded along and built a e-commerce website, but I can't stop thinking about why it's so counterintuitive to build things that just work? I also felt anxious when I started to clone our legacy website because there are too many options to start in React world.

Javascript Tooling Hell

TBO, I was totally new to javascript tooling before the React course. I choose Parcel as my first bundler. At first, "Zero Config" amazed me, it supports JSX, typescript, SASS, dev server, hot reloading out of the box. But things got complicated when I added Jest.

You need to manage your Jest, Babel, TS config and add different presets/ plugins so you can use ESM and async/ await in your tests. I spent about two weeks to work on these before I wrote my first several tests. It was frustrated, and I can't even imagine if I used Webpack from the get go.

Another big problem is I can't build my SPA in production and embed it in my php file without modifying hashed filenames and deal with PHP relative path every single build.

Then I tried Vite. Vite also supports lots of things out of the box (especially ESM!). Its HMR - hot module replacement is blazing fast. I can also use plugin like vite-jest (switch to esbuild-jest later on) to replace Babel and speed up my tests. This is the first time I have a relatively "GOOD" DX.

One day, I heard about Svelte in Syntx.fm podcast From React to SvelteKit

I immediately decided to give it a try and I ❤️ it.

“Low Floor, Wide Walls, High Ceiling” - Svelte Goodness

I heard this analogy from JS Party episode "So much Sveltey goodness" (22:14):

JS Party #205 Live! - YouTube

Rich Harris from Svelte joins Amal and Amelia to share in some Sveltey goodness!Chat in the #jsparty channel 👉 https://jsparty.fm/community

youtube.com

Low floor - Easy to get started on.
Wide walls - You can do lots of different things with it.
High ceiling - You can work on increasingly sophisticated projects over time.

This analogy resonated with me because this is exactly how I feel after I rebuilt one of our legacy services with Svelte in 2 months.

What is Svelte & SvelteKit?

There are tons of great resources and tutorials out there, so here just a high-level introduction.

Svelte is a component framework that allows you to break up your application into reusable chunks. Svelte compiles itself away during the build process into vanilla html, css, and javascript. (Quotes from Brittney's Let's Learn Svelte.)

SvelteKit is a Svelte framework for building web applications of all sizes, with a beautiful development experience and flexible filesystem-based routing.

In latest State of JS, Svelte & SvelteKit both got incredible highest satisfaction:

How Svelte saves my mental fatigue?

1. Gentle Learning Curve

Svelte has a great official tutorials with REPL, I only spent a weekend to finish it.

I also coded along with a great SvelteKit crash course from @jamesqquick

Once you understand the structure of svelte file, coding in svelte just like what we did in the past, writing pure HTML and CSS, but with powerful syntactic sugar!

2. Less Javascript & boilerplate

This is my true feeling. Why should I write lines of code just to update a value?
Reactivity, this is all I need. Svelte really gives me productivity to focus on designing great UX but don't compromise on DX.

3. Svelte Store - Native State Management

I totally get the "goodness" of immutability, especially after I learned how redux works.
Even I switched to RTK - Redux ToolKit later, I still feel awful to write so many codes JUST to update a value safely in React.

Reactivity shines again, svelte store is easy to use and customizable. Co-worker and future self can easily understand it's a store with a $ prefix. I didn't feel a pain to manage global stores when my Svelte project scaled.

4. HTML & CSS as a first class citizen

You can write native html & css (scoped) without a pain. That's a huge win for me because I just don't like className & CSS-in-JS.

Svelte released style directives in 3.46 which provides handy syntax sugar to write dynamic styles. You can check Geoff's
(Svelte maintainer) article for use cases.

5. SvelteKit - Go-to framework to build Svelte app

React is a javascript library but not a framework, which means you need to figure out how to build your app properly. I believe for most React beginners, what we need is a single recommendation but not paradox of choice. What is the best option which gives me overall the best outcome with least amount of trade-offs?

So I was excited when I found SvelteKit is the go-to framework in Svelte world. Here are features I love about SvelteKit:

Filesystem-based Routing:
It's way easier to understand and use compare to React Router.
Layout:
It's a component that you can put elements that should be visible on every page, such as top-level navigation or a footer. It's really convenient and easier to manage your global layout and route content.
Apadters:
My purpose is to build each service to SPA at the first stage of migration. adapter-static is a great fit to build SPA and I can easlily switch to other adapters depends on our requirement in the future.

// svelte.config.js
import preprocess from 'svelte-preprocess'
import adapter from '@sveltejs/adapter-static'

/** @type {import('@sveltejs/kit').Config} */
const config = {
    // Consult https://github.com/sveltejs/svelte-preprocess
    // for more information about preprocessors
    preprocess: [preprocess({
        postcss: true
    })],

    kit: {
        // I can even build php file which I can use in current project seamlessly!
        adapter: adapter({
            fallback: 'myapp.php' 
        }),
        files: {
            assets: 'src/assets',
        },
        appDir: 'myapp',
    }
}

export default config

Use Vite under the hood: As I mentioned earlier, Vite had saved me from javascript tooling hell. After two months development in SvelteKit, everything works like a charm.

6. Svelte Community

I think this is the most important reason that I love Svelte so much, and it's definately part of the DX.

Svelte Summit - An event dedicated to Svelte and its community.
Svelte Society - Great resources and support from the community.
Svelte Sirens - A Svelte Society for women, non-binary people, & allies
Svelte Discord Channel - Go-to online forums and chats

Wrapping up - Better DX, better UX, better product 💪

Before wrapping up, I like to share this informative talk from Lee Robinson and Rich Harris (author of Svelte):

We're a nimble team with only few developers, we need to cover design, development, testing, maintenance, even customer service at the same time. To prevent us from burning out during migration, DX is a critical factor we need to consider of.

About the importance of a "Good" DX, I learned the hard way in my journey from React to Svelte.
Shorter feedback loop, from dev server to product users, enables me to spend more time on design, functionality, usability and iterate faster, which results in better UX and better product.

We're still on the road to rebuild our products in Svelte, I'm really exciting and can't wait to share more with you in the future!

Have you tried Svelte?
How was your DX in web development?
What tools have improved your DX?

Please kindly share your experience and thoughts below!

Thank you for your reading 🙌
You can follow me on Twitter for my future works. Stay tuned!

Forem: David Peng

Test Svelte Component Using Vitest & Playwright

Different types of test

Test Runners

Vitest: A Vite-native unit test framework. (alternative: Jest, uvu)

Playwright: A framework for Web Testing and Automation. (alternative: Cypress)

When to use each test runner?

Vitest + @testing-library/svelte

Take Svelte Demo App (TypeScript) for example

Playwright Experimental Component Test

Take Svelte Demo App (TypeScript) for example

Vitest (as test runner) + Playwright

Wrapping Up

Resources

Discussion

Issues

Articles

Videos

Build a Svelte Component To Draw SVG On Hover

Let's talk about motivation

Can we DRY them up?

A Trick That Makes Drawing SVG Lines Way Easier

Make a wrapper component for abstraction

Easy to use + less code to write FTW 💪

Wrapping up

Better Backend DX: JSON Schema + TypeScript + Swagger = ✨ Vol. 2

Improve Code Readability & Testability

Build a Simple Blog Post CRUD API

Request & Response Schemas

JSON Schema $ref Keyword

Route Schemas

Handler Functions

Fastify Route Method

Swagger UI & OpenAPI Specification

Test API Using Thunder Client

🙌 Wrapping up

Better Backend DX: JSON Schema + TypeScript + Swagger = ✨ Vol. 1

What is JSON Schema?

Why do we validate routes?

Why do we serialize outputs?

How to use JSON Schema in Fastify?

Stop typing twice 🙅‍♂️

API Documentation: Swagger UI

OpenAPI Documentation Generator

That's the Vol.1 of JSON Schema + Typescript = ✨

Better Backend DX: Fastify + ESBuild = ⚡️

Better Backend DX (Developer Experience): Fastify + ESBuild = ⚡️

How is the general backend DX?

Vite for Node apps?

Use esbuild as a compiler and backend bundler

Why did I choose Fastify?

Let's build a simple fastify + esbuild app!

Enables filesystem-based routes & plugins using @fastify/autoload

⚠️ Windows' solution for the esbuild script

That's it! 🎉 This is Part 1.

Add lint-staged To SvelteKit Project

What is 🚫💩 lint-staged?

Install

Setup

Anatomy of the .lintstagedrc

What about svelte-check?

Wrapping Up

Add Commitlint, Commitizen, Standard Version, and Husky to SvelteKit Project

What are Conventional Commits & SemVer?

Conventional Commits

SemVer - Semantic Versioning

How to adopt in your existing SvelteKit (or any other JavaScript) Project?

commitlint

🐶 husky

commitizen

standard-version

🐶 husky - more git hooks

Caveats

Wrap up

Useful resources:

Svelte Starter Kit - Helpful Learning Resources & Packages

Learning Resources

Official Tutorial

Crash Course

Advanced Svelte from the core team

Vitest + `@testing-library/svelte`

JSON Schema `$ref` Keyword

Let's build a simple `fastify` + `esbuild` app!

Enables filesystem-based routes & plugins using `@fastify/autoload`

⚠️ Windows' solution for the `esbuild` script

Anatomy of the `.lintstagedrc`

What about `svelte-check`?