🎭 playwright-cli-select

Dennis Bergevin — Sat, 15 Feb 2025 18:39:15 +0000

Problem context

Playwright comes out of the box with a collection of ways to run test files, test titles, and test tags, including the VS Code extension and UI Mode.

However, I had conversations with colleagues at work who wanted to run Playwright end-to-end tests on their local machine and have an easier time in the terminal running specific tests or suites, maybe a couple tags, and a test file all in one run.

That was where I noticed a gap in test tool enablement, and what prompted me to work on playwright-cli-select.

Why do I use this?

1. Interactive buddy: Use with any of your desired Playwright command line parameters while using an interactive interface including test files, suite/test titles, and/or tags available in your project.

2. Discoverability enhancement: If a user wants to run tests on their local machine but...

a. Doesn't know which files, suites/tests or tags are available to choose from
b. They don't know how to run a specific collection of files/tests/tags

3. Customize tests to run: Maybe some tests just failed and you want to run Playwright's --last-failed flag to inspect but you don't want to run all of the failed tests, or maybe you want to run Playwright's --only-changed but have option to choose a specific test in a changed spec and not all tests in the changed spec.

4. Increase independence: If you are looking for a local development tool to enable your colleagues to run Playwright tests without a bunch of documentation or reference for which command to run or how to run tests, then look no further!

How do I use this?

The package is available on Github and via npm.

Once installed, users can run a base command in their Playwright project and include any Playwright specific command line parameters/flags:

# base command for interactive cli prompts
npx playwright-cli-select run

# Example: user wants to choose tests given a specific project
npx playwright-cli-select run --project=chrome

# Example: user wants to run a specific changed test
npx playwright-cli-select run --only-changed --titles

# Example: user wants to run a specific last failed test
npx playwright-cli-select run --last-failed --titles

Additionally, there is a --help command available showing available package-specific parameters:

Under the hood

When you run npx playwright-cli-select run in your Playwright project, it is using npx playwright test --list --reporter=json command under the hood, with allowance for any Playwright command line arguments, to gather available test files, suites, tests, and tags.

The available test files, suites, tests and tags are then displayed in interactive prompts.

Hint 1: Passing arguments to a flag such as --project would be passed onto this --list command, which will show all test files, suites, tests, and tags associated with that specific project.

Hint 2: Passing --last-failed and --only-changed will alter the results from the --list command, allowing you to tailor the choices seen in the various prompts to dial deeper into a specific test from those commands.

Rather than using Playwright's --grep to pass selections to npx playwright test command for starting a test run, the file:line method is used (i.e. my-spec.ts:42).

...
        {
          "title": "Routing",
  ------> "file": "demo-todo-app.spec.ts",
  ------> "line": 382,
          "column": 6,
          "specs": [
            {
              "title": "should allow me to display active items",
              "ok": true,
              "tags": [],
              "tests": [
                {
                  "timeout": 30000,
                  "annotations": [],
                  "expectedStatus": "passed",
                  "projectId": "chromium",
                  "projectName": "chromium",
                  "results": [],
                  "status": "skipped"
                }
              ],
              "id": "849da1cee0c412ce5534-168ac99248367a9c5f75",
     ------>  "file": "demo-todo-app.spec.ts",
     ------>  "line": 391,
              "column": 7
            },
            {
              "title": "should respect the back button",
              "ok": true,
              "tags": [],
              "tests": [
                {
                  "timeout": 30000,
                  "annotations": [],
                  "expectedStatus": "passed",
                  "projectId": "chromium",
                  "projectName": "chromium",
                  "results": [],
                  "status": "skipped"
                }
              ],
              "id": "849da1cee0c412ce5534-f4a7bfde65c589b439d6",
      ------> "file": "demo-todo-app.spec.ts",
      ------> "line": 401,
              "column": 7
            },
...

While --grep has it's obvious uses and is very powerful, there are many considerations and complexities when enabling test selection using this command, notably that different shells require different escaping using --grep.

Additionally, it can become tedious crafting a long --grep of multiple OR conditions if you are looking to run various combinations of test files, suites, tests and tags.

The playwright-cli-select will parse the --list of available test files, suites, tests and tags and store the file:line for each suite, test, and tag. Then, when a user picks a given selection, the stored file:line will be sent as argument to the npx playwright test command to run the selections.

Where can I get more info?