Cypress with Mochavesome reporter with fail test screenshots. Fix viewport screen size in headless mode.

Andrii Lysenko — Sun, 03 Apr 2022 14:20:26 +0000

What you learn:

install and configure Cypress from zero
install and configure mochawesome reporter plugin for create very beautiful HTML report
how add fail test screenshots to mochawesome report (it's not by default)
how to fix viewport size for headless run (it's always 1280x720 by default)
how use it with Git hook

Installing Cypress and additional components

Install Cypress, here is the link to the official site.

npm install cypress --save-dev

Install the npm-run-all package (if not present)

npm install npm-run-all --save-dev

Install simple-git-hooks package if not used on the project. If this package or an analogue, for example, husky, skip this point. We will move on to the configuration in the next section.

npm install simple-git-hooks --save-dev

Install:

mochawesome - creates a report in json format as a result of running spec file;
mochawesome-merge - generates from several json reports one common json;
mochawesome-report-generator - generates HTML/CSS report to visualize the test run.

npm install --save-dev mochawesome mochawesome-merge mochawesome-report-generator

Configuration and usage

1. Start Cypress.

In the directory with the project run Cypress with the command npx cypress open. The test runner will be opened and the Cypress will be initialized.
The following things will be created:

cypress.json config file;
cypress folder in the root of project. Set configurations in the cypress.json file:

{
"baseUrl": "https://some-domain.com",
"viewportHeight": 1080,
"viewportWidth": 1920,
"video": false,
"screenshotsFolder": "cypress/results/mochawesome-report/assets"
}

At this link you can find additional configuration settings.

In the cypress.json file, you must also add configuration parameters to make the tests run again if the first attempt is unsuccessful.
This is necessary to avoid false-negative test results. A false-negative outcome can be caused by a sudden single application failure, internet connection failure, etc. More information can be found here.
So now cypress.json will look like this:

{
"baseUrl": "https://some-domain.com",
"viewportHeight": 1080,
"viewportWidth": 1920,
"video": false,
"retries":{
    "runMode": 1,
    "openMode": 1
    }
}

2. Set reporter configurations in the cypress.json file.

So now cypress.json will look like this:

{
"baseUrl": "https://some-domain.com",
"viewportHeight": 1080,
"viewportWidth": 1920,
"video": false,
"retries":{
    "runMode": 1,
    "openMode": 1
    },
"reporter": "mochawesome",
  "reporterOptions": {
    "reportDir": "cypress/results",
    "overwrite": false,
    "html": false,
    "json": true
  }
}

3. Create commands to delete the folder with JSON reports in the package.json file.

Delete files and folders if exist, then recreate these folders

{
"delete:reports": "node cypress/support/node_fs/removeDir.js",
"create:reportFolders": "node cypress/support/node_fs/createDir.js"
}

4. Create removeDir.js and createDir.js files.

Create the removeDir.js and createDir.js files in the strictly defined path: cypress/support/node_fs/ - because the above commands will not work.

// removeDir.js

var fs = require('fs')
const dir = 'cypress/results'

fs.readdir(dir, (err, files) => {
  console.log(files)
  if (files) {
    fs.rmdir(dir, { recursive: true }, (err) => {
      if (err) {
        throw err
      }

      console.log(`${dir} is deleted!`)
    })
  } else {
    return console.error(err)
  }
})

//createDir.js

var fs = require('fs')

fs.mkdir('cypress/results', (err) => {
  if (err) {
    return console.error(err)
  }
  fs.mkdir('cypress/results/mochawesome-report', (err) => {
    if (err) {
      return console.error(err)
    }
  })
})

5. Create a script in the package.json file to merge mochawesome JSON report files into a common JSON file.

This file will be named mochawesome.json, the script will create it and place it in the folder cypress/results/mochawesome-report

{
"mochawersome:merge": "npx mochawesome-merge 'cypress/results/*.json' > cypress/results/mochawesome-report/mochawesome.json && npx marge cypress/results/mochawesome-report/mochawesome.json -f report -o cypress/results/mochawesome-report --autoOpen=true"
}

6. Create scripts in the package.json file:

running a test run in headless mode in the chrome browser;
delete the folder with the report of the previous run, if it exists;
generating HTML report of the current run.

{
"cy:chrome": "npx cypress run --browser=chrome",
"pretest": "npm-run-all --sequential delete:reports create:reportFolders",
"test": "npm-run-all --sequential cy:chrome mochawersome:merge --continue-on-error"
}

npm-run-all - package we installed above
--sequential - flag, which starts the commands sequentially
--continue-on-error - a flag that starts the command even if the previous one failed

7. Delete the test files.

By default, the folder cypress/integration contains folders with training test files, it is better to delete them from the real project, you can copy them to another folder on your computer for reading or initiate a separate cypress test project for study.

8. Editing a .gitignore file.

The files that are the result of the test run reporting do not need to be sent to a remote git repository, so we make some additional changes to the .gitignore file

# Cypress testing files and folders
cypress/results

9. Add screenshots of failed tests to the mochawersome HTML report

a) at the top of the file cypress/support/index.js insert the command

import addContext from "mochawesome/addContext";

b) in the same file below add event

Cypress.on("test:after:run", (test, runnable) => {  
  if (test.state === "failed") {    
    const screenshot =`assets/${Cypress.spec.name}/${runnable.parent.title} -- ${test.title} (failed) (attempt 2).png`;    
addContext({ test }, screenshot);  
  }
});
// very much attention to the screenshot constant, because if you specify a wrong mask of the file name, the screenshot will not be in the report

Done! 🤠 The report will now show screenshots.

10. Configure simple-git-hooks.

Study the description of the package and how it works, as a result you need to make additions to the package.json file of the project:

{
  "simple-git-hooks": {
    "pre-push": "npm run test"
  }
}

Don't forget about point #3 in the simple-git-hooks instructions - perform configuration updates by running the npx simple-git-hooks command.

11. Fix the size of the browser window of the application under test, taking into account the test runner.

What do I mean with the test runner in mind? Even though the test run is running in headless mode, cypress still runs the tests in test runner, it just doesn't show up.

Even though you specify viewportWidth and viewportHeight in the test run configurations, Cypress will run the tests in headless mode with screen size = 1280x720px.
There's a very important feature here: don't confuse the screen size and the size of the iFrame in which the application is displayed, 1280x720px is along with the test runner.
Read this article to better understand what I mean, and here's a link to the cypress documentation.

On your project, each team member will have a different screen size (viewport size), so I'll give you this example with mine.

Example:

I need to run a test run for a 1920x1080 screen at 100% zoom, if I ran the default tests I would get about this situation.

As you can see the scale of the page = 43% - this does not suit us, it is not 1920x1080.

Why is this so and where did the above mentioned 1280x720 size go, where does it appear?
By default cypress covers the entire screen area during screening, so this is what we have:

So you need to increase the default screen size, this can be done by changing the browser settings, which will be applied just before running tests.
You need to add this code to this file: cypress/plugins/index.js

module.exports = (on, config) => {
  on("before:browser:launch", (browser = {}, launchOptions) => {
    console.log("launching browser %s is headless? %s", browser.name, browser.isHeadless)

    // the browser width and height we want to get
    // our screenshots and videos will be of that resolution
    const width = 3840
    const height = 2160

    console.log("setting the browser window size to %d x %d", width, height)

    if (browser.name === "chrome" && browser.isHeadless) {
      launchOptions.args.push(`--window-size=${width},${height}`)

      // force screen to be non-retina and just use our given resolution
      launchOptions.args.push("--force-device-scale-factor=1")
    }

    if (browser.name === "electron" && browser.isHeadless) {
      // might not work on CI for some reason
      launchOptions.preferences.width = width
      launchOptions.preferences.height = height
    }

    if (browser.name === "firefox" && browser.isHeadless) {
      launchOptions.args.push(`--width=${width}`)
      launchOptions.args.push(`--height=${height}`)
    }

    // IMPORTANT: return the updated browser launch options
    return launchOptions
  })
}

I took a large enough screen size so that my iframe would not scale in the test runner and got this result:

Notice that we are now at 100% scale, which is exactly what we need!

Proof of this can be seen in the file dimensions

Thus solved the problem of running tests for any viewport size in the headless mode. So now everything is fine, you can sleep well and not worry that the tests will break because of a non-valid screen resolution.

Here is my GitHub repository with a test realisation.

I hope this information helped someone or at least served as a trigger for their own realisation.
Good luck!

Forem: Andrii Lysenko