Forem: Chris Fung

Running Rails 5 System Tests on Travis CI with Chromedriver

Chris Fung — Sun, 29 Jul 2018 22:35:59 +0000

Rails 5.1 introduced the concept of system tests using Capybara and Selenium. System tests takes the practice of "feature" tests that many Rails apps were already doing and brings it into an official part of the Rails testing strategy. Since system tests utilize a browser, they can execute Javascript, something that Rails integration tests could not do.

System tests work great out of the box on local machines, but getting them set up to run on a remote server like Travis CI is a little harder. There's not a lot of good documentation out there for what needs to be done, and a lot of solutions have to be dug up through Stack Overflow and GitHub comment threads.

In this post, I'll show how I set up system tests to run on Travis CI using headless Chrome and Chromedriver.

Rails 5.1.6

Although system tests were announced with 5.1, you really should make sure you are using Rails 5.1.6 to get the best experience. This is because after the initial release there were a few other small quality of life improvements made to the framework that are great to have.

Among these are a change to ActiveRecord that allows the Capybara thread and the test server thread to use the same database connection (see rails/rails#28083). This is important when using transactions because otherwise the two threads would not be able to see DB changes made by the other thread.

In the past, you would have had to work around this by turning off transactions and adding Database Cleaner or some similar gem to clean up data created during test runs. But this would have meant that either Rails would need to pull Database Cleaner in as a dependency, or anyone wanting to use system tests would need to add another gem to their Gemfile manually. Neither of those were good for the future adoption of system tests, so instead the Rails team updated ActiveRecord to allow the connection to be shared.

RSpec 3.7

I prefer using RSpec as my test framework. RSpec 3.7 added support for system tests into the framework, replacing the previous feature specs.

If you are using RSpec, you will need to make sure you are using version 3.7 in order to use system tests.

Installing Chromedriver

Chromedriver allows Chrome to be controlled by Selenium and is necessary in order for us to use Chrome as the browser in system tests.

You can download Chromedriver from the project's website, or it can be installed with Homebrew Cask:

$ brew cask install chromedriver

Removing chromedriver-helper

If you have worked with Chromedriver in the past, you may have installed the Ruby gem chromedriver-helper. If you want to use Chromedriver directly, you will need to remove chromedriver-helper and it's executables from your system.

If you're using rbenv, you can use the commands I used:

$ gem uninstall chromedriver-helper
$ ls ~/.rbenv/versions/*/bin/* | grep chromedriver | xargs rm
$ rbenv rehash

Note: Removing chromedriver-helper may or may not be necessary for you, depending on your setup. I found it was causing problems for me, so I decided to remove it.

Selecting a driver

System tests abstract Capybara's drivers using the driven_by method. With regular system tests, you would include driven_by in your ApplicationSystemTestCase. RSpec, however, does not use ApplicationSystemTestCase, so you need to configure the driver in a before block.

The easiest way to do this is by adding a global before(:each) to rails_helper.rb:

RSpec.configure do |config|
  config.before(:each, type: :system) do
    driven_by :selenium, using: :chrome
  end
end

This will set all system tests to be driven by Selenium using Chrome.

type: :system indicates that this before block will only apply to tests marked as system tests. RSpec will infer that a test is a system test if it is located in spec/system, or you can tag specs manually.

Now running bundle exec rspec should open a Chrome window and run your system specs. Cool!

Going headless

Opening a browser window works fine on your local machine, but it won't work on a remote server because there's no monitor to render the window. Instead, we have to use Chrome in headless mode.

To use headless Chrome, we can pass some extra arguments to driven_by:

 RSpec.configure do |config|
   config.before(:each, type: :system) do
-    driven_by :selenium, using: :chrome
+    driven_by :selenium, using: :chrome, options: { args: ["headless"] }
   end
 end

There are other ways to use headless mode, and the downside of this method is that it causes Selenium to throw a deprecation warning. For reasons we'll see later, we have to use this method of passing arguments.

Now running bundle exec rspec should not open a browser window. Neat!

Running on Travis

Getting the tests to run on Travis is the trickiest part. For my build, I wanted to use Travis's Docker infrastructure with sudo: false in my .travis.yml file. This meant that I had to work around several limitations.

Installing Chrome and Chromedriver

To install packages on Travis, you need to use the addons section in your config file. Although Travis has an addon specifically for Chrome, I ended up having some problems using this. Instead, I used the apt addon to install both Chrome and Chromedriver:

addons:
  apt:
    - google-chrome-stable
    - chromium-chromedriver

Selenium expects to find Chromedriver on the PATH, but the default installation on Travis does not put the binary on the PATH. To fix this, I created a symlink in the before_script step:

before_script:
  - ln -s /usr/lib/chromium-browser/chromedriver ~/bin/chromedriver

Now the Travis environment is set up correctly.

Chrome crashes on launch

Even with the environment set up correctly, my tests were still failing because Chrome would immediately crash on launch. After a lot of digging, I found this troubleshooting page.

There are two takeaways:

We need to run Chrome with --no-sandbox in order for it to work properly on Travis.
We need to disable usage of /dev/shm because Docker provisions too little space in that device and Chrome crashes if there's not enough space.

In order to do this, we need to be able to pass custom arguments to Selenium, which is why we are using the driven_by options that we have.

 RSpec.configure do |config|
   config.before(:each, type: :system) do
-    driven_by :selenium, using: :chrome
+    driven_by :selenium, using: :chrome, options: { args: ["headless", "disable-gpu", "no-sandbox", "disable-dev-shm-usage"] }
   end
 end

The disable-gpu argument is there on recommendation from this page.

Now your system tests should successfully run on Travis! Awesome!

Extra credit: Filter system tests with metadata

As we have it set up now, RSpec will run all system tests whenever we run the full test suite. But system tests are slow and we probably don't want to run them all the time. Luckily RSpec has a way to filter out certain specs from the run based on the metadata on the spec.

Let's tag all specs in spec/system with the browser tag. To do this, we can use define_derived_metadata:

RSpec.configure do |config|
  config.define_derived_metadata(file_path: /spec\/system/) do |metadata|
    metadata[:browser] = true
  end
end

Then we can use filter_run_excluding to filter browser specs by default:

RSpec.configure do |config|
  config.filter_run_excluding browser: true
end

Now when running bundle exec rspec, system tests will be filtered out. To run the system tests, we can use the -t option to include browser specs.

$ bundle exec rspec -t browser

Remember to change your .travis.yml to use -t to run the system tests. Having system tests split off from the rest of the test suite allowed me to parallelize running the slow system tests with running the rest of the suite.

Conclusion

System tests are a really easy way to get started with feature testing. The default configuration shipped with Rails should work for most people, and extending it to use a different browser or pass different options is straightforward. What was harder was trying to figure out how to use system tests on Travis, with not a lot of documentation or examples to go off of.

With this post, I have gathered together everything I learned in getting system tests running on Travis CI. I hope it's useful!

Setting up Jest with React and Webpacker

Chris Fung — Thu, 10 May 2018 06:11:51 +0000

Webpacker is a great library that was introduced in Rails 5.1. It gives an easy, out-of-the-box way to configure the latest Javascript tools, including Babel, Webpack, and React, to work seamlessly with Rails.

Generating a new React project with Webpacker is as easy as running one rails command. But how to write tests for your new React components is a little less clear. There's a myriad of different testing libraries out there for Javascript code, and the ecosystem can be a little confusing.

In this post, I'll show how I set up testing for my React components using Jest, and how I integrated Jest into my existing CI workflow for my Rails app.

Installing Jest

Jest is a Javascript testing library that's easy to set up and fast. It was developed by Facebook and has lots of support for testing React components.

Because Webpacker uses Babel, setting up Jest requires installing one extra package: babel-jest. This allows Jest to transpile code using Babel while testing. Install them both with bin/yarn:

bin/yarn install jest babel-jest

Creating a test

By default Jest will look for any file in a __tests__ directory or any file ending in *test.js and treat those files as test suites. "Suite" is Jest's term for a file containing one or more tests.

With Webpacker, React components are stored in the app/javascript directory. I had a directory structure like this:

app/javascript
├── packs
└── pulls
    └── components

The components directory is where all my components live, so I made a __tests__ directory inside that.

Running Jest

To run Jest, I added a new script to my package.json:

{
  "scripts": {
    "test": "jest"
  }
}

Now you can run Jest with Yarn:

bin/yarn test

However when I did this initially, I got an error:

$ bin/yarn test
yarn run v1.5.1
$ jest
 FAIL  config/webpack/test.js
  ● Test suite failed to run

    Your test suite must contain at least one test.

What's going on here is that Jest is assuming Webpacker's test.js config file is a test file because it ends in test.js. That's obviously not right! Luckily we can tell Jest to ignore certain file patterns with the testPathIgnorePatterns configuration option.

Jest configuration goes in package.json under the jest key. We just need to add config/webpack/test.js to the list of ignored paths:

{
  "jest": {
    "testPathIgnorePatterns": [
      "/node_modules/",
      "<rootDir>/config/webpack/test.js"
    ]
  }
}

Babel modules

Another error I ran into was:

SyntaxError: Unexpected token import

The problem here is that because Jest runs in Node, it needs to compile your code into CommonJS modules. However, by default, Webpacker configures Babel with "modules": false which tells Babel to skip transpiling ES6 modules into another format.

There's two ways to fix this. The first way is to remove "modules": false from Babel in the test environment. You can do this using the env option in .babelrc. However, when I tried this, I got different errors when Babel was transpiling my components. Ultimately, I decided that this way was not going to work out for me.

The second way is to add babel-plugin-transform-es2015-modules-commonjs to the test environment. This plugin will transform ES modules into CommonJS modules. I installed the plugin and added it to my .babelrc:

{
  "env": {
    "test": {
      "plugins": ["transform-es2015-modules-commonjs"]
    }
  }
}

Now running bin/yarn test is successful!

Running on Travis CI

I use Travis CI to run my tests, so I wanted to run Jest on Travis as well. Adding the test command to my .travis.yml file was simple enough, but I started getting more errors from Jest trying to run other Javascript files just because they were named with test.js at the end.

This turned out to be because Travis installs the Webpacker gem to the vendor directory in your project, which puts it in the path of searchable directories for Jest to discover tests in. I had to add the vendor directory to my list of ignored paths.

Conclusion

All in all, setting up Jest to work with Webpacker had a few minor hiccups but they were all easy to overcome. Testing React components with Jest is very easy, and I was surprised that it worked so well with Webpacker's config out of the box!