Forem: Doug Stull

How to use modals with forms in Rails using Turbo

Doug Stull — Fri, 08 Jan 2021 05:04:09 +0000

Goal

Using the Hotwire, show how to setup a basic implementation of a modal with a form.

We'll break this down into talking about the controller, view templates and javascript.

There are many other things that were setup in the example application due to the choices made in the tech stack and some setup/configuration has been derived from things learned on Go Rails. We'll assume some familiarity with Stimulus in order to keep focused on the benefits of Turbo.

Source code for this guide - updated to turbo-rails 0.5.9

Posts page being referenced

Creating a new post using the modal

New link and modal setup

Note this github comment is what I used as guide for this setup.

This all starts in index.html.erb



<%= turbo_frame_tag 'post' %>

<div class="mt-8" data-controller="post-modal" data-post-modal-prevent-default-action-opening="false">
  <%= render partial: 'posts/modal_form' %>
...
<%= link_to 'New Post', new_post_path, class: "btn btn-primary", data: { action: "click->post-modal#open", 'turbo-frame': 'post' } %>

turbo_frame_tag allows us to tell turbo that links with a matching data-turbo-frame value matching it will return an html response including a matching frame tag(not sure on this wording, as we supply it here to merely tell turbo to leave the url unchanged when the link is clicked). Our action upon New Post click will be a replace that is defined in the new.html.erb template. This will populate the modal via use of the target value on the turbo-stream element in the template.
render our modal_form partial which will contain the modal definition and define a div with an id that the turbo-stream response replaces.
data-controller="post-modal" names our post-modal controller and causes it to connect on render.
data-post-modal-prevent-default-action-opening="false" will allow us to open the modal with the link and also fetch new_post_path from the controller and render our modal content.

Clicking the `New Post` link

This action will take us into the posts_controller.rb.



def new
  @post = Post.new
end

We render the html response of new.html.erb



<%= turbo_frame_tag 'post' do %>
  <turbo-stream target="post_form" action="replace">
    <template>
      <%= render partial: "posts/form", locals: { post: @post } %>
    </template>
  </turbo-stream>
<% end %>

We render the turbo_stream response inside the template with a target that matches the id inside the modal. This will then replace that element inside the modal with our form content.
Below is an example response

While the data is being fetched, the modal is given the signal to open, so we'll listen for an event from another Stimulus controller. That controller is the post_form_controller.js, and it keeps us from seeing unprepared html and stale data from an invalid form submission.
We can achieve that by adding an eventListener to post_modal_controller.js's open function.
- this is extended from the original modal supplied from tailwindcss-stimulus-components



document.addEventListener("postForm:load", () => {
  this.containerTarget.classList.remove(this.toggleClass);
});

The above is done in order to take advantage of the connectCallback that seems to be the only way to know when the replace action from turbo-stream is completed.

Rendering the response

At this point the modal that is defined in _modal_form.html.erb is opening and this element below is being replaced with the turbo-stream response that has the rendered form.



<%= tag.div nil, id: 'post_form' %>

Submitting the form

When we submit the form, the modal closes via close function in the extended post-modal stimulus controller and reach the create method in the posts_controller.rb



def create
  @post = Post.new(post_params)

  respond_to do |format|
    if @post.save
      format.html { redirect_to posts_url, notice: 'Post was successfully created.' }
    else
      format.turbo_stream do
        render turbo_stream: turbo_stream.replace('post_form',
                                                    partial: "posts/form",
                                                    locals: { post: @post })
      end
    end
  end
end

If successful, we'll just redirect to the page the modal is launched from, and flash a message. This will all feel seamless since Turbo takes care of replacing items in place; negating the need for a full page reload.
When the form submission is not successful, turbo_stream format is rendered and the form is replaced(leaving the modal open and adding in the errors automatically)

Editing a post using the modal

This uses the same concepts as the new submission and starts off in index.html.erb as seen below.



<div class="align-middle min-w-full overflow-x-auto shadow overflow-hidden sm:rounded-lg" data-controller="post-modal" data-post-modal-prevent-default-action-opening="false">
  <table class="min-w-full divide-y divide-cool-gray-200">
    <thead>
    <tr>
      <th class="px-6 py-3 bg-gray-50 text-left text-xs leading-4 font-medium text-cool-gray-500 uppercase tracking-wider">Title</th>
      <th class="px-6 py-3 bg-gray-50 text-left text-xs leading-4 font-medium text-cool-gray-500 uppercase tracking-wider">Body</th>
      <th colspan="1" class="px-6 py-3 bg-gray-50 text-left text-xs leading-4 font-medium text-cool-gray-500 uppercase tracking-wider"></th>
    </tr>
    </thead>
    <tbody class="bg-white divide-y divide-cool-gray-200">
    <% @posts.each do |post| %>
      <tr>
        <td class="px-6 py-4 whitespace-nowrap text-sm leading-5 text-cool-gray-900">
          <%= post.title %>
        </td>
        <td class="px-6 py-4 whitespace-nowrap text-sm leading-5 text-cool-gray-500">
          <%= post.body %>
        </td>
        <td class="px-6 py-4 whitespace-nowrap text-sm leading-5 text-cool-gray-500">
          <%= link_to 'Edit', edit_post_path(post), class: "btn btn-secondary", data: { action: "click->post-modal#open", 'turbo-frame': 'post' } %>
        </td>
      </tr>
    <% end %>
    </tbody>
  </table>
</div>

The rest of edit and delete can be found in the example application

In closing

Disclaimer: The code above is far from perfect

In some places the implementation could definitely use refinement(feel free to suggest an improvement).

I chose a modal here as I had that particular issue to solve on a project and didn't see a ready-made solution/guide.

Overall I am happy with what Turbo enables and appreciate how the hard work of others to build these types of things, make it easier for everyone else to produce things quickly.

Setting up Rails with Docker and Caddyserver

Doug Stull — Wed, 26 Jun 2019 20:05:05 +0000

Goal

Show how to setup a basic implementation of Ruby on Rails with Docker, utilizing Caddyserver as the reverse proxy, tls, and load balancer.

For this tutorial, I will be using a simple demo Rails application, which you can find the source code for here. There is also a repo for the basic Caddyserver setup.

I'll break this down into 2 steps:

Create Caddyserver Docker image
Configure Rails and Caddyserver with Docker

Create Caddyserver Docker image

For this part we will go over the caddyfile, Dockerfile and docker-compose.yml files.

caddyfile

Note: all items below will reference the file linked above, and would be helpful if it were open in a separate window to reference.

Site/host declaration



https://localhost

for our simple demo, we'll just set this up for our local development. In production, this would be set to your dns name that is validated in your tls certificate

Proxy setup



proxy / http://rails:3000 http://rails2:3000

this will instruct Caddy to route all requests to our 2 containers running puma/rails, both listening on port 3000 for requests on the docker network where rails and rails2 are setup as aliases



transparent

according to the docs, this is shorthand for: ```

header_upstream Host {host}
header_upstream X-Real-IP {remote}
header_upstream X-Forwarded-For {remote}
header_upstream X-Forwarded-Port {server_port}
header_upstream X-Forwarded-Proto {scheme}

websocket

- according to the [docs](https://caddyserver.com/docs/proxy), this is shorthand for:

header_upstream Connection {>Connection}
header_upstream Upgrade {>Upgrade}

- in rails we would need this if using [Action Cable](https://guides.rubyonrails.org/action_cable_overview.html)

policy round_robin

- I changed this from the default of `random` to have a little more predictability, which helps when troubleshooting.

fail_timeout 30s
max_fails 1
try_duration 90s
health_check /stats?token=stats
health_check_interval 30s
health_check_port 9191
health_check_timeout 10s

- I was confused/couldn't wrap my head around the [official documentation](https://caddyserver.com/docs/proxy) on how this all worked together, so here is my attempt at explaining in a way that made sense to me...
   - fail_timeout: 
      - try for X amount of time before declaring a request failed and moving on to try another backend, kicking in the `try_duration`.
   - try_duration: 
      - after `fail_timeout` is reached for the first time, this will then pick up and will try and find another backend so that the request doesn't fail.
      - so if `fail_timeout` is 30s and `try_duration` is 90s, and both backends are down...it would be 2 minutes before you get a bad gateway (502) response from Caddy.
      - `try_duration` must be greater than `fail_timeout` + time to find backend and serve request(rails request part), else it will respond with 502 on the first response from a failed backend.
   - health_check:
      - hitting a `puma` control app to get status, therefore the port needs to be specifically set.  For more insight into this setting, see the backend rails app settings in the [puma config](https://github.com/dstull/docker-rails/blob/caddy/config/puma.rb#L60)

errors stdout
header / {
Strict-Transport-Security "max-age=31536000"
}
log / stdout "{combined} cache={cache_status}"
gzip
tls self_signed

- Docker prefers logging to `stdout`, so we'll do that here, adding some formatting for cacheing, in case we turn it on in the future.
- Caching is off for now due to this [issue](https://github.com/nicolasazrak/caddy-cache/issues/18) on the current Caddyserver image we are using.
- gzip content to speed everything up.  I thought there might be issues with this and Rails, but there wasn't and it provides great speed improvements in page load.
- use a simple self signed cert for this demo

## [Dockerfile](https://github.com/dstull/caddy_rails/blob/master/Dockerfile)
In this file, I will be highlight a few items that might not be strait forward.

RUN apk add --no-cache \
libcap \
&& \
:

RUN setcap cap_net_bind_service=+ep /usr/sbin/caddy

- install the alpine linux package libcap, which will enable us in the next line to grant binding privileges to the caddy binary.
- this will in turn allow us to run a caddy container as an unprivileged user(non root) on a port below 1024(443).

VOLUME /tmp

- if using the caddy [cache module](https://caddyserver.com/docs/http.cache), this needs to be writeable for the cache to be written.
   - note: we are running the container as [read only](https://nickjanetakis.com/blog/docker-tip-55-creating-read-only-containers) by default, therefore, any area that needs to have files written to it will need be declared as [volumes](https://docs.docker.com/storage/volumes/).

## [docker-compose.yaml](https://github.com/dstull/caddy_rails/blob/master/docker-compose.yaml)
In this file we are declaring some items that will help build and test out our app:

### build locally

15:25 $ docker-compose build caddy_rails
Building caddy_rails
Step 1/10 : FROM jumanjiman/caddy:v0.11.0-20181002T1350-git-3d0ba71
---> 6b039a312afc
Step 2/10 : USER root
---> Using cache
---> ab346bccfd06
Step 3/10 : COPY src/caddyfile /etc/caddy/caddyfile
---> Using cache
---> 27bcf28474ae
Step 4/10 : COPY src/init.sh /usr/bin
---> Using cache
---> 047528cc4a11
Step 5/10 : COPY src/healthcheck /var/opt/healthcheck
---> Using cache
---> d15a10faa15c
Step 6/10 : RUN apk add --no-cache libcap && :
---> Using cache
---> e0a9d0b2b44e
Step 7/10 : RUN setcap cap_net_bind_service=+ep /usr/sbin/caddy
---> Using cache
---> d23679349861
Step 8/10 : VOLUME /tmp
---> Using cache
---> 2e94a84380e6
Step 9/10 : USER caddy
---> Using cache
---> b2d343318687
Step 10/10 : ENTRYPOINT ["/usr/bin/init.sh"]
---> Using cache
---> 8b6a28c0ab20

Successfully built 8b6a28c0ab20
Successfully tagged caddy_rails:latest


### Run locally

15:25 $ docker-compose up -d
Creating network "caddy_rails_default" with the default driver
Creating caddy_rails_caddy_rails_1 ... done
15:26 $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
13e9cf1dd292 caddy_rails "/usr/bin/init.sh" 4 seconds ago Up 4 seconds 0.0.0.0:443->443/tcp caddy_rails_caddy_rails_1


# Configure Rails and Caddyserver with Docker
In this section, I am going to focus on the setup required for Rails to work with our Caddy setup, glossing over more specific Rails/Docker items at times.

## [Dockerfile](https://github.com/dstull/docker-rails/blob/caddy/Dockerfile)

ENTRYPOINT ["/web/script/entrypoint"]
CMD ["puma", "-C", "config/puma.rb"]

- set the entrypoint script, which will merely exec the command passed and allow it to take over [PID 1](https://tandrepires.wordpress.com/2016/11/15/the-importance-of-pid-1-in-containers/)

## [docker-compose.yml](https://github.com/dstull/docker-rails/blob/caddy/docker-compose.yml)
In this file, we setup our local build and runtime environments.

We define:
- 2 Rails instances so that caddy can load balance across `rails` and `rails2`.  We declare aliases on our networks for these instances, which then enables caddy to reference them as backends in the [caddyfile](https://github.com/dstull/caddy_rails/blob/master/src/caddyfile#L2)
- Basic Healthcheck settings for docker to determine the container health status as seen from `docker ps`

healthcheck:
  test: ["CMD", "curl", "http://localhost:3000"]
  interval: 10s
  timeout: 10s
  retries: 20

- I have configured the caddy_rails example above to automatically build images on update of master branch on [dockerhub](https://cloud.docker.com/repository/docker/hammer098/caddy_rails), and reference it now in the file.

image: hammer098/caddy_rails:latest


- On Caddy, expost port 443 to the underlying host

ports:
  - 443:443

- Finish setting up our dependency chain so that when we start everything with a `docker-compose up -d caddy`, it will start in sequence of `rails` -> `rails2` -> `caddy`

depends_on:
  - rails2


## Build

15:54 $ sdlc/build
Building rails
Step 1/11 : FROM hammer098/ruby_24
---> ba1ba3ccbaca
Step 2/11 : WORKDIR /web
---> Using cache
---> 1d49ff0018bb
Step 3/11 : COPY .ruby-version /web/.ruby-version
---> Using cache
---> f187344f9c6c
Step 4/11 : COPY Gemfile /web/Gemfile
---> Using cache
---> 5592d6874d91
Step 5/11 : COPY Gemfile.lock /web/Gemfile.lock
---> Using cache
---> 96ae9333a90e
Step 6/11 : RUN /bin/bash -l -c "bundle install"
---> Using cache
---> c593b6c4f41e
Step 7/11 : COPY . /web
---> 7ab2c7c6ac96
Step 8/11 : ENV TEMP /web/tmp
---> Running in 610cb512b3d1
Removing intermediate container 610cb512b3d1
---> ba400993482d
Step 9/11 : VOLUME /web/tmp
---> Running in aa2e71bc9236
Removing intermediate container aa2e71bc9236
---> 4489c90a69bb
Step 10/11 : ENTRYPOINT ["/web/script/entrypoint"]
---> Running in 9bb7c471c455
Removing intermediate container 9bb7c471c455
---> 602ff0fd2660
Step 11/11 : CMD ["puma", "-C", "config/puma.rb"]
---> Running in 99f2a4c59410
Removing intermediate container 99f2a4c59410
---> cc6672106b95
Successfully built cc6672106b95
Successfully tagged rails:latest

real 0m2.346s
user 0m0.328s
sys 0m0.098s

REPOSITORY TAG IMAGE ID CREATED SIZE
rails latest cc6672106b95 Less than a second ago 1.13GB


## Run

15:54 $ sdlc/run

Starting application container(s)

Creating network "docker-rails_railsnet" with driver "bridge"
Creating docker-rails_rails_1 ... done
Creating docker-rails_rails2_1 ... done
Creating docker-rails_caddy_1 ... done

15:55 $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a85a08b577c1 hammer098/caddy_rails:latest "/usr/bin/init.sh" 5 seconds ago Up 3 seconds 0.0.0.0:443->443/tcp docker-rails_caddy_1
20e44a42434a rails "/web/script/entrypo…" 7 seconds ago Up 4 seconds (health: starting) docker-rails_rails2_1
f07ac602d1d4 rails "/web/script/entrypo…" 18 seconds ago Up 16 seconds (healthy) docker-rails_rails_1


## Open Browser to see Rails running through Caddy
open to `https://localhost` and accept the certificate warning.

![](https://thepracticaldev.s3.amazonaws.com/i/9kk0w5fp20a2e7gzcdl9.png)

StimulusJS with Rails Action Cable and a bit of Sidekiq

Doug Stull — Fri, 31 Aug 2018 19:48:11 +0000

Goal

Show a simple implementation of Rails Action Cable with the modest js framework named StimulusJS.

For this tutorial, I will be using a simple demo Rails application, which you can find the source code for here.

I am going to gloss over the particulars of Sidekiq here, and focus on the StimulusJS and Action Cable pieces. I believe that will be most valuable here, as the other pieces have been covered many times on numerous blogs and tutorials. However, I may revisit the other items at a later date.

I'll break this down into 2 steps:

Create Sidekiq worker and Action Cable Channel
Setup StimulusJS

Create Sidekiq worker and Action Cable Channel

For this part, I want to show how an index page listing cars that have many drivers can be updated when a driver's name changes or the particular car they belong to.

To accomplish that for this example, I decided to make the Sidekiq job trigger off and after_touch callback on the Car model.

Car/Driver Models - the Active Record Models

class Car < ApplicationRecord
  has_many :drivers, dependent: :destroy

  after_touch :update_driver_names

  def drivers_list
    drivers.pluck(:name).join(',')
  end

  private

  def update_driver_names
    CarsWorker.perform_async(id)
  end
end

class Driver < ApplicationRecord
  belongs_to :car, touch: true

  delegate :name, to: :car, prefix: true
end

In the above file, I am triggering the after_touch callback named update_driver_names on the Car model by adding touch: true to the Driver model.
The update_driver_names method reaches out to Sidekiq and calls an async job called CarsWorker.perform_async, sending the id of the Car that the Driver has assigned.

CarsWorker (cars_worker.rb) - the Sidekiq Worker

class CarsWorker
  include Sidekiq::Worker

  def perform(car_id)
    # some contrived work...
    car                = Car.find car_id
    new_driver_changes = car.driver_changes + 1
    car.update_attribute(:driver_changes, new_driver_changes)

    car_drivers = car.drivers_list
    ActionCable.server.broadcast('cars', drivers: car_drivers, car_id: car_id, driver_changes: new_driver_changes)
  end
end

In the above file, I am:

Incrementing the driver_changes on the driver's car and committing that on the car.
Finding all the drivers for that car and broadcasting out to the Action Cable channel the new drivers list as a string in car_drivers, along with the number of driver_changes.

Here is an example of the transmission from the CarsChannel:

CarsChannel transmitting {"drivers"=>"Jacalyn Bauchblah,Wes Goodwin,Guy Keeling,Miss Pasquale Doyle,Candy Welch", "car_id"=>23, "driver_changes"=>4} (via streamed from cars)

CarsChannel (cars_channel.rb) - the Action Cable Channel definition

class CarsChannel < ApplicationCable::Channel
  def subscribed
    stream_from 'cars'
  end
end

The above is merely the standard boilerplate channel definition that is defined here.

Setup StimulusJS

For this part I will show the HTML erb pieces and the StimulusJS setup.

Cars Index (cars/index.html) - the HTML piece

<p id="notice"><%= notice %></p>

<div class="page-header" data-controller="cars">
  <h1>Cars</h1>
</div>

<table class="table table-hover">
  <thead>
  <tr>
    <th>Name</th>
    <th>Drivers</th>
    <th>Driver Changes</th>
    <th>Make</th>
    <th>Color</th>
    <th>Model</th>
    <th colspan="3"></th>
  </tr>
  </thead>

  <tbody>
  <% @cars.each do |car| %>
    <tr id="car_id_<%= car.id %>">
      <td><%= car.name %></td>
      <td class="cars--drivers"><%= car.drivers_list %></td>
      <td class="cars--driver-changes"><%= car.driver_changes %></td>
      <td><%= car.make %></td>
      <td><%= car.color %></td>
      <td><%= car.model %></td>
      <td><%= link_to 'Show', car %></td>
      <td><%= link_to 'Edit', edit_car_path(car) %></td>
      <td><%= link_to 'Destroy', car, method: :delete, data: { confirm: 'Are you sure?' } %></td>
    </tr>
  <% end %>
  </tbody>
</table>

<br>

<%= link_to 'New Car', new_car_path %>

The important part to point out in the above is the data-controller data attribute. Setting this to the name of the StimulusJS controller, cars, will then cause it to reach out and invoke the cars_controller.js connect function upon page render; subscribing the user to the cars action cable channel. There is documentation on the StimulusJS website explaining how that part works.

Cars Controller (cars_controller.js) - the StimulusJS Controller

import { Controller } from 'stimulus';
import createChannel from '../exports/cable';

export default class extends Controller {
  connect() {
    this.initChannel();
  }

  initChannel() {
    createChannel('CarsChannel', {
      received(data) {
        const carRow = $(`#car_id_${data.car_id}`);
        const driverChanges = carRow.find('.cars--driver-changes');
        const drivers = carRow.find('.cars--drivers');
        driverChanges.text(data.driver_changes);
        drivers.text(data.drivers);
      },
    });
  }
}

In the above, we:

Importing the basic cable setup from exports/cable.js
Init the Channel from the connect function, which is fired whenever we land on the cars index page due to the data-controller data attribute.
Update the cars index page when a messages is received on the CarsChannel.

Cable Javascript (cable.js) - the basic Action Cable JS setup that is imported when needed

import cable from 'actioncable';

let consumer;

export default function (...args) {
  if (!consumer) {
    consumer = cable.createConsumer();
  }

  return consumer.subscriptions.create(...args);
}

The above is the initial/standard Action Cable setup. I went the 'extra mile' here and also used yarn to install actioncable, ensuring it was the same version as Rails. This helps keep me completely out of the asset pipeline/sprockets area.

In Closing...

I particularly wanted to get completely out of the Rails asset pipeline/sprockets setup and be page specific about my channel subscription.

I hope this short demo is helpful to someone. I searched many places to gather the bits and pieces of how to string this together, and felt I should share with the community that I have benefited so much from myself. I had only a few hours to throw this together before I had to get back to being a parent :) ...perhaps later I will add a blog on how I went about implementing testing all of this from soup to nuts.

The basis for some of this work came from this wonderful blog by Evil Martians.

A simple Stimulus table filter demo

Doug Stull — Sun, 04 Feb 2018 13:01:03 +0000

Goal

Show the before and after implementation of table filtering with the new modest js framework named stimulus

We'll focus on a simple page with a table and filter capability.

I'll break this down into 2 steps:

Implementation without Stimulus
Implementation with Stimulus

Implementation without Stimulus

To accomplish this part I created a simple project. There are 2 files that are involved:

HTML File

CoffeeScript file

Here I am demonstrating the filtering of table rows when a user presses enter.

Implementation with Stimulus

To accomplish this part I utilized the master branch. This commit shows the diff of the conversion. Final demo app can be found here.

There are 3 files that are involved:

HTML File

The notable changes here to notice are the utilization of the data attributes of target, controller, and action. The action specifically defines the javascript event keyup and the controller and function it will invoke.

Application JS file

Not much to say about this file, as it is the basic wiring needed as the entry-point for Stimulus use.

Hello Controller JS file

Here I am demonstrating the conversion into Stimulus from original CoffeeScript.

Conclusions

Stimulus really shines when you can use it to describe the event handlers in the data attributes instead of having to deal with them in JavaScript. If you are using server side rendering of html, this framework can really help you organize your javascript and manage events.

Stimulus with Turbolinks, select2 and formValidation

Doug Stull — Sat, 03 Feb 2018 23:14:08 +0000

Goal

Show the before and after implementation of select2/formValidation/Turbolinks with the new modest js framework named stimulus

I am utilizing Rails as my framework here, but the setup really isn't about the framework. However, this demonstrates a good use case of server side rendered html without frontend framework separation.

I'll break this down into 2 steps:

Implementation without Stimulus
Implementation with Stimulus

Implementation without Stimulus

To accomplish this part I created a simple project. There are 2 files that are involved:

Erb File

In this file I am following the declarative/html5 data attributes way of using FormValidation as I prefer to keep the javascript as generic as possible.

CoffeeScript file

Here I am demonstrating:

Select2 with normal, multiple, and tags with createTag as a way to constrain allowable input
FormValidation on Select2
FormValidation with custom callback on name field
FormValidation using revalidation and hidden input change when select2 change event is triggered
Removing Select2 from DOM before Turbolinks caches the page, which keeps double Select2's from showing up in the page

Implementation with Stimulus

To accomplish this part I utilized the master branch. This commit shows the diff of the conversion. Final demo app can be found here.

There are 3 files that are involved:

Erb File

The notable changes here to notice are the utilization of the data attributes of target, controller, and placeholder.

Application JS file

Not much to say about this file, as it is the basic wiring needed as the entry-point for Stimulus use.

Cars Controller JS file

Here I am demonstrating:

Conversion into Stimulus from original CoffeeScript
Additional storing of the selected items so they are preserved on forward and back buttons

Conclusions

I really like how this gives an easy way to leverage data attributes and hook easily into Javascript. While I didn't show it here, but will in another blog, Stimulus really shines when you can use it to describe the event handlers in the data attributes instead of having to deal with them in JavaScript.

I'd encourage people to give it a look. As can be seen here, you can easily hook into existing jquery libraries using it.

Thanks to this repo for getting me past a few initial humps.

Docker + Rails + System tests with Headless Chrome

Doug Stull — Sun, 12 Nov 2017 12:50:17 +0000

Goal

Show a simple setup of headless system testing inside a container with Ruby on Rails.

There are likely many ways to achieve this goal. However, my aim was to hook into an eventual CI system where the Rails application could be validated by building the docker image and running the test suite.

I'll break this down into 3 steps:

Create docker image with CentOS and ruby 2.4
Create docker images for Rails and headless chrome testing
Configure Rails for headless chrome system testing

Create docker image with CentOS and ruby 2.4

To accomplish this part I created a simple project which provides a base image for the Rails application. I decided to create this base image because I wanted to stick with CentOS, which at the time of creating, I wasn't able to find on Docker Hub for ruby 2.4.

Dockerfile

FROM centos:centos7

# Install the appropriate software
RUN yum -y update && yum clean all
RUN yum -y install \
            epel-release \
        which \
            && \
        yum clean all \
            && \
        :

# Install rvm, default ruby version and bundler.

RUN gpg --keyserver hkp://keys.gnupg.net --recv-keys D39DC0E3
RUN /bin/bash -l -c "curl -L get.rvm.io | bash -s stable"
RUN /bin/bash -l -c "echo 'source /etc/profile.d/rvm.sh' >> /etc/profile"
RUN /bin/bash -l -c "rvm install 2.4.2"
RUN /bin/bash -l -c "rvm cleanup all"
RUN /bin/bash -l -c "echo 'gem: --no-ri --no-rdoc' > .gemrc"
RUN /bin/bash -l -c "gem install bundler --no-ri --no-rdoc"

I then created a repository on Docker Hub, and setup an automated build against the Github repo. This allows it to automatically rebuild the image when changes are pushed to Github.

Create docker images for Rails and headless chrome testing

Now that the base image for ruby is built and available on Docker Hub, I can use it as the base in my example Rails application.

Here starts the tricky part:

I wanted to keep the headless chrome pieces outside of the Rails application image that would go to production. In order to do that, there are a few ways this can be accomplished:

Setup a container side by side with my application, and integrate using prebuilt Selenium containers. I initially tried this, with the help of some Docker pros, but the test suite speed was just too slow (non headless setup/impatient developer/didn't try too hard to diagnose).
Setup a container with chrome installed along with the Rails application. I ended up choosing this route, while accepting that the application image and the testing application image were not exactly the same.

Rails Application image setup

In a build script I performed the steps that would create the needed Dockerfile at runtime, and execute the build of the image. This allows the specific application build steps to be kept common to this image and the test image. Note that the Dockerfile should not be checked into the repo, but is in this application for demonstration purposes.

Build script(rails image function)

build_rails() {
  cat Dockerfile.prefix.in \
  sdlc/Dockerfile.in \
  > Dockerfile

  time docker-compose build \
  rails
}

As seen above, I am concatenating the Ruby base piece (Dockerfile.prefix.in) with the common Rails part (sdlc/Dockerfile.in), and then into the final Dockerfile that acts as the input for the docker-compose build. In a real world setup the Dockerfile 'in' file pieces maybe a bit larger, and have a postfix with variables defined to tag the image.

Final Dockerfile

FROM hammer098/ruby_24
# common rails build steps
WORKDIR /web

# Install rvm, default ruby version and bundler.
COPY .ruby-version /web/.ruby-version
COPY Gemfile /web/Gemfile
COPY Gemfile.lock /web/Gemfile.lock

RUN /bin/bash -l -c "bundle install"

COPY . /web

Chrome + Rails Application image setup

In the build script similar steps were performed to craft the Dockerfile for the test image at time of build.

build_rails_test() {
  cat sdlc/Dockerfile.rails-test.in \
  sdlc/Dockerfile.in \
  > sdlc/Dockerfile.rails-test

  time docker-compose build \
  rails-test
}

In this function similar steps are being performed as in the Rails application build. The steps unique to the test image are included in sdlc/Dockerfile.rails-test.in. The common Rails part are then added, and sdlc/Dockerfile.rails-test is created to be used by the rails-test image. Do not add sdlc/Dockerfile.rails-test to the repo as it is created 'on the fly' by this process.

Final Dockerfile.rails-test

FROM hammer098/ruby_24

RUN yum -y install \
            chromedriver \
            chromium \
            gnu-free-sans-fonts \
            xorg-x11-server-Xvfb \
            && \
            yum clean all \
            && \
            :
# common rails build steps
WORKDIR /web

# Install rvm, default ruby version and bundler.
COPY .ruby-version /web/.ruby-version
COPY Gemfile /web/Gemfile
COPY Gemfile.lock /web/Gemfile.lock

RUN /bin/bash -l -c "bundle install"

COPY . /web

Configure Rails for headless chrome system testing

For this part, I needed to configure 2 pieces:

Rails test configuration

In this part we need to setup the headless chrome driver and enable specific chrome options.

application_system_test_case.rb

require 'test_helper'
require 'selenium-webdriver'
require 'capybara'

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase


  Capybara.register_driver :headless_chrome do |app|

    # options explained https://peter.sh/experiments/chromium-command-line-switches/
    # no-sandbox
    #   because the user namespace is not enabled in the container by default
    # headless
    #   run w/o actually launching gui
    # disable-gpu
    #   Disables graphics processing unit(GPU) hardware acceleration
    # window-size
    #   sets default window size in case the smaller default size is not enough
    #   we do not want max either, so this is a good compromise

    capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
        chromeOptions: { args: %w[no-sandbox headless disable-gpu window-size=1400,1400] }
    )

    Capybara::Selenium::Driver.new(
        app,
        browser:              :chrome,
        desired_capabilities: capabilities
    )
  end
  driven_by :headless_chrome
end

The corresponding example tests that use this configuration can be found here. An explanation of why I chose to use Rails system tests over rspec can be found here.

Xvfb install and Docker entrypoint setup for test execution

In the setup at work we needed xvfb installed for the tests to function properly. However, we were also using an older chrome version(v59). On this example setup I wasn't able to prove out that xvfb was required. This was either due to the newer version of chrome (v62), or a less complex testing setup.

Setup:

Install the xvfb-run rpm as seen above in the test docker image.
Modify the entrypoint directive in docker-compose.yml to include xvfb-run and rails test execution steps.

docker-compose.yml rails-test-run definition

  rails-test-run:
    image: rails-test
    entrypoint:
      - bash
      - --login
      - xvfb-run
      - rails
      - test:system
      - RAILS_ENV=test

In the above I am telling docker via the command docker-compose run --rm rails-test-run to interactively login (bash --login), run xvfb, execute rails system tests, then exit. The RAILS_ENV variable is set to test so that Rails doesn't by default first look at the development db setup.

Here is an example output of a test run utilizing a test script:

docker-rails$ sdlc/test
Starting rails-test, running and exiting

Run options: --seed 22514

# Running:

Puma starting in single mode...
* Version 3.10.0 (ruby 2.4.2-p198), codename: Russell's Teapot
* Min threads: 0, max threads: 1
* Environment: test
* Listening on tcp://0.0.0.0:35683
Use Ctrl-C to stop
...

Finished in 8.587443s, 0.3493 runs/s, 0.4658 assertions/s.
3 runs, 4 assertions, 0 failures, 0 errors, 0 skips

Conclusion

I found this setup to be the one that worked in our eventual goal of hooking into a CI system.

The setup, as shown here, is far from complete for a Rails application destined for production. Here are some of the things that I believe are out of scope and were left out of this article and the example application:

Nginx/Unicorn install and setup
This container is running as root, in our production setup, we run as a non root user.
Setting container to read-only and adding VOLUME directives in the Dockerfile for certain areas Rails needs for writing. Changing of permissions/ownership to those files and directories at build time.
Mysql containers for test/development.
Environment variable breakout.
Handling of compiled assets and js runtime env. ...many more

My sins against Webpacker/Yarn and compiled Rails assets

Doug Stull — Fri, 14 Jul 2017 14:31:31 +0000

What did I do...

Used webpacker gem in Rails 5.x, and then checked all the webpacker/js packages into git that were installed by yarn.
Compiled Rails assets, while still in dev, and checked them into git.

Why did I do it?

For Webpacker/Yarn, even though it has a yarn.lock file and such, I didn't see a reason not to. Although you could make 100 reasons for this being 'wrong'. Plus, outside of our macs, gaining access to the outside world, installing yarn, etc, was close to impossible(well not impossible, just probably not worth it if we had another way). So if we were to develop anywhere outside of our macs, we'd have issues...of course, we wouldn't be able to download new packages anyway at that point if not on our macs...hmmm
For the Rails assets, since yarn install and webpacker:compile is now 'baked in' to the rake assets:precompile command, we were stuck. It is a monumental task to get yarn/node installed past our dev environment.

How to make this work with precompiled assets in git and still develop like normal with uncompiled assets.

The webpacker install and actually using it can be found on the webpacker github page.

Force development and test to still use uncompiled assets as normal.

Inside the config/environments/development.rb and test.rb add config.asssets.prefix directive to override the defaults:

  # this allows us to precompile for non dev and commit, yet still use uncompiled for dev
  config.assets.prefix = '/dev-assets' # can be anything really

How this works: since we will compile to the usual assets path, development and test will not find compiled assets and will default back to the uncompiled ones.

If assets change, now you need an extra step before checking into git:

Note: Do this After running tests...this part is important as I've seen the manifest.json get overwritten and pointing back to the webpacker local server on port 8080.
In your terminal:

RAILS_ENV=production rake assets:precompile

Submit files into git as normal. Since production env is setup by default when webpacker is installed, that is the optimal env to use for this, even if you have more than one type of 'production' environment. Of course there are some that would say you should just have 3 environments in rails and use ENV vars to control everything else...depends on your setup/perspective, but I digress...

Conclusion/Thoughts:

This seems to be working so far, although there are some that would say I am breaking some cardinal rules of yarn/assets/etc, and some that probably say 'yeah I did that too'. All I know is we needed a solution for the closed environment our apps live in, and this seemed to be a viable solution. I didn't come up with this solution on my own, a bunch of googling, reading rails docs, and putting it all together here... I hope this spurs some fruitful conversation and perhaps a more elegant solution than this one for the closed from the outside world/hard to get anything installed outside development/using your own infrastructure and not AWS/Heroku world.

One other thing to mention...I believe this will help ease any transition we have into docker as well. I've seen some strategies for compiling and maintaining assets for docker, and they all seem messy. I believe this one will keep the docker image immutable and provide us with less friction between environments.

My journey from Rspec feature tests to Rails system tests

Doug Stull — Thu, 13 Jul 2017 00:30:41 +0000

Goals:

Achieve reliable headless testing with system tests that utilize javascript
Learn the new Rails system test setup

Quite often I find myself spending hours debugging(and sometimes finally deleting) a flaky feature test. Finally I decided to stop doing that, and after watching the Rails Conf talk about system testing, decided it was time to act.

Since my first goal is to always keep it simple, I didn't jump right away into the Rails system tests. Due to the rest of my testing suite being written in Rspec, I first tried to go headless with that setup.

Rspec failure(s)

Trying headless chrome with capybara

spec/rails_helper.rb

Capybara.register_driver :headless_chrome do |app|
  capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
    chromeOptions: { args: %w(headless disable-gpu) }
  )

  Capybara::Selenium::Driver.new app,
    browser: :chrome,
    desired_capabilities: capabilities
end

Capybara.javascript_driver = :headless_chrome

Result:

Selenium::WebDriver::Error::unknownError...basically it gave me the 'is not clickable at point' error. I tried working around these by doing simple fixes to capybara finders, digging more for answers, and in the end(about 30 minutes), couldn't get it resolved. So I moved on to the next attempt...

Trying capybara webkit

Result:

Well this didn't go very far as it needed a Qt dependency, and when I tried to brew install that, the compiler barfed due to, what seems to be, Xcode upgrade dependency. Due to not having admin privileges on my mac, I took the easy way out and abandoned webkit for now.

Trying poltergeist

spec/rails_helper.rb

require 'capybara/poltergeist'

Capybara.javascript_driver = :poltergeist

Modifying some tests for known 'click_on' issues:

Before:

click_on 'Create Engine'

After:

find("input[type=submit][value='Create Engine']").trigger('click')

Result:

Success! ...or so I thought. All features passed. However, when ran with entire test suite, they randomly failed. So I tried fixing them with different swap-outs for 'have_content' checks and such. Those then passed, ran again, they failed and the others(that I didn't fix) passed. At this point I was thoroughly frustrated, because either I run my test suite for features separately from other tests, or I need to fix this...
Note: I assume this has to due with the database cleanup and different AR threads and such that I've read about. I did have my database cleaner setup to truncate on js tests, but when ran in headless mode with the rest of my test suite, it was flaky. Also, before going down this headless path, they worked fine.

Being fed up with flaky tests and 3 failed attempts at remedying this in Rspec, I decided to now move on and try the Rails System Tests...

Rails System Tests:

Setup:

Prerequisites:

Rails 5.1
Poltergeist
Mocha
Puma
Some other gems I like: FactoryGirl/Capybara(of course), standard test setup

test/test_helper.rb

Simple setup really, only added a few helpers and such:

require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require 'mocha/test_unit' # for stubs
require 'minitest/spec' # for let/it/etc

# not sure about this, but rails kinda does it here...feels sloppy, but I hate
# typing require in all my test files
Dir[Rails.root.join("test/support/**/*.rb")}.each { |f| require f }

class ActiveSupport::TestCase
  # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
  fixtures :all

  # Add more helper methods to be used by all tests here...
  extend MiniTest::Spec::DSL
  include WaitForAjax # hack for remote js calls that waits for ajax to be done
end

test/application_system_test_case.rb

This file comes with Rails 5.1, but I switched it to use poltergeist:

require 'test_helper'
require 'capybara/poltergeist'

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  driven_by :poltergeist
end

test/system/app/my_map_test.rb

This is just one of the test files, but good enough to show the setup
I like to abstract away a lot of the standard Capybara DOM traversal steps, so those will be in the next file.

require 'application_system_test_case'

class MyMapTest < ApplicationSystemTestCase
  class StandardType < MyMapTest
    let(:subject) { MyMapForm.new('standard_type') }

    test 'page has different types' do
      subject.visit_page

      assert has_content?('first type')
      assert has_content?('second type')
      assert has_content?('third type')
    end

    test 'page has address link' do
      subject.visit_page
      assert has_link?('Address Import')
    end
  end
end

Note: this was originally written in Rspec, but the conversion was simple...

s/describe/class, s/it/test, s/expect..to/assert, s/have_content/has_content?..etc

test/support/forms/my_map_form.rb

This is my file to hold all the repeatable/dirty details about traversing the DOM

class MyMapForm
  include Capybara::DSL
  include FactoryGirl::Syntax::Methods
  include Rails.application.routes.url_helpers

  def initialize(type)
    # using factory here of course and converting to find the factory
    @item = create(type.to_sym)
    # one more...
    create(type.to_sym)
    @app = item.app
  end

  def visit_page
    # don't care about authenticating here
    # in rspec, this uses allow_any_instance_of...
    ApplicationController.any_instance.stubs(:require_long).returns(true)
    ApplicationController.any_instance.stubs(:can_edit?).returns(true)

    visit app_path(app)
    find('a', test: 'My Map').click # in non headless, click_on 'My Map' works
    self
  end

  private 

  attr_reader :item, :app
end

Conclusions:

Pros:

It seems to work so far...and let's be honest, that is the most important thing.
Default functionality of a screenshot in my terminal on failure. Basic, but nice.
Seems to solve the multiple db connection issue, and therefore reducing the flakiness of these gui driven system tests
Runs faster than non headless implementation
I suppose it makes running tests in CI env easier, as we can skip some of the browser setup?
Gives the team some exposure to Rails test framework (this can also be seen as a con below)
By using a different test suite for system tests, it gives a clear line of delineation in configs and troubleshooting, etc.
Is part of Rails core, so I have full confidence in the future support and improvements

Cons:

Adds technical debt to the team since we do everything else in Rspec
Documentation/examples for Rspec is miles ahead from what I've been able to find for Rails System tests (hence some of the reason for this writeup)

Final Thoughts:

While this journey into Rails System tests feels good today, it is quite possible I will look back in a month or so and wonder...what was I thinking using this and multiple test suites... But then again, don't we always look back and have those thoughts about our development decisions?

Of note, I also see that rspec-rails, is planning on supporting this soon as well, I think, as seen by this issue. If that comes to fruition, it will be worth a look, and definitely worth consolidating back into one test suite...we'll see.

Forem: Doug Stull

How to use modals with forms in Rails using Turbo

Goal

Posts page being referenced

Creating a new post using the modal

New link and modal setup

Clicking the New Post link

Rendering the response

Submitting the form

Editing a post using the modal

In closing

Setting up Rails with Docker and Caddyserver

Goal

Create Caddyserver Docker image

Site/host declaration

Proxy setup

StimulusJS with Rails Action Cable and a bit of Sidekiq

Goal

Create Sidekiq worker and Action Cable Channel

Car/Driver Models - the Active Record Models

CarsWorker (cars_worker.rb) - the Sidekiq Worker

CarsChannel (cars_channel.rb) - the Action Cable Channel definition

Setup StimulusJS

Cars Index (cars/index.html) - the HTML piece

Cars Controller (cars_controller.js) - the StimulusJS Controller

Cable Javascript (cable.js) - the basic Action Cable JS setup that is imported when needed

In Closing...

A simple Stimulus table filter demo

Goal

Implementation without Stimulus

Implementation with Stimulus

Conclusions

Stimulus with Turbolinks, select2 and formValidation

Goal

Implementation without Stimulus

Implementation with Stimulus

Conclusions

Docker + Rails + System tests with Headless Chrome

Goal

Create docker image with CentOS and ruby 2.4

Dockerfile

Create docker images for Rails and headless chrome testing

Rails Application image setup

Build script(rails image function)

Chrome + Rails Application image setup

Final Dockerfile.rails-test

Configure Rails for headless chrome system testing

Rails test configuration

Xvfb install and Docker entrypoint setup for test execution

docker-compose.yml rails-test-run definition

Conclusion

My sins against Webpacker/Yarn and compiled Rails assets

What did I do...

Why did I do it?

How to make this work with precompiled assets in git and still develop like normal with uncompiled assets.

Force development and test to still use uncompiled assets as normal.

If assets change, now you need an extra step before checking into git:

Conclusion/Thoughts:

My journey from Rspec feature tests to Rails system tests

Goals:

Rspec failure(s)

Trying headless chrome with capybara

spec/rails_helper.rb

Result:

Trying capybara webkit

Result:

Trying poltergeist

spec/rails_helper.rb

Modifying some tests for known 'click_on' issues:

Before:

After:

Result:

Rails System Tests:

Setup:

Prerequisites:

test/test_helper.rb

test/application_system_test_case.rb

test/system/app/my_map_test.rb

test/support/forms/my_map_form.rb

Conclusions:

Clicking the `New Post` link