Forem: Rangle.io

Van Gogh, InVision Design System Manager and a new NPM package

cdepaul — Mon, 20 Apr 2020 19:55:18 +0000

“Great things are not done by impulse, but by a series of small things
brought together. And great things are not accidental, but must
certainly be willed.” — Vincent van Gogh

Last month, Rangle became an InVision Partner and the developers at Rangle leapt at the opportunity to play around with InVision’s Design System Manager (DSM).

After building a sample project, I created invision-dsm-utils, a Node.js package to support front-end developers with:

Downloading InVision DSM design tokens into their projects, and
Transforming those design tokens into JavaScript objects that meet the Styled System Theme Specification.

In this article, I’ll take you through the use cases for the scripts and how you can integrate this library into your development process.

Sample Project

As a developer in Amsterdam, I have been looking for an excuse to work on a side-project that could incorporate Vincent van Gogh — one of the Netherland’s most famous exports — in some way. With InVision DSM in mind, I came up with a small app idea that could be designed in Sketch, uploaded to InVision DSM, and developed in React.

I called this project, WhereVanGogh: An app that provides the museum location of publicly available van Gogh works of art.

Sample Design

Here is the design I made for a Painting page, developed in Sketch and uploaded to InVision DSM. Note, the images have a public-domain license from museums like Rijkmuseumenter link description here and The Metropolitan Museum of Art.

Integrating design tokens into code base

Designing the page layout in Sketch in a manner that would integrate with InVision DSM was straight forward and well supported by InVision’s excellent documentation (see below).

When it came time to develop my React components, I knew I wanted to:

Utilize the design tokens provided by InVision DSM
Build the components using styled-components and styled-system

Out of the box, achieving this would require some manual effort; and that’s not fun for anyone.

Specifically, the two manual interventions were:

Integrating the design tokens into the code base: The DSM provides you with nine URLs to different design token file types, leaving it up to you to decide how to integrate them into your project (i.e. download and distribute them in your bundle).
Transforming the design token schema into a Styled System theme: The design token files use a schema that is not the same as the styled-system theme schema, so manual mapping from one schema to the other is required.

Developing some handy Node scripts

I sought to solve this problem through a couple of Node.js scripts that Rangle has now released with an MIT license and has made available on the NPM registry. The package is called invision-dsm-utils and it currently contains two scripts:

Download: Download design token files from InVision DSM
Transform: Transform those design tokens into a JavaScript object that meets the Styled System Theme Specification

Installation

npm install --dev invision-dsm-utils

Download design tokens in JSON

invision-dsm-utils download json ./src

Set-up configuration file .invisiondsmutilsrc

{
    dsmExportUrl: string,
    key: string
}

Transform JSON into JavaScript object that meets the Styled System Theme Specification

invision-dsm-utils transform ./src/style-data.json ./src/theme.dsm.js

Use scripts in development flow

In my React app, I added these scripts into my package.json as follows:

{
 "scripts": {
    "tokens:download": "invision-dsm-utils download json ./src",
    "tokens:transform": "invision-dsm-utils transform ./src/style-data.json ./src/theme.dsm.js"
  }
}

You can be even more creative with this by triggering these scripts on a postinstall.

Use theme in App

import React, { Component } from 'react';
import { ThemeProvider } from 'styled-components';
import themeDSM from './src/theme.dsm';
// You could add additional values to themeDSM by
// extending the object
const theme = {
   ...themeDSM,
   space: [0, 4, 8, 16, 32, 64, 128, 256, 512],
};
class App extends Component {
  render() {    
    return (
      <ThemeProvider theme={theme}>
        <div>Example</div>
      </ThemeProvider>
    );
  }  
}
export default App;

Contribute to package

invision-dsm-utils is an open source project and we encourage you to submit pull requests to improve and extend this library for your InVision DSM needs.

About Rangle and InVision partnership

Rangle and InVision are proud to announce a strategic partnership, enabling Rangle to advise InVision’s clients on successful approaches to Design Systems and Design Operations. This new offering will cover a wide range of topics, including building and maintaining a design system using InVision Design System Manager, ensuring wide adoption, and methods to improve collaboration between designers and developers.

Styled Components, Styled Systems and How They Work

cdepaul — Thu, 12 Dec 2019 15:48:55 +0000

Sometimes the libraries that we use daily can seem a little bit like magic. By taking the time to understand the fundamental JavaScript features that make these libraries possible, we become better equipped to improve our use of those libraries, applying their functionality in unique and relevant ways.

In React, there are two libraries that work together to make the creation of presentational components very simple: styled-components and styled-system.

Here is an example of these libraries working together, courtesy of the styled-system documentation.

import styled from 'styled-components'
import { color } from 'styled-system'


const Box = styled.div`
  ${color}
`

This code created a React component named Box that receives the props color and bg.

<Box color="#fff" bg="tomato">
  Tomato
</Box>

In this article I am going to cover:

Generating React Components with Tagged Templates
Implementing a simple version of Styled Components
Diving into how styled-components and styled-systems actually operate together

Template Literals

In order to understand how styled-components and styled-systems work, it's best to first understand where these libraries get their power from: Template Literals.

The most common use-case of Template Literals is string concatenation.

// Template Literal
const string = `I am a template literal, because I am enclosed by the back-tick`;

// Template Literal with placeholder
const expressions = 'dynamic values';
const string = `I can contain ${expressions}`;

Template Literals are clean, they are effective and they have been so well adopted for string concatenation that some of us (e.g. me) don’t even care how this syntax works, or realize that its functionality can be expanded upon.

Template Literals and the preceding function

When we implement Template Literals, something unintuitive happens: The contents of the Template Literal (e.g. the text and placeholders) are passed to a function.

Which function? In the two examples above, a default function with the job of concatenating the text and placeholders into a single string.

const expressions = 'dynamic values';
const example = `I can contain ${expressions}`;

console.log(example);  // >> I can contain dynamic values

But Template Literals are not confined to performing only string concatenations. JavaScript lets us use our own custom functions to do whatever we want with the text and placeholders within the Template Literal. This type of function is called a Tag and to use it, you simply reference the function name — the Tag — in front of the Template Literal. Doing this results in the creation of a Tagged Template.

For example, here is a simple function that does not accept any parameters and prints a static message to the console.

const printStaticMessage = () => { console.log('My static message'); }

We can invoke this function in two ways: As a traditional function call and as a Tagged Template.

printStaticMessage();  // >>> My static message
printStaticMessage``;  // >>> My static message

Notice that each invocation generated the same result. So, we can conclude that Tagged Templates are just an alternative way to invoke a function.

Using the Template Literal as Arguments for the Tag

A more useful Tagged Template would utilize the text and placeholders within the Template Literal. Let’s create a Tag that prints out its arguments.

const printArguments = (...args) => { console.log(...args); }

const var1 = "my";
const var2 = "message"

printArguments`This is ${var1} custom ${var2}!`;
// 
//   ["This is "," custom ","!"],
//   "my",
//   "message"
//

The first argument is an array of strings that represent the text in the Template Literal, separated into elements based on the location of the placeholders.

The remaining n arguments are strings with the value of each placeholder, ordered based on when they are defined in the Template Literal.

Knowing that these are the arguments that Tags receive, we can guess what the implementation of the default Template Literals concatenation function looks like:

const defaultFunction = (stringArray, ...values) => {
  return stringArray.reduce((acc, str, i) => {
    return values[i] ? acc + str + values[i] : acc + str;
  }, '');
}

const var1 = "my";
const var2 = "message"

const example = defaultFunction`This is ${var1} custom ${var2}!`;
console.log(example);  // >>> This is my custom message!

Passing functions into the Tagged Template

Since the Tag is simply receiving the Template Literal placeholders as argument variables, those variables can contain any JavaScript object, like a number, string or function. Here is an non-useful example where we pass a function in the Template Literal placeholder and execute it in the Tag.

const executeFirstPlaceholder = (textArray, placeholder) => {
  placeholder();
}

executeFirstPlaceholder`${() => { console.log('first placeholder')}}`; 

// >>> first placeholder

Returning functions from a Tagged Template

Like any other function in JavaScript, a Tagged Template can return objects, strings, numbers, and, of course, functions.

const multiply = (multiple) => (num) =>
  parseInt(num[0]) * parseInt(multiple[0]);

const double = multiply`2`
const result = double`4`;

console.log(result);  // >>> 8

Making the leap to React

React’s “function components” are very simply JavaScript functions that can be rendered into the DOM. Here is an example of a Tagged Template returning a React function component.

const withGreeting = ([greeting]) => 
  ({ name }) => <h1>{greeting}, {name}!</h1>;

const Greet = withGreeting`Greetings`;

// Render component
<Greet name="Chris" />

// Renders in DOM
<h1>Greetings, Chris</h1>

This is the crux for how Styled Components generate React components.

Styled Components

Utilising tagged template literals (a recent addition to JavaScript) and the power of CSS, styled-components allows you to write actual CSS code to style your components. It also removes the mapping between components and styles — using components as a low-level styling construct could not be easier!
https://www.styled-components.com

Styled Components use Tagged Templates to return React components.

In the following example, styled.h1 is used to create a simple React component containing a <h1> HTML tag, display using the CSS styles specified within the Template Literal

import styled from 'styled-components';

const Title = styled.h1`color: blue;`;

// Render
<Title>Regular Title</Title>

// Renders to DOM
<h1 style="color: blue;">Regular Title</h1>

The styled object contains keys named after common HTML tags — like H1, H2, and div. These keys reference a function that can be used as the Tag in a Tagged Template.

A simple implementation of styled.h1

Let’s try to make a simple implementation of styled.h1. At it's simplest, the styled.h1 function receives the CSS styles in the back-ticks and transforms them into a style object that it attaches to the underlying element (e.g. h1).

const styledH1 = ([styles]) => ({ children })=> {
  const lines = styles
                  .split(';')
                  .map(line => line.trim())
                  .filter(line => line !== "")

  // Function courtesy of mck89 on StackOverflow
  const convertToCamelCase = (key) =>
    key.replace(/-([a-z])/g, (x, up) => up.toUpperCase())

  const style = lines.reduce((acc, line) => {
    const lineParsed = line.split(':');
    const key = convertToCamelCase(lineParsed[0]);
    const val = lineParsed[1];
    return { ...acc, [key]: val };
  }, {});

  return <h1 style={style}>{children}</h1>
}

const H1 = styledH1`
  color: red;
  font-size: 18px;
`;

// Render
<H1>Hello</H1>

// Renders in DOM
<h1 style="color: red; font-size: 18px;">Hello</h1>

At this point, the style we are passing to the function is hard coded and fixed; not able to dynamically change, based on the prop values the component receives.

Let’s look at how the ability to pass functions into our tagged templates can enable things to become more dynamic.

Using functions to access Props in Template Literals and Styled Components

As discussed we discussed, a function passed to a Template Literal placeholder can be executed. Styled Components utilize this feature to generate dynamic styles.

import styled from 'styled-components';

const Button = styled.button`
  color: ${ (props) => props.primary ? 'blue' : 'red' };
`;

class StyledComponentsDemo extends Component { 
 render() {
   return(
      <>
        <Button primary>Primary</Button>
        <Button>Secondary</Button>
        <Button primary 
                onSubmit={() => handleSubmit()}>Submit
        </Button>
      </>
    )
  }
}

When the Styled Component is rendered, each function in the Template Literal is passed the component’s props and those props can be used to impact the presentation of the component.

Note: not all props passed to a Styled Component need to impact the presentation (e.g. onSubmit); they could also be used only by the underlying HTML element.

Styling regular, custom components

Styled Components allow you to style any custom component that you’ve created. First, the custom component must receive the prop className and pass it to the underlying DOM element. Once that is done, pass the custom component to the styled function and invoke it as a Tagged Template to receive a new Styled Component.

import styled from 'styled-components';

const Button = ({ className, children }) => 
  <button className={className}>{children}</button>

const ButtonBlue = styled(Button)`color: blue`;

// Render
<ButtonBlue>Blue Button</ButtonBlue>

Styling the Styled Components

Styled Components uses the CSS preprocessor stylis, supporting SCSS-like syntax for automatically nesting styles.

const Thing = styled.button`
  color: black;

  :hover {
    color: blue;
  }
`

Within SCSS syntax, & references the current component. You can also reference other components like you would reference any other type of selector (e.g. .class or #id) by simply referencing ${OtherComponentName} , but only if it is a Styled Component.

import styled from 'styled-components';

const Item = styled.div`
  color: red;
`

const Container = styled.div`
  & > ${Item} {
    font-size: 2rem;
  }
`

class StyledComponentsDemo extends Component { 
 render() {
    return(
      <Container>
        <Item>Item 1</Item>
      </Container>
    )
  }
}

As you can see, we have the ability to not only specify the styles in our components, but also the ability to add some dynamic functionality. Building on this, we are able to better accommodate some common use cases such as adding themes to our applications.

Using Themes

Theming is accomplished by exporting the ThemeProvider component, passing an object to its theme prop, and wrapping the entire app in the ThemeProvider component. This will give every Styled Component access to the theme object.

import styled, { ThemeProvider } from 'styled-components';

const Item = styled.div`
  color: ${( props ) => props.theme.color.primary}
`

const theme = {
  color: {
    primary: 'red'
  }
}

class StyledComponentsDemo extends Component { 
  render() {
    return(
      <ThemeProvider theme={theme}>
        <Item>Item 1</Item>
      </ThemeProvider>
    )
  }
}

Components that are not Styled Components can also access theme by using the withTheme function.

import { withTheme } from 'styled-components';

class MyComponent extends React.Component {
  render() {
    return <p>{this.props.theme.color.primary}</p>
  }
}

export default withTheme(MyComponent);

Styled System

Styled System is a collection of utility functions that add style props to your React components and allows you to control styles based on a global theme object with typographic scales, colors, and layout properties.
https://styled-system.com

If you created a Button component from Styled Components and you want it to receive foreground and background color props, you can use the styled-system utility function color and pass it as a placeholder function in the Template Literal to enable these props.

import styled, { ThemeProvider } from 'styled-components';
import { color } from 'styled-system'

const theme = {
  colors: {
    primary: 'blue'
  }
}

const Box = styled.div`
  ${color}
`

class StyledSystemDemo extends Component {
  render() {
    return (
      <ThemeProvider theme={theme}>
        <>
          <Box color="#fff" bg="tomato">Tomato</Box>
          <Box color="white" bg="primary">Tomato</Box>
        </>
      </ThemeProvider>
    )
  }
}

Note: The name of the generated props are all outlined in the styled system API.

If there is a theme available, the utility function will try to match the prop value to the theme before using the value as the raw value (e.g. #fff).

Structuring Theme objects

The structure of the theme object and styled-system are tightly coupled. The structure follows a Work-in-Progress specification called System UI Theme Specification.

For instance, the fontSizes and colors keys follow this specification, and their values (arrays or objects) also follow this specification.

export const theme = {
   fontSizes: [
      12, 14, 16, 20, 24, 32
   ]
   fontSizes.body = fontSizes[1]
   colors: {
      blue: '#07c',
      green: '#0fa',
   }
}

With the above theme, the fontSize prop on a component could receive the index value of the array, or the alias body.

Under the hood of color

Let’s look at how styled-system implements the utility function color. Remember that a utility function is called like this:

const Button = styled.div`
   ${color}
`

This is what the function looks like.

// <https://github.com/styled-system/styled-system/blob/v2.3.6/src/styles.js>

export const color = props => ({
  ...textColor(props),
  ...bgColor(props)
})

Which is akin to writing this in the Template Literal:

const Button = styled.div`
   ${(props) => ({
      ...textColor(props),
      ...bgColor(props)
   })}
`

The textColor and bgColor functions will return style objects that are spread within the function. These functions look like this.

// <https://github.com/styled-system/styled-system/blob/v2.3.6/src/styles.js>
export const textColor = responsiveStyle({
  prop: 'color',
  key: 'colors', // theme key
})
export const bgColor = responsiveStyle({
  prop: 'bg',
  cssProperty: 'backgroundColor',
  key: 'colors'
})

The responsiveStyle function handles all the breakpoints, fallbacks and prop naming. Below, I simplified the styled-system code for demonstrative purposes.

// <https://github.com/styled-system/styled-system/blob/v2.3.6/src/util.js>
// I simplified this for demonstrative purposes
export const responsiveStyle = ({
  prop,
  cssProperty,
  key,
}) => {
  const fn = props => {
    cssProperty = cssProperty || prop
    const n = props[prop];
    const theme = props.theme;

    // ...

    return {
      [cssProperty]: theme[key][n]
    } 

    // ...
  }
  return fn
}

Which can be represented to look something like this:

const Button = styled.div`
   {
     ${...(props) => (
         { color: props.theme['colors'][props.color] }
     )}
     ${...(props) => (
         { backgroundColor: props.theme['colors'][props.bg] }
     )}
   }
`

Breakpoints and responsive themes

For responsive themes, styled-system lets you establish breakpoints in the theme and then lets you pass an array as a prop with different values for each breakpoint. styled-system takes a mobile first approach, so the first index will always be the smallest breakpoint.

<Box
  width={[
    1, // 100% below the smallest breakpoint (all viewports)
    1 / 2, // 50% from the next breakpoint and up
    1 / 4, // 25% from the next breakpoint and up
  ]}
/>
// theme.js
const breakpoints = ['40em', '52em', '64em', '80em']
export default { breakpoints };

Conclusion

I was inspired to see how the developers of styled-components and styled-system employed the extended functionality of Template Literals and Tagged Templates to provide users with an intuitive way for adding SCSS to React components.

Have you seen any interesting uses of common functionality in your work recently? Please share!

Sources

Styled System

Styled Components

The magic behind 💅 styled-components

MDN Web Docs: Template Literals

Are Your Unit Tests Failing for the Expected Reasons?

Evan Schultz (he/him) — Wed, 11 Dec 2019 13:16:56 +0000

Unit tests can be an invaluable tool in the developers toolbox. You don't need to be a strict TDD purist to make unit testing worthwhile. Once you get into the flow of writing tests, it can be rather satisfying to watch the Nyan Cat Reporter go across your screen as the number of tests passing increase.

As with any other tool though, it can be misused, and not always provide the benefit that you want or expect.

The other day I was doing some code clean up, and came across a test that started to make me ask questions instead of having them answered.

Some of the questions that a unit test should be answering:

What is it testing?
What is it doing?
What is expected behavior?
What is the actual behavior?
Does it pass or fail for the expected reasons?

The test that made me go 'hrmm?' looked something like this:

/// .... mock removed for now

it('should call /location/calculate-distance correctly', () =>
   locationApi.calculateDistance(10,20,30,40).then(res => {
     expect(res.distance).to.equal('ok');
   })
 );

Reading this unit test, it's not really clear what is actually being tested. The it block is saying that it should call /location/calculate-distance correctly, but the expect at the bottom is looking at the response.

Currently this test is passing, but it's not passing for a reason that the test states it should be. Yes, there is something responding, but response.distance being ok has nothing to do with the behavior we wanting to verify.

This application was using ismorphic-fetch and fetch-mock to mock out HTTP requests. The mock for this test looks like:

before(() => {
  fetchMock
  .mock(
    (url, options) =>
      url === [
        LOCATION_ENDPOINT,
        'calculate-distance?&lat1=10&long1=20&lat2=30&long2=40',
      ].join('/')
      && options.method === 'GET',
    {
      body: JSON.stringify({ distance: 'ok' }),
      status: 200,
      headers: { 'content-type': 'application/json' },
    }
  );
});

This starts to give a bit more insight, and after digging around that was going on in locationApi.calculateDistance - it seems like this test is wanting to verify that for a given set of parameters, the URL is formed up correctly to query the API to calculate the distance.

When the test runs, it currently passes. If it fails though, am I getting useful information? If I tweak the location code in how it forms the URL, the errors that get reported look like:

1) api/location should call /location/calculate-distance correctly:
     Error: only absolute urls are supported
      at node_modules/node-fetch/index.js:54:10
      at new Fetch (node_modules/node-fetch/index.js:49:9)
      at Fetch (node_modules/node-fetch/index.js:37:10)
      at module.exports (node_modules/isomorphic-fetch/fetch-npm-node.js:8:19)
      at FetchMock.fetchMock (node_modules/fetch-mock/src/fetch-mock.js:265:17)
      at exports.default (src/api/helpers/composer-request.js:14:12)
      at Object.exports.getTaxEstimate (src/api/location-api.js:8:10)
      at Context.<anonymous> (src/api/location-api.test.js:27:13)

And hidden away at the top of the unit tests running and easy to miss, is:

api/location
unmatched call to /location!calculate-distance?lat1=10&long1=20&lat2=30&long2=40

This isn't terribly useful information. The error that is getting reported doesn't tell me anything about what the expected and actual results were.

Error: only absolute urls are supported

This is an error thrown by fetch-mock before our expect statements have even been hit. There is a hint at the top of the unit test reports where fetch-mock will complain about an unmatched call.

This message is easy to miss, and it also requires the person reading the unit test results to understand a bit of how fetch-mock works get pointed in the right direction.

For a seemingly simple unit test, it asks more questions than it answers.

A more accurate description of this test would be:

describe('fetch-mock behavior', () => {
  it('should return the object I told it to if no error is thrown', () => {

  });
});

If we were the authors of fetch-mock, that could possibly be a useful test. But, we are wanting to write unit tests for the system we are building, not for the mocking frameworks we are using.

The fact that expect(res.distance).to.equal('ok'); is more of a coincidence than the behavior you want to test.

What can we do to make it clear what the intended purpose of this test is, and that it provides meaningful errors when it fails?

Let's revisit the questions we asked at the start, and clean up the test to start answering them.

What is it testing?

When the calculateDistance is called, then an API is called with a specific URL and query parameters. The response object is inconsequential here. Let's adjust the unit test to start being more descriptive and accurate.

it('should call calculate-distance with correct query parameters', () => {  
  //
});

The 'it' sentence starts to describe what we are doing without needing to read the test body.

What is it doing?

The initial test wasn't that bad at answering this one, it's calling the locationApi.calculateDistance,

it('should call calculate-distance with correct query parameters', () => {  
  return locationApi
  .calculateDistance(10,20,30,40)
  .then(() => {
    // ..
  });
});

What is the expected / actual behavior

This is where the previous test started to fail at answering these questions.

We don't care if the response object has a distance of 'ok', we want to verify the URL that is being hit.

In answering this question, we can state what the expected and actual results are.

it('should call calculate-distance with correct query parameters', () => {  
  const EXPECTED_URL = `${LOCATION_ENDPOINT}/calculate-distance?lat1=10&long1=20&lat2=30&long2=40`;
  return locationApi
  .calculateDistance(10,20,30,40)
  .then(() => {
    const ACTUAL_URL = fetchMock.lastUrl();
    expect(EXPECTED_URL).to.equal(ACTUAL_URL);
  });
});

Now when reading the unit test, the expected reasons for passing/failing are more obvious. The thing that I am asserting is reflective of the behavior that I want.

However, if the test fails, the output still isn't very useful and still complains about: Error: only absolute urls are supported

This leads to answering the final question:

Does it pass or fail for the expected reasons?

Currently fetch-mock is throwing an error before we even hit our expect statement, so we can't guarantee that things are passing or failing for the reasons we expect.

Adjusting our mock is pretty straightforward. Instead of having the mock be very specific for a URL, I am adjusting it to match just about anything that begins with a slash.

before(() =>fetchMock.mock(`^/`,
   { body: JSON.stringify({ data: 'ok' }),
     status: 200,
     headers: { 'content-type': 'application/json' },
   }));

When we run our test and if it fails, the error that gets reported is now:

1) should call calculate-distance with correct query parameters:

      AssertionError: expected '/location!calculate-distance?lat1=10&long1=20&lat2=30&long2=40' to equal '/location/calculate-distance?lat1=10&long1=20&lat2=30&long2=40'
      + expected - actual

      -/location!calculate-distance?lat1=10&long1=20&lat2=30&long2=40
      +/location!calculate-distance?lat1=10&long1=20&lat2=30&long2=40

      at src/api/location-api.test.js:21:29
      at process._tickCallback (internal/process/next_tick.js:103:7)

With a few minor changes to the unit test and the mock, the test is starting to answer questions, instead of making us ask them.

The mock has been simplified and is easier to understand.
The expect statement makes it clear the behavior that we are testing.
The test fails for the correct reason - actual does not meet expected behavior.
When the test fails, it is descriptive to what the error is.

Next time you're writing a unit test or reviewing others, maybe double check to ensure that they are passing or failing for the reasons that you expect.

If reading a unit test makes you ask questions, then it could be a sign that you need to clean them up to make them more useful.

initially posted on the rangle.io blog

Improving your mental model of useEffect

Evan Schultz (he/him) — Wed, 11 Dec 2019 12:55:15 +0000

Hooks landed in React a few months ago, and there has been a lot of excitement around them in terms of figuring out how to best use them, best practices, and how they map to current concepts in React and the lifecycle.

Many React developers are familiar with the React Component lifeCycle, and hooks like:

etc.

When trying to understand the useEffect hook, it's natural to want to map it to the lifecycle methods we already know. At first glance, useEffect seems to be like a combination of componentDidMount and componentDidUpdate.

While this can be a useful way of looking at it at first, it may not be the most accurate.

Instead of thinking in terms of 'what do I want to do when I mount, or when I update', it's more useful to ask:

What values does this effect depend on?

To better understand where the idea of useEffect = componentDidMount + componentDidUpdate comes from, we will first look at a typical class-based component that is doing some data fetching.

export default SearchComponent extends Component {
  constructor(props) {
    super(props);
    this.state = {
      results: []
    }
  }
  componentDidMount() {
    this.query(this.props.id)
  }
  componentDidUpdate(prevProps) {
    if(this.prevProps.id !== this.props.id) {
      this.query(this.props.id);
    }
  }
  query(id) {
    this.setState({isLoading: true})
    fetch(`/some/url/${id}`)
      .then(r=>r.json())
      .then(r=>this.setState({
        results: results
      });
    )
  }
}

When the component first mounts, we fetch data for the id that has been passed down as a prop. When the component updates, many things other than the id prop changing can cause this method to run, so we want to ensure that id has actually changed - or some poor server is going to get a DDoS attack with a bunch of API calls that we don't need.

While the lifecycle hooks of componentDidMount and componentDidUpdate with class-based components are common places to make a request based on a property, the fact that the component is mounting or updating is not really the thing we are concerned with.

What we are actually concerned with?

"what data does the query depend on?"

Before looking at how to handle this with useEffect, lets quickly review the API of useEffect:

Accepts a function
If it returns a function, it will do cleanup when the component is unmounted
Has an optional 2nd argument to pass in the data it depends on

One of the key things to keep in mind is the importance of that second argument, the React Docs go into this in detail, but a summary is:

If we leave it blank - it will run on every single render.
If we pass in an empty array - it will execute only when the component mounts, and not on any updates
If we pass in a value - it will execute when any of those values change
If you are using the react-hooks eslint plugin (and you should) - not providing the dependencies to your useEffect will give you warnings.

export default SomeComponent = ({id}) => {
  let [results, setResults] = useState([]);
  useEffect(()=>{
    fetch(`/some/url/${id}`)
      .then(r=>r.json())
      .then(r=>setResults(r))
  },[id])
}

In the class-based version, making API calls feels very imperative - when this method is called, I want to check if/how a value has changed, and if it has changed - I want to call a method.

If the component is being created or updated often isn't the thing the matters. What we actually care about is "has the values that I care about changed?".

Before hooks were introduced, componentDidMount and componentDidUpdate were the best tools for the job at the time.
With the hook based version, we are able to express this intent in a more declarative way: "I want to fetch data when the id changes"

How do we identify what the effect depends on?

The eslint plugin can guide you in the right direction, but the short version is: "is there a variable that impacts how we run the effect?" If so, add it to the dependencies.

To demonstrate this, let's add an extra query parameter to our search:

export default SomeComponent = ({id, filter}) => {
  let [results, setResults] = useState([]);

  useEffect(()=>{
    fetch(`/some/url/${id}?filter=${filter}`)
      .then(r=>r.json())
      .then(r=>setResults(r))
  },[id])
}

Even though we have added filter to the fetch query string, we have not added it to the dependencies of useEffect.

As we update the filter, we won't be calling the API on any of the other updates, and it will only run when the id has changed.

Fixing this can be simple enough - add the filter to the list of dependencies for the useEffect.

export default SomeComponent = ({id, filter}) => {
  let [results, setResults] = useState([]);

  useEffect(()=>{
    fetch(`/some/url/${id}?filter=${filter}`)
      .then(r=>r.json())
      .then(r=>setResults(r))
  },[id, filter])
}

As you can see, to properly use useEffect, in this case, we don't care if the component is mounting, or updating, or where it is in the life cycle.

What we do care about is what data does this effect depend on.

useEffect is a very useful tool to add to our toolbox when working with React, but it can also be one of the more difficult hooks to fully understand.

Hopefully, this post can help clarify things a little bit better, but if you are curious for a deeper dive, be sure to check out Dan Abramovs' post, A Complete Guide to useEffect

this article was initially posted on the rangle.io blog and medium

Running Jenkins and Persisting state locally using Docker

Rangle.io — Tue, 10 Dec 2019 17:44:55 +0000

Jenkins is one of the most popular Continuous Integration and Delivery Servers today, so it's only natural that you're probably interested in learning more about it. When starting out, you'll need to first run it on your local machine. However, the problem with that is the Jenkins configuration files will then live directly on your machine. A better solution is to run it as a Docker container, here are some of the reasons why:

All of your Jenkins configuration files live inside the container rather than the host machine. Knowing that all the files you need are inside the container, you can eliminate the issue of accidentally mixing your files with Jenkins configuration files.
Docker instances are easier to manage if you are interested in running Jenkins on multiple platforms
You can easily create and destroy the Jenkins server and remove all the Jenkins data

Another benefit of using containers is persisting the state of your Jenkins server using Docker volumes. Why do this?

You get to keep all your projects and configurations even after restarting your computer (local machine)
You don't need to run the whole Jenkins setup again
You can remove your container instance and still able to recover the state of your Jenkins server

With that in mind, I'll show you how you can start configuring Jenkins and persisting state on your local machine. Let's get started!

Jenkins setup

Before we get started, you'll need to install Docker on your machine. If you're not sure how you can refer to this official documentation.

After installing Docker, download the latest stable Jenkins image by running:

docker image pull jenkins/jenkins:lts

You should see something like this:

Persisting Jenkins Data

You can create a volume by running the command below:

docker volume create [YOUR VOLUME]

Volumes are used to make sure that you don't lose your Jenkins data. If you are using the -v flag on container creation ( docker container run), feel free to skip this step since Docker will automatically create the volume for you.

Run the container by attaching the volume and assigning the targeted port. In this example, we'll also run it in detached mode. Here is the command to run your Docker container:

docker container run -d \ -p [YOUR PORT]:8080 \ -v [YOUR VOLUME]:/var/jenkins_home \ --name jenkins-local \ jenkins/jenkins:lts

If you were wondering what the arguments stand for, here is what each means:

-d: detached mode
-v: attach volume
-p: assign port target
-name: name of the container

And now, we're ready to take a look at an example of how you could run this command:

docker container run -d -p 8082:8080 \ -v jenkinsvol1:/var/jenkins_home \ --name jenkins-local \ jenkins/jenkins:lts

After running the command, you should be able to see the code to be used for the next step on your setup.

In the command we ran, /var/jenkins_home is the path to where the Jenkins state is stored on our container instance. The most important argument we pass when it comes to data persistence in this example is the -v **[YOUR VOLUME]**:/var/jenkins_home. This argument is what helps Docker link the volume to the file inside the container. To learn more about Docker volumes, you can check out the official documentation. If you're not familiar with Docker, you can start with this helpful docker blog series, Learning Docker - The Command Line Interface. You'll notice that I am using port 8082 instead of the default 8080. The reason, other than demonstrating that you can use other ports, is that port 8080 is used by some web frameworks.

If you run docker ps, you should see your docker container running

After confirming that your container is running, go to localhost:[YOUR PORT] ( localhost:8082 on my example) on your browser and you should see this page:

As a part of the Jenkins setup, we need to view the password inside the container instance. In order to do this, we need to use the CONTAINER ID (or the name) and run docker exec.

Here is the full command:

docker container exec \ [CONTAINER ID or NAME] \ sh -c "cat /var/jenkins_home/secrets/initialAdminPassword"

After running the command, you should see the code. Copy the code and paste it on the webpage to unlock Jenkins. After unlocking, click on Install suggested plugins on the Customize Jenkins page.

Wait until the installation is complete and then you can proceed in creating your first admin user.

After creating the admin user, setup the Instance configuration. Since you are only using Jenkins locally, leave the URL to your localhost URL. Click on Save and Finish to start using Jenkins.

Confirming Jenkins state is persisted

Once you get to the Jenkins home page, create a new job.

Name the job Test and set it to be a Freestyle project.

Leave everything as default and add a new Shell Execution under Build. Add this as the command:

echo "Working"

Save the job and click on Build Now to start running the job.

Click on the Build Status (blue ball) under Build History (left sidebar) to view the console output. You should see that our command ran with no problems.

Recovering Jenkins

After confirming that the job runs, we want to make sure that we can recover our Jenkins configuration if needed. In Jenkins, all of the configuration is stored in /var/jenkins_home/ by default. Remember that we are using docker volume to store information about our Jenkins instance.

In order to check if the persistence works correctly, let's destroy our current docker container and see if we can log in as an admin and view our job build history.

First, run docker container kill [CONTAINER ID] to stop the instance. After doing so, run docker container rm [CONTAINER ID] to completely remove the container instance.

Visit localhost:[YOUR PORT] ( localhost:8082 on my example) to confirm that the Jenkins instance is not running anymore. You should see something like this:

Run the same command we used to create the container instance in order to recover the Jenkins instance.

docker container run -d -p 8082:8080 \ -v jenkinsvol1:/var/jenkins_home \ --name jenkinslocal \ jenkins/jenkins:lts

You should get a new CONTAINER ID after running the command. Visit localhost:[YOUR PORT] ( localhost:8082 on my example). You should see the login page. Login with the admin credentials you set during Jenkins initialization.

After logging in, you should see the job you created and view the console output.

In summary

I hope you can now see why Docker is the best way to start learning or at least playing around with Jenkins. You can easily run a Jenkins instance as a Docker container and persist your Jenkins server state using Docker Volumes. In case you need to restart or recover your Jenkins instance, all of the state is stored inside the Docker Volume. If you want to read more about Jenkins, Docker and DevOps, check out our other blogs, here.

Docker for Frontend Devs: Custom Docker Images for Development

Rangle.io — Fri, 06 Dec 2019 21:16:04 +0000

By: Benjamin Martin

Let's take a moment to consider what is important for local development. For me, I want to make sure all my developers are using the same dependencies, and I don't want to worry about what versions they have installed. No more "but it works on my machine" excuses. At the same time, I want to make sure we retain the conveniences of HMR (Hot Module Replacement) so that developers don't need to constantly refresh the application to see their changes reflected. We don't want to lose fast feedback.

In this article, we'll look at how we can setup Docker for a boilerplate VueJS app with custom Dockerfiles from which our images and containers will be built and how we gain efficiencies from these.

In case you missed the first part in this series, check here to learn more about the command line interface that Docker ships with. We need to use the commands from that article in this section. If you are already familiar with Docker CLI, please continue to follow along.

Prerequisite: Create our project

This is of course a Docker article, so please ensure you have Docker installed. You can follow the official install instructions for Docker here. Since I'm using Vue, I've used the VueCLI to spin up a quick workspace with vue create docker-demo.

The configuration I selected (seen below) will be relevant to do E2E testing and unit testing which will become part of our CI/CD pipeline.

Once everything is installed, cd into our new project folder, open an IDE and let's dig in.

Custom Docker Image for Development

If you've played with Docker but not built your own image, you probably know we specify an image when we execute our docker run command. Those images are pulled from Docker Hub or some other remote repository (if that image is not found locally). In our case though, we want to build a custom image.

In the root of our project, create a file named Dockerfile.dev. This will be our development image. In that file, copy the following code into it.

# Base Image
FROM node:9.11.1

ENV NODE_ENV=development
ENV PORT=8080

WORKDIR /usr/src/app
COPY package*.json /usr/src/app/
RUN cd /usr/src/app && CI=true npm install

EXPOSE 8080
CMD ["npm", "run", "serve"]

Ok... but what does all this do? Let's dig into it.

Dockerfile Commands and Keywords

FROM specifies the preexisting image on which to build our custom image. Since we are running a node application, we've chosen one of their official Docker images.

FROM node:9.11.1 means our application image will start with the node v 9.11.1 image

ENV sets environment variables

ENV PORT=8080 sets the environment variable PORT for later use

ENV NODE_ENV=development sets the environment variable NODE_ENV for use within our app

WORKDIR sets the working directory within the container

WORKDIR /usr/src/app defines /usr/src/app/ as our working directory within the docker image

COPY copies new files, directories or remote files into the container/image

COPY package*.json /usr/src/app/ copies our package.json and package-lock.json into our working directory

RUN executes a command in a new layer on top of the current image and commits it. When you run the build, you will see a hash representing each layer of our final image

RUN cd /usr/src/app/ && CI=true npm install changes the working directory to where the package.json is and installs all our dependencies to this folder within the image. This makes it so that the image holds frozen copies of the dependencies. Our Docker image, not our host machine, is responsible for our dependencies

EXPOSE allows us to access a port on the container from our host machine

EXPOSE 8080 matches the port on which our app is running, inside the container and allows us to access our app from our host machine

CMD provides the default initialization command to run when our container is created, like a startup script

CMD ["npm", "run", "serve"] sets this as our default command when we start our container. This is not run when building the image, it only defines what command should be run when the container starts.

I know you're anxious to get this running, but hold your horses. Let's look closer at our Dockerfile.dev and understand why we did what we did.

Dockerfile Structure Recommendations

So, Where's my app?

Right. We didn't use the COPY command to copy our full workspace. Had we done so, we'd need to run docker build and docker run for every code change. We don't want to do this over and over for development. We can be more efficient

Caching Dependencies

We are taking advantage of how Docker layers the images. As Docker builds our image, you'll see a hash for each layer as it is completed. What's more is that Docker also caches these layers. If Docker can see that nothing has changed on that layer from a previous build (and previous layers are also identical) then Docker will use a cached version of that layer, saving you and your developers precious time! When a layer changes, any cached layers on top of it are invalidated and will be rebuilt.

Therefore, if there is no change to our package.json or the package-lock.json then our entire image is cacheable and doesn't need to be rebuilt!

Priority

This is also why you want to have other Dockerfile commands that change less frequently near the top of our file. As soon as one layer of our cache is invalidated, for example, if you change ENV PORT=8080 to another port, that cached layer and every cached layer after it is invalidated and Docker will have to rebuild those layers.

Building the Custom Docker Image

Now, build the image with this command: docker build --tag docker_demo:latest --file Dockerfile.dev .

Using --tag in the docker build command allows us to easily reference this image from our docker run command

The . at the end of the docker build command references the context where our custom Dockerfile can be found. So, this command should be run from the root of our project directory

You can run it with docker run docker_demo:latest, but unfortunately, we have more work to do to get it working quickly and easily from the command line.

Running our Container: Quality of Life Improvements

We're going to be executing our docker run command daily, if not more frequently. However, if we simply execute the docker run docker_demo:latest command, Docker will create a new container each time. Docker won't stop the old container unless you do so explicitly. This is very useful in many cases, but since we've hardcoded the host port, we'll run into port collisions on our host machine.

In order for us to easily stop and remove our old containers, we should name them so we can easily refer to them later. Additionally, I want the running container to be removed if I cancel the running process.

docker run --rm -it\
--name docker_demo_container\
docker_demo:latest

What was added?

We added a --name field to the end of our run command. This allows us to reference the container without looking up the hash. Now, we can easily stop our container by name.

We also added the --rm and -it flags to our docker run command. The --rm flag tells Docker to remove the container if and when it is stopped. The -it flag keeps the terminal live and interactive once the container is started.

Mounting Host Directories

Let's go back to our docker run command and let's find a way to mount our workspace directory to a folder within our container. We can do this by adding a mount point to our container in the docker run command. This will tell Docker that we want to create an active link between our host machine's folder (src) and the Docker container folder (dst). Our new command should look like this:

docker run --rm -it\
--name docker_demo_container\
--mount type=bind,src=`pwd`,dst=/usr/src/app\
docker_demo:latest

But this could conflict with our host machine's node_modules folder since we're mounting our entire pwd to our app's location in the image (in case one of our developers accidentally runs npm install on their host machine). So, let's add a volume to ensure we preserve the node_modules that exists within our container.

docker run --rm -it\
--name docker_demo_container\
--mount type=bind,src=`pwd`,dst=/usr/src/app\
--volume /usr/src/app/node_modules\
docker_demo:latest

Accessing Ports Inside the Container

If you tried the above command (and you're running a VueJS app), you should see:

 App running at:
  - Local:   http://localhost:8080/

  It seems you are running Vue CLI inside a container.
  Access the dev server via http://localhost:<your container's external mapped port>/

Docker is giving you a hint that we need to expose a port from our container and publish it on our host machine. We do this by adding the --publish flag to our run command. (We already have the EXPOSE command in our Dockerfile.dev)

--publish <host-port>:<container-port> tells Docker that traffic to the host machine (i.e. via localhost) on port <host-port> should be directed towards the container at the <container-port> that you define.

`docker run` in One Command

Let's take a look at our final run command:

docker run --rm -it\
--name docker_demo_container\
--publish 4200:8080\
--mount type=bind,src=`pwd`,dst=/usr/src/app\
--volume /usr/src/app/node_modules\
docker_demo:latest

Running the above command will finally allow us to access our app via http://localhost:4200.

Testing it out

Let's build a fresh copy and run it. If you try changing one of our file's templates, you'll see everything is still functioning as it should be.

But speaking of testing, what about unit tests? Well, once our container is running, we can open a new terminal and docker exec a command to run in our container.

docker exec -it docker_demo_container npm run test:unit

The above command will create an interactive terminal connection with our container docker_demo_container and execute the command npm run test:unit in it, allowing us to run unit tests for our app.

In Closing

We now have a way to build our development images and run them locally while maintaining the conveniences of Hot Module Replacement to keep our development workflow efficient. Our developers don't need to worry about dependencies on their host machine colliding with those in the image. No more "but it works on my machine" excuses. And, we also have a command we can easily run to execute our unit tests.

If you find anything I missed or want to chat more about Docker, please reach out to me!

Connecting Stripe events to the AWS EventBridge

Rangle.io — Wed, 20 Nov 2019 15:28:39 +0000

Stripe is a great platform for running an online business, especially on account of the developer-centric API that makes it easy to collect payments, set up subscriptions and more.

Many of these APIs result in Stripe generating one or more events that you can subscribe to via a webhook. These events could represent a successful payment, or perhaps a new subscriber to your SaaS platform. In total, there are well over 100 different types of events that you can subscribe to, so how can we build a loosely-coupled cloud-native event-driven architecture to handle these events? How can we make this serverless?

It's easy, we'll use AWS EventBridge.

Why AWS EventBridge?

For those of you just learning about AWS EventBridge, it is a serverless event bus supporting event-driven architectures, released July 2019. An event bus is a central location where business events can be published and routing rules can be configured to send those events to downstream functions or services that operate on them. By using an event bus as an intermediate layer, this decouples services from needing direct knowledge of how to communicate with each other, which allows development teams to build and operate independently.

One reason, among many, why AWS EventBridge should be considered is the cost. With AWS EventBridge, you have a fully managed cloud-native event bus, requiring no effort to set up and no servers to pay for - and it costs a very reasonable $1 per million events. Latency is typically about half a second. By going with a fully managed solution, you're saving your team from spending time on 'undifferentiated heavy lifting' associated with infrastructure, giving them more time to focus on delivering customer value.

The solution

We'll show here how we can set up AWS EventBridge so that it can begin ingesting events from Stripe. As part of that ingestion, we'll check that the event is truly from Stripe by verifying the signature on the event. Let's get started.

Using an an open-source project we created called stripe-eventbridge, you can quickly set up the plumbing to connect Stripe events to AWS EventBridge via a Lambda function that will then validate the authenticity of incoming events (so that downstream functions won't have to). This allows for the events that are published on the event bridge to be routed to various downstream functions through simple routing rules which we show at the end of the post.

With this deployed, you can set up many downstream Lambda functions to handle the wide array of events generated by Stripe and configure the routing to these event handlers with AWS EventBridge based on the Stripe event type. All of this without having to worry about event signatures, since events are only placed on the EventBridge if they pass validation.

The stripe-eventbridge project creates an endpoint that you configure in the Stripe Dashboard as the destination for the webhook events. When an event arrives, the endpoint will invoke a Lambda that we provide which has these responsibilities:

to validate that the event has a valid signature (i.e. that it was generated by Stripe)
to place validated events (only) on the AWS EventBridge
to notify an SNS topic if an event fails to validate (e.g. due to an invalid signature)

Solution overview

stripe-eventbridge uses the Serverless Framework to generate the infrastructure in the AWS section of the diagram above. The Serverless Framework allows us to automatically create and deploy the CloudFormation stacks needed to realize the above configuration. The stacks can be deployed via sls deploy and rolled back via sls remove using the command line client.

Serverless components in stripe-eventbridge

Let's walk through what stripe-eventbridge creates:

A Lambda - this function validates incoming events, and if they are from Stripe (i.e. signed correctly) then they are relayed to the AWS EventBridge where downstream services can read from them.
An API Gateway endpoint - this is the webhook endpoint that the Lambda above is listening to for new events.
An AWS SecretManager secret - this secret is created with an empty value, and you populate this with the signing key assigned to your webhook endpoint in the Stripe Dashboard
An SNS topic - if the webhook fails validation or processing, then a notification is sent to a topic called stripe-webhook-event-failed-to-validate. You can configure subscriptions to this topic, so that you are emailed (for example) whenever an event fails processing.

If you'd like to get started, you can visit the GitHub project here:

https://github.com/rangle/stripe-eventbridge

Routing by event type

Now that you have events being published to the EventBridge, you can configure routing based on the event type using Rule patterns like this. The detail-type values are exactly the values of the underlying Stripe events - you can see a full list here: https://stripe.com/docs/api/events/types

{

  "detail-type": [

    "payment_intent.succeeded"

  ],

  "source": [

    "Stripe"

  ]

}

Alternately, if you are creating your event handlers using the Serverless Framework, then you can declare the same routing in your infrastructure code (this is really good practice) in your serverless.yml as follows:


    handler: handler.myLambdaFunction

    events:

      - eventBridge:

          pattern:

            source:

              - Stripe

            detail-type:

              - payment_intent.succeeded

And that's all!

I hope you have found this valuable. Stay tuned, we'll be exploring more with Serverless in upcoming posts.

Running Jenkins and Persisting state locally using Docker

Rangle.io — Fri, 15 Nov 2019 20:42:15 +0000

All of your Jenkins configuration files live inside the container rather than the host machine. Knowing that all the files you need are inside the container, you can eliminate the issue of accidentally mixing your files with Jenkins configuration files.
Docker instances are easier to manage if you are interested in running Jenkins on multiple platforms
You can easily create and destroy the Jenkins server and remove all the Jenkins data

Another benefit of using containers is persisting the state of your Jenkins server using Docker volumes. Why do this?

You get to keep all your projects and configurations even after restarting your computer (local machine)
You don't need to run the whole Jenkins setup again
You can remove your container instance and still able to recover the state of your Jenkins server

With that in mind, I'll show you how you can start configuring Jenkins and persisting state on your local machine. Let's get started!

Jenkins setup

Before we get started, you'll need to install Docker on your machine. If you're not sure how you can refer to this official documentation.

After installing Docker, download the latest stable Jenkins image by running:

You should see something like this:

Persisting Jenkins Data

You can create a volume by running the command below:
docker volume create [YOUR VOLUME]

Volumes are used to make sure that you don't lose your Jenkins data. If you are using the -v flag on container creation (docker container run), feel free to skip this step since Docker will automatically create the volume for you.

Run the container by attaching the volume and assigning the targeted port. In this example, we'll also run it in detached mode. Here is the command to run your Docker container:
docker container run -d \ -p [YOUR PORT]:8080 \ -v [YOUR VOLUME]:/var/jenkins_home \ --name jenkins-local \ jenkins/jenkins:lts

If you were wondering what the arguments stand for, here is what each means:

-d: detached mode
-v: attach volume
-p: assign port target
---name: name of the container

And now, we're ready to take a look at an example of how you could run this command:
docker container run -d -p 8082:8080 \ -v jenkinsvol1:/var/jenkins_home \ --name jenkins-local \ jenkins/jenkins:lts

After running the command, you should be able to see the code to be used for the next step on your setup.

If you run docker ps, you should see your docker container running

After confirming that your container is running, go to localhost:[YOUR PORT] (localhost:8082 on my example) on your browser and you should see this page:

As a part of the Jenkins setup, we need to view the password inside the container instance. In order to do this, we need to use the CONTAINER ID (or the name) and run docker exec.

Here is the full command:
docker container exec \ [CONTAINER ID or NAME] \ sh -c "cat /var/jenkins_home/secrets/initialAdminPassword"

Wait until the installation is complete and then you can proceed in creating your first admin user.

After creating the admin user, setup the Instance configuration. Since you are only using Jenkins locally, leave the URL to your localhost URL. Click on Save and Finish to start using Jenkins.

Confirming Jenkins state is persisted

Once you get to the Jenkins home page, create a new job.

Name the job Test and set it to be a Freestyle project.

Leave everything as default and add a new Shell Execution under Build. Add this as the command:
echo "Working"

Save the job and click on Build Now to start running the job.

Click on the Build Status (blue ball) under Build History (left sidebar) to view the console output. You should see that our command ran with no problems.

Recovering Jenkins

In order to check if the persistence works correctly, let's destroy our current docker container and see if we can log in as an admin and view our job build history.

First, run docker container kill [CONTAINER ID] to stop the instance. After doing so, run docker container rm [CONTAINER ID] to completely remove the container instance.

Visit localhost:[YOUR PORT] (localhost:8082 on my example) to confirm that the Jenkins instance is not running anymore. You should see something like this:

Run the same command we used to create the container instance in order to recover the Jenkins instance.

docker container run -d -p 8082:8080 \ -v jenkinsvol1:/var/jenkins_home \ --name jenkinslocal \ jenkins/jenkins:lts

You should get a new CONTAINER ID after running the command. Visit localhost:[YOUR PORT] (localhost:8082 on my example). You should see the login page. Login with the admin credentials you set during Jenkins initialization.

After logging in, you should see the job you created and view the console output.

In summary

CD with Docker and Feature Branch Testing

Rangle.io — Fri, 25 Oct 2019 17:28:06 +0000

By: Ben Martin

If you've been following along with my Docker series (you can find my latest article about Continuous Integration (CI) here) then you must be pretty happy to have your CI pipeline solving all the world's problems. Your developers are pretty content, but we know there's more we could do. And, I mean, isn't developer happiness the real reason you're reading a DevOps article?

In this article, I'll outline how you can take the CI pipeline one step further to address the Continuous Deployment (CD) aspect of CICD. More specifically, we'll address how one would configure feature branch specific deployments of our app in order to quickly and easily manually test their features, even on mobile devices, before merging their feature branch PR to the develop branch.

Prerequisite Knowledge

If you haven't been following along with my previous articles, to get the most out of this article, you should first have a Dockerized application. If you also have a CircleCI pipeline configured, that's a huge head start! We will be building off that. In our hypothetical situation, we're using an AWS EC2 instance from the free tier in AWS. If you're new to AWS, this could easily be replaced by other technologies. One would only need a server to deploy to. In the past, I used a $5 Digital Ocean droplet to accomplish the same effect, but it's up to you to determine where you want to deploy.

Here's what you'll need:

An AWS Account with an EC2 instance configured to expose ports 22, 80 and 443 and an SSH key for CircleCI to use
Source code of Dockerized App hosted on GitHub
Docker Hub (or some other Docker image repository you can publish to)
CircleCI connected to your app's GitHub repo and your Docker Hub repo
Domain name pointing to your server
Wildcard TLS Certificate for the above domain name

Note: Details for most of the above, aside from anything deployment related, can be found in my previous article.

The Goal: Fast Feedback

As a huge proponent of Agile development methodologies, fast feedback is critical for success, and the higher fidelity the feedback, the better! Unit tests and end-to-end tests all have their place. But, having human eyes review UI changes running in a production-like environment in the browser helps us identify bugs before they become an issue that could impact other teams.

Why should I bother?

In my case, I was working on a project recently where I was developing an Angular application to be consumed through a mobile device. The app required use of the mobile device's camera and accelerometer for the features I was developing. Thus, I wanted to be able to quickly test these features with my actual phone while I was still working out the implementation details.

Sounds simple enough, I just needed to deploy my code somewhere I could access from my device. But this process needed to be automated.

I decided to leverage my existing CI pipeline and extend it to do Continuous Deployments (CD). My CI pipeline was already configured to run my unit tests and end-to-end tests, but I wanted to also deploy my code to a server that I could access from my phone for manual testing.

Here are some key considerations for extending this CI/CD pipeline.

Feature Branch Environments

If you've worked on an enterprise webapp, you may be familiar with terms like dev, int(egration) and staging. Typically, these are environments where different states of our application reside. Each of these environments are meant to host various states of your app, whether that be the branch actively under development, your next release or the last release that you're still supporting with bug fixes.

What often gets lost in the mix is our feature branches. When our developers pick up a ticket, they may want to actively test their code on a suite of devices to ensure cross device and cross browser compatibility. Plus, having this requirement for our developers acts as an additional safeguard for preventing bugs from getting into the develop branch of our code.

If you have ever asked your boss for branch specific environments, you may well have been scoffed at claiming "there's no room in the budget". Well, what if I told you that through the power of Docker and the free tier on AWS (or an existing server you may have running), you can have branch-specific environments beyond dev, int and staging.

Docker To The Rescue!

Docker is the perfect tool for solving this issue for several reasons. In order to see why, consider what would be difficult about running multiple copies of our app on the same environment. Well, even if we imagine a static site there's a lot to cover off. When we deploy our app, what folder do we deploy it to? How do we handle port collisions? Asset loading with relative URLs? How do we notify nginx of the new route/site? There's a lot of configuration and mess that goes into doing it directly on the host machine that we can delegate to Docker to handle for us.

In the end, we'll have a long lived reverse nginx proxy running in a container on our host machine. This container will listen to Docker for other containers we run. Using environment variables in the docker run command, we will be able to tell this nginx proxy to update the configuration to point dynamic subdomains to our feature branch specific containers.

Docker with nginx

Most of the heavy lifting is done by the nginx proxy container. I found an excellent dockerized nginx proxy on GitHub to aid in this process. What is useful about this Docker image is that once you have it running, it will listen to your other docker commands that are run on the host machine, in particular your docker run command. In particular, it will look for environment variables attached to your docker run command to interpret how to update the internal configuration and point it to your app container that is being started.

This reverse proxy image should be running in a container on your server. This container's role is to listen to other Docker commands that you run and automatically update the nginx configuration within this reverse proxy container.

Passing Inputs to your Docker Run

Once you have the nginx proxy running in a Docker container, you can leverage your CI pipeline (like CircleCI) to get the environment variables needed to pass to your server in the docker run command. Just make sure that CircleCI is authorized to access your server via SSH. A secure way to do this is using SSH keys. Generate a key and provide the private key to CircleCI.

Then, add the public key to the ~/.ssh/authorized_keys of your server. From there, make a bash script to manage your branch-specific containers.

CircleCI exposes a number of environment variables to your pipeline that you can use in your jobs to pass to your bash script.

In my case, I used CircleCI to run scp to copy a deployment script to the testing server. Once the bash file is there, it would be executed with the arguments to tell it what sub-domain to use for this container. I used a sanitized version of the branch name. For example, feature/ABC-42-my-feature becomes feature_abc-42-my-feature.myexampledomain.com That subdomain would also be the alias for the container, allowing my script to stop a running container with outdated code when new commits are made to that particular branch.

Here's an example of the run command from my deployment script that provides the environment variables (VIRTUAL_HOST, VIRTUAL_PROTO and VIRTUAL_PORT) expected by the reverse nginx proxy container. The nginx container is listening to the Docker process on your host machine. When it sees a docker run command, it is also looking for those three environment variables. If it sees them in the command, it will update the nginx configuration to point the virtual host to the container that is being run. This is the command that will be run on the host machine to deploy. These environment variables are defined in the CI pipeline.

docker run --expose 443 -e VIRTUAL_HOST=${URL_SUBDOMAIN}.myexampledomain.com -e VIRTUAL_PROTO=https -e VIRTUAL_PORT=443 -d --rm=true --name ${CONTAINER_NAME} ${DOCKER_IMAGE}

The environment variables I am using here are set inside my bash script to ensure I pull the correct image, apply the right name to the container and assign the URL, port and protocol. Putting it together, we can add a "job" to our CircleCI configuration aliased as deploy. This deploy job would look something like this.

scp deploy-${IMAGE_NAME}.sh ${PROD_SERVER_USER}@${PROD_SERVER_HOST}:/root/
ssh -o StrictHostKeyChecking=no ${PROD_SERVER_USER}@${PROD_SERVER_HOST} "/bin/bash /root/deploy-${IMAGE_NAME}.sh $IMAGE_NAME:$TAG ${URL_SUBDOMAIN}"

Note: PROD_SERVER_USER and PROD_SERVER_HOST are set in the .circleci/config.yml file

Note: $IMAGE_NAME:$TAG and ${URL_SUBDOMAIN} are passed as arguments to the deploy script and are used in our docker run command above.

In the bash file, it is important to make sure there isn't already a container running with the same name. It should first be stopped and removed.

Note: In my case, because I was using HTTPS in order to gain access to the mobile device's camera, I needed to setup TLS with a wildcard domain so that I could add subdomains on the fly and still have TLS support.

Results

Now, anytime this repository receives updates to one of the feature branches, that code (assuming it passes all the steps defined in our CICD pipeline prior to deployment) will be deployed to our testing server, on a custom subdomain.

In my project, I took the branch name, sanitized it by removing any special characters, and used that as my subdomain. So, when I push commits to my branch called feature/mobile-view, I could see my changes by visiting this subdomain: https://feature_mobile-view.myexampledomain.com.

The value here being that in order for your developers to see code they're working on in a production-like environment, they only need to push their feature branch to the remote repository. Even if their code is incomplete, they can quickly prototype and test features that would be otherwise difficult to do so in a local environment, like mobile device features, etc.

Further Improvements

We got what we wanted, but that doesn't mean it's perfect. There are many ways we could continue to polish this solution. Let's look at a few.

HTTPS

Although we didn't dig into it here, it is convenient to add a wildcard TLS certificate to the server you are running. You can also look at https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion to see how people have combined a separate container to automatically generate the certificates for the Docker containers being spun up. Similar to the image we used for the proxy, this one would also listen to your docker run commands for environment variables to automate this process.

In the project that inspired this setup, I needed HTTPS support to access the device's camera and accelerometer, so, given that this was a fun little side project, manually uploading a single wildcard certificate was sufficient for me.

Another alternative is to put this solution behind an application load balancer (ALB) through AWS. Do pay attention to which services are compatible with the free tier of new AWS accounts. Otherwise be warned that there will be costs involved in fleshing out the solution further.

nginx Restarting

In case the nginx proxy container ever crashed, you'd want to define what your restart policy is. A simple addition, is to add --restart unless-stopped to the docker run command for your nginx reverse proxy. But you also need to think about if the server restarts. There are many tools that one could use to manage starting containers on boot. Even just systemd to turn it into a service would meet these requirements.

Deployment Script

This was something that resided in the code repository so that CircleCI would have access to it when it pulled the code from the repo. Alternatively, this could also reside on the deployment server instead. There will be pros and cons regardless. Discuss with your team which approach works best for you.

AWS EC2 Configuration

There is a lot that could be said here from security to resource management and configuration automation. Some things you want to remember is always start with the fewest permissions and add more as needed. If you're using EC2, leverage the user script for your AWS instance to ensure your server is fully configured and has all the services running to ensure the nginx-proxy stays online. Write cron scripts to automatically remove containers that have been online for longer than "X" days. If there is a new commit to a particular branch, that container would be refreshed. Old containers would likely represent feature branches that have been merged.

Speaking of which...

GitHub Webhooks

Configuring webhooks to ping your server when a branch gets merged or deleted would be another way to ensure you don't have too many inactive containers on your server.

Conclusion

Hopefully this gives you an idea of what needs to be considered when updating your CICD pipeline to support feature branch specific environments and how easy it can be. Just remember the goal: fast feedback.

I am confident that if you implement a solution similar to that described above, your developers will let you know how much they love it, ranting and raving about how great it is. Just be careful. When they eventually move to a new project, they might just realize how much you've spoiled them ;)

Forem: Rangle.io

Van Gogh, InVision Design System Manager and a new NPM package

Sample Project

Sample Design

Integrating design tokens into code base

Developing some handy Node scripts

Installation

Download design tokens in JSON

Set-up configuration file .invisiondsmutilsrc

Transform JSON into JavaScript object that meets the Styled System Theme Specification

Use scripts in development flow

Use theme in App

Contribute to package

About Rangle and InVision partnership

Recommended InVision DSM documentation

Styled Components, Styled Systems and How They Work

Template Literals

Template Literals and the preceding function

Using the Template Literal as Arguments for the Tag

Passing functions into the Tagged Template

Returning functions from a Tagged Template

Making the leap to React

Styled Components

A simple implementation of styled.h1

Using functions to access Props in Template Literals and Styled Components

Styling regular, custom components

Styling the Styled Components

Using Themes

Styled System

Structuring Theme objects

Under the hood of color

Breakpoints and responsive themes

Conclusion

Sources

Are Your Unit Tests Failing for the Expected Reasons?

What is it testing?

What is it doing?

What is the expected / actual behavior

Does it pass or fail for the expected reasons?

Improving your mental model of useEffect

What we are actually concerned with?

How do we identify what the effect depends on?

Running Jenkins and Persisting state locally using Docker

Jenkins setup

Persisting Jenkins Data

Confirming Jenkins state is persisted

Recovering Jenkins

In summary

Docker for Frontend Devs: Custom Docker Images for Development

Prerequisite: Create our project

Custom Docker Image for Development

Dockerfile Commands and Keywords

Dockerfile Structure Recommendations

Caching Dependencies

Priority

Building the Custom Docker Image

Running our Container: Quality of Life Improvements

What was added?

Mounting Host Directories

Accessing Ports Inside the Container

docker run in One Command

Testing it out

In Closing

Connecting Stripe events to the AWS EventBridge

Why AWS EventBridge?

The solution

Serverless components in stripe-eventbridge

Routing by event type

And that's all!

Running Jenkins and Persisting state locally using Docker

Jenkins setup

Persisting Jenkins Data

Confirming Jenkins state is persisted

Recovering Jenkins

In summary

CD with Docker and Feature Branch Testing

Prerequisite Knowledge

The Goal: Fast Feedback

Why should I bother?

Feature Branch Environments

`docker run` in One Command