Forem: Justus Soh

Using GitHub Actions to seamlessly deploy Expo applications (Part 2)

Justus Soh — Mon, 24 Aug 2020 04:45:45 +0000

Continuation

In part 1 of this series, I went through some of the pre-requisites to use Expo with GitHub actions and demonstrated how to set up your first action. If you want to learn more about how to run tests and publish branches, you can check out part 1 here!

In this section, I will explain the purpose of versioning and how to build the application using the Expo turtle service for the distribution of the application.

Importance of versioning

Why should we version? Is it necessary? How will this improve user experience? As the project grows, the number of dependencies grows and gets more difficult to manage. Eventually, one might find it difficult to upgrade without breaking contracts with other services. Version control is a simple way we can manage these dependencies with a common set of rules so that everyone can be on the same page. This is Semantic Versioning and follows a given rule in the format Major.Minor.Patch. Major changes include breaking API changes, Minor changes add functionality that is backward compatible and Patch adds bug fix.

The proper use of versioning could bring about a slew of advantages to the application. In-app versioning could allow for easy debugging of reported bugs by verifying code base and app version. Configuring over the air updates (OTA) to only send minor and patch updates would prevent breaking changes from bricking your application. This would also allow for different major versions of the application to existing and be maintained for users who haven't grab the latest version from the respective application stores.

In my implementation, I will specifically be using the GitHub release tag to trigger my workflows and also specify the name of the tag to match prod-Major.Minor.Patch so as to inform the action which release-channels to deploy the application to.

There are a lot of things going on in the code. I will first provide the code followed by a detailed explanation of some choices I have made.

name: Create Release
on:
  push:
    tags:
      - "prod-[1-9]+.[0-9]+.[0-9]+" # Push events to matching prod-*, i.e.prod-20.15.10

jobs:
  deploy_prod:
    name: Deploy To Production
    needs: test
    runs-on: ubuntu-latest
    outputs:
      releaseChannel: ${{ steps.releaseChannel.outputs.releaseChannel }}
      latestBinaryVersion: ${{ steps.latestBinaryVersion.outputs.version }}
    steps:
      - uses: actions/checkout@v2
        with:
          ref: ${{ github.head_ref }}
      - name: Fetch Tags
        run: |
          git fetch --prune --unshallow --tags -f
      - uses: actions/setup-node@v1
        with:
          node-version: 12.x
      - uses: expo/expo-github-action@v5
        with:
          expo-packager: npm
          expo-username: ${{ secrets.EXPO_CLI_USERNAME }}
          expo-password: ${{ secrets.EXPO_CLI_PASSWORD }}
          expo-cache: true
      - uses: rlespinasse/github-slug-action@v2.x
      - name: Generate Release Channel # Release Channels are named prod-<Major Release>, i.e. prod-1, prod-3
        id: releaseChannel
        run: |
          RELEASE_CHANNEL=$(echo ${{ env.GITHUB_REF_SLUG }} | sed -r 's/\.[0-9]+\.[0-9]+$//')
          echo "::set-output name=releaseChannel::$RELEASE_CHANNEL"
      - name: Install Packages
        run: npm install
      - name: Get Latest Binary Version # Binary Version will be x.x.x based on the latest tag
        id: latestBinaryVersion
        run: |
          # Release tag finds the lastest tag in the tree branch - i.e. prod-x.x.x
          RELEASE_TAG=$(echo $(git describe --tags --abbrev=0))
          # Using param substitution, we output x.x.x instead
          echo "::set-output name=version::${RELEASE_TAG#*-}"
      - name: Echo Version Details
        run: |
          echo Build number is $GITHUB_RUN_NUMBER
          echo Latest release is ${{ steps.latestBinaryVersion.outputs.version }}
      - name: Expo Publish Channel
        run: expo publish --non-interactive --release-channel=${{ steps.releaseChannel.outputs.releaseChannel }}

In the first few lines, we can see that this action will only occur if a tag with name prod-x.x.x is pushed. This gives us the flexibility to run the action either with the git command-line tool or the GitHub GUI create release page.

Next, we can take a closer look at the step Fetch Tag and Get Latest Binary Version. Normally if we were to just grab the latest tag from the repo, the latest release tag will be grabbed, so instead, I decided to make this fail-proof through only taking the latest tag from that branch we are referencing.

A buildNumber also must be provided to the app.json for the app to be built. In this case, I have chosen to use the GitHub actions run number (it is the small number beside the name of each run on the actions tab of the repo which counts the unique times the action is played not including reruns, you can find more info here).

In the case of Create Release out buildNumber will be 3

I also altered the release-channel for Expo to be represented as prod-Major grabbing only the major version from the latest tag as we want to route the OTA updates to those channels. This also allows us to do hotfixes for older versions of the application by creating release on another branch so nothing will be broken.

Writing releases

Being able to see and track changes from release to release is a game-changer. However, the hassle of creating a changelog for release to release brings about the stress of missing out on important changes. Fear not more, the community has written many some actions which we can utilize to help us generate a changelog from the past commits, this way no changes will be missed out and we can integrate it directly into the job. In my case, I have chosen metcalfc/changelog-generator because it's simple and gets the job done. After this is done, we will be using another community action ncipollo/release-action to create or update the existing release tag.

The example job is as follows:

create_release:
    name: Create Release
    needs: deploy_prod
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: rlespinasse/github-slug-action@v2.x
      - name: Generate Changelog
        id: changelog
        uses: metcalfc/changelog-generator@v0.4.0
        with:
          myToken: ${{ secrets.GITHUB_TOKEN }}
          base-ref: 'prod-0'
      - name: Creating Release
        uses: ncipollo/release-action@v1
        with:
          body: |
            Changes in this Release: 
            ${{ steps.changelog.outputs.changelog }}
          token: ${{ secrets.GITHUB_TOKEN }}
          name: Release ${{ env.GITHUB_REF_SLUG }}
          allowUpdates: true

Building and Distribution

The last and most important step in this would be building and distributing the application to the masses. The build time for the application is usually dependent on the number of projects queuing for the Expo build service, this process is usually quite lengthy (20 to 30 mins). Imagine waiting for about an hour just to build both Android and iOS binaries before every release, that would be a pain. But thankfully for us, we can write an action every release automatically and upload the binaries on the release itself or even the respective app stores! There are generally two approaches when it comes to using the free Expo build service that is to either wait for the service to complete before grabbing the URL or generated binaries or using a hook to send a POST request to a server when the build job is done. For simplicity and not wanting to spin up job listening for hooks in GitHub actions, I decided to go with the former option.

Here is a code snippet of how the job may look like for an android apk package, iOS would just be a mirror running build:ios instead:

 build_android:
    needs: [deploy_prod, create_release]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: rlespinasse/github-slug-action@v2.x
      - uses: expo/expo-github-action@v5
        with:
          expo-packager: npm
          expo-username: ${{ secrets.EXPO_CLI_USERNAME }}
          expo-password: ${{ secrets.EXPO_CLI_PASSWORD }}
          expo-cache: true
      - name: Install Packages
        run: npm install
      - name: Build Android Release
        env:
          APP_BUILD_VERSION: ${{ github.run_number }}
          APP_BINARY_VERSION: ${{ needs.deploy_prod.outputs.latestBinaryVersion }}
        run: |
          expo build:android --release-channel=${{ needs.deploy_prod.outputs.releaseChannel }} > buildLogAndroid.txt
          cat buildLogAndroid.txt
      - name: Parse Asset URL
        id: androidUrl
        run: |
          ASSET_URL=$(cat buildLogAndroid.txt | tail | egrep -o 'https?://expo\.io/artifacts/[^ ]+')
          echo The android url is $ASSET_URL
          echo "::set-output name=assetUrl::$ASSET_URL"
      - name: Download APK Asset
        run: wget -O example-${{ env.GITHUB_REF_SLUG }}.apk ${{ steps.androidUrl.outputs.assetUrl }}
      - name: Upload Release Asset
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: ./example-${{ env.GITHUB_REF_SLUG }}.apk
          asset_name: example-${{ env.GITHUB_REF_SLUG }}.apk
          tag: ${{ github.ref }}

One challenge I faced was grabbing the .apk asset when the build was complete. In some cases, using the expo url:apk would yield the wrong result as only the latest build would not be updated, hence I decided to stick to using good old trusty regex to solve the problem. After which the uploading of assets to the release was smooth sailing.

The final output of the release should look something like this! Concise with all the parts we need.

Improvements

Some improvements we could make would be to use a community action like GitHub Tag Bump to store the latest version as a tag and automatically bump up the version base on the PR description, but that will be a story for another time. Another area for improvement would be to automatically add the upload the built .apk and .ipa binaries to the app stores directly. If you would like to do that you can read more about the process here. There are definitely many other areas of improvement and I'll continue working on the repo to add functionality to the CI/CD workflow.

Conclusion

Overall, it has been a fun journey planning out how I wanted the workflow to run and how it should be implemented. Also, documenting the process was enjoyable so that others can learn from and use GitHub Actions for themselves. I want to give a shout out to my mentor Sebastian as well for guiding me in the process and reviewing the same jobs and workflows for another project.

You can check out the entire repo here! Do fork it and play around with it however you like.

justussoh / github-actions-expo-boiler-template

Expo working together with GitHub Actions starter

This is the example explained in a dev.to post, more details will be available soon.

View on GitHub

Using GitHub Actions to seamlessly deploy Expo applications (Part 1)

Justus Soh — Sun, 23 Aug 2020 12:53:57 +0000

Introduction

My first experience with mobile application development was using react native with Expo (in case you didn't know Expo is an open-source platform for making universal native apps for Android, iOS, and the web with JavaScript and React, this means that you didn't need to know java or swift to create native applications).

Coming from a React background, it was quite easy for me to adapt and pick it up. I was especially impressed with the ability to preview and live-reload the application on my phone just like a web application and also with the packages made available by Expo to interact with the phone hardware. I felt that all these improvements helped to improve the speed and quality during development.

However, the release process was not really ideal. As a newcomer, I was confused with the concept of publishing building and uploading to the respective app stores. Besides that, every new release involved getting a senior engineer to:

Find time to do a release
Publish the changes to a staging channel
Test that staging functionality is working
Build the application using Expo's instance of turtle Cli
Wait 30 minutes for each app binary to complete (iOS and Android)
Uploading of files to the Testflight/Google store

What if there was an easier way to update the application? I wanted to improve on this existing flow making sure to automate the testing and release process.

The following article will be a 2 part series on first setting up Continuous Integration (applying tests and publishing PR reviews) then the second part will cover Continuous Deployment with GitHub Actions for your Expo projects.

An integrated solution

I have been using GitHub actions for a while now as a CI/CD, it is not only versatile in allowing for self-host runners but also comes with a slew of actions made by the community. It is also free for any public repository making it a very appealing choice for our project.

Expo also offers a GitHub action tool called expo/expo-github-action which allows us to run expo commands in our workflows. This means that we can publish and review any version of the code to ensure everything is working.

Setting up Expo deployments

Before we get started, we need to sign up for an Expo account in order to publish applications. We will then need to store these credentials within the secure GitHub Secrets Store for later use. I recommend using EXPO_CLI_USERNAME and EXPO_CLI_PASSWORD variables. You can find out how to store secrets on the GitHub documentation site.

For now, we only plan to publish the Expo application for user testing. However, for distributing and uploading applications to the respective stores, a detailed guide can be found on here on the Expo site.

Running Unit Test and Linting

Testing is essential, its purpose is to validate that each unit of the software code performs as expected and helps engineers to spot bugs early before production, bring cost and productivity savings late on.

To begin we first have to make a .github/workflows directory where we will be storing all our workflows.

We can use the following example as a boiler template to perform our test automatically on each pull request:

name: Deploy Branch Preview

on: [pull_request]

jobs:
  test:
    name: Lint & Test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
        with:
          node-version: 12.x
      - name: Cache Node Modules
        uses: actions/cache@v2
        env:
          cache-name: cache-node-modules
        with:
          # npm cache files are stored in `~/.npm` on Linux/macOS
          path: ~/.npm
          key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-build-${{ env.cache-name }}-   
            ${{ runner.os }}-build- 
            ${{ runner.os }}-
      - name: Install Packages
        run: npm install
      # Peform a type check if you are using typescript
      - name: Typecheck
        run: npx --no-install tsc --noEmit
      - name: Check Lint
        run: npm run lint
      - name: Test
        run: npm run test -- --coverage

The repo link can be found at the bottom of the article

Automated PR deployments

One thing I enjoy and find especially helpful in my react native development process is to publish and check the application on code review before pushing any code. This allows me to better understand the changes made in that pull request and provide a better review.

The following piece of code is a job to publish a release for a PR review on the Exponent server:

  deploy_branch_preview:
    name: Deploy Branch Preview
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
        with:
          node-version: 12.x
      - uses: expo/expo-github-action@v5
        with:
          expo-packager: npm
          expo-username: ${{ secrets.EXPO_CLI_USERNAME }}
          expo-password: ${{ secrets.EXPO_CLI_PASSWORD }}
          expo-cache: true
      - name: Cache Node Modules
        uses: actions/cache@v2
        env:
          cache-name: cache-node-modules
        with:
          path: ~/.npm
          key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-build-${{ env.cache-name }}-
            ${{ runner.os }}-build-
            ${{ runner.os }}-
      - name: Install Packages
        run: npm install
      - name: Expo Publish Channel
        run: expo publish --non-interactive --release-channel pr${{ github.event.number }}
      - name: Add Comment To PR
        uses: mshick/add-pr-comment@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          EXPO_PROJECT: "@justussoh/github-actions-expo-boiler-template" # Put in your own Expo project name here
        with:
          message: |
            ## Application
            ![Expo QR](https://api.qrserver.com/v1/create-qr-code/?size=250x250&data=exp://exp.host/${{ env.EXPO_PROJECT }}?release-channel=pr${{ github.event.number }})
            Published to https://exp.host/${{ env.EXPO_PROJECT }}?release-channel=pr${{ github.event.number }}

The repo link can be found at the bottom of the article

Deploying always generates a fixed URL for anyone to open the application. Here I used a QR generator to churn a QR from the link for the application which will be commented on the pull request so that anyone can easily check it.

You can the application via scanning the QR code with your Expo client application.

Publishing to a staging environment

To further maximize efficiency, we can also configure the action to deploy to a staging environment after merging a pull request into master, this was a staging env will always be present before posting to production.

The code will be relatively similar to the PR actions, therefore I will link the file here.

Conclusion

In the above to parts, we achieved a robust CI to help run tests and publish the application for user testing. Keep a lookout for part 2 where I will be explaining how to version and publish your application to the app store to allow for over the air updates to your application. If you want to check out the full repo, the link is down below.

justussoh / github-actions-expo-boiler-template

Expo working together with GitHub Actions starter

This is the example explained in a dev.to post, more details will be available soon.

View on GitHub