Forem: Cardinal

Storing currency values: data types, caveats, best practices

Cardinal — Tue, 10 Jan 2023 20:34:44 +0000

Intro

Repeatedly facing questions, debates, and mistakes related to storing and representing currency amounts, I decided to collect all facts and advice regarding this topic in one place. This article is not the final source of the truth, but it contains useful information that you should take into consideration when designing software.

Domain info: currencies

Just facts:

There is an ISO 4217 standard that describes currency codes and their minor units. Data are also available in XML and CSV representations (following the links from the page).

A currency has a 3-letter code, a numeric code, and a name. A currency may have multiple listed locations.
Some currencies have exchange rates that are pegged (fixed) to another currency.
Minor unit is the smallest unit of a currency, e.g. 1 dollar equals 100 cents (with 2 decimals).
Most currencies have 2 decimals; some have none, and some have 3 decimals.
Mauritania and Madagascar does not use a decimal division of units, setting 1 ouguiya = 5 khoums, ariary = 5 iraimbilanja.
Cryptocurrencies can have up to 18 decimals (ETH).
The number of decimals can change over time due to inflation.
The same can happen because of redenomination, but a new currency code should be introduced.
For some currencies, there are no physical denominations for the minor unit.
Storing prices of small units of goods (probably as a result of conversion from another currency) can require using more decimals than are defined for a currency.

Storage requirements

Obvious one: store currency amounts along with a link to the currency specification (foreign key in databases, special class in programming languages) to interpret and operate with it correctly.
Storing a specification for a currency you should include:
- Minimum accountable unit instead of or in addition to precision (see fact 5).
- Lowest physical denomination of the currency if you deal with cash operations (see fact 9).
Ensure precision for currency amounts equals the max precision of all supported currencies.
Consider adding additional precision for operational needs: accumulators and intermediate calculations or for storing prices of small units of goods. For example, you may want to accumulate a 10 % fee from 1 cent operations, sum them up until they reach the minimum accountable unit (cent) and withdraw from a client.

Data types

There are different data types that can technically store money values. Let’s see how the listed requirements can be fulfilled by utilizing different data types.

1️⃣ Integer number of minor units

One of the popular (Stripe approaches) is storing an integer number of minor units. Simply put, you store 5 $ as 500 cents. This way you can do accurate calculations and comparisons internally and then display the result formatting the number in a proper way as an amount of dollars.

Taking into consideration the requirement about additional precision (let's say 3 extra decimals) you will represent 5 $ as 500 000 of "micro units", where 1 micro unit equals 1 000 cents.

Issues and limitations:

It's preferable to consider the precision beforehand.
It's complicates the business logic of the application and introduces error-prone value conversions between units, micro-units and normal currency amounts that are presented to a user or external systems.
Due to the fact 7 (minor unit of a currency can change) or because of the need to add additional precision you may need to rescale all values in the future.
External systems you interact with can misinterpret the magnitude of an integer-represented value after rescaling:
- 3rd-party services/customers which are not aware of the rescaling.
- Your own services that can't be immediately updated with the new definitions of the currencies (limitation of the deployment process, caches).
- A message queue, or an event streaming platform where you can't modify messages during the rescaling process.
This problem can be solved by explicitly passing the scale of a number everywhere along with the number.

Suggested type: BigInt

It's a not-native data type that handles integers of arbitrary (or big enough) size and provides math operations on it. It's the most suitable way of storing minor units or micro units. Don't confuse it with SQL bigint that in fact is Int64.

Most of the languages (
JavaScript,
PHP,
Go,
Python,
Java,
C#,
C++
) support it natively or with 3rd-party libraries.
To store these values in databases you should use Decimal column type with precision = 0 (
SQL databases,
MongoDB
).

For correct (de)serialization you will probably need to use strings for compatibility between different implementations.

Usually, more memory is required by BigInt type, compared to native integers, and computations take longer because CPUs don't have hardware support for this data type.

Actually, it can depend on the binary representation of a value in a concrete implementation.
Normally, standard libraries provide optimal
(2^32-base, or 2^64-base) representation and have only constant overhead, while 3rd-party string-based
(C++, PHP) types have linear
overhead (due to 10-base representation). In most databases the binary representation of decimal type is not
optimal (
100-base or
10000-base).

With some concerns: Int64

Some choose signed or unsigned Int64 (also referenced as BigInt in SQL, which can lead to confusion) for storing their cents or smaller subunits of currency. Even though it may seem to be sufficient for your use-case, I want to highlight the following issues with it:

It may not be sufficient for storing minimal units of cryptocurrencies (see fact 6) or values with extended precision (see requirement 5).
Some languages require casting to Float64 for math operations (Go). The problem is Float64 has only 52 bits of mantissa; it's not enough to fit arbitrary Int64 value.
Not all programming languages support Int64 values:
- PHP running on x32 architectures cannot handle Int64 values.
- Some languages (Java, PHP) do not support any unsigned integers.
- JavaScript uses signed Float64 as an internal representation for the number data type. This means that even if one can serialize Int64 numbers to JSON in their backend application, by default a JavaScript application will overflow its number type trying to deserialize JSON containing this value.

The problem with support of Int64 in external systems can be mitigated by serializing Int64 to a string and using BigInt types to handle these values, but it reduces the benefits from using hardware-supported Int64 values.

Int64 in JavaScript

The problem with JavaScript numeric type is relevant if you use JSON.parse() and JSON.stringify() without
additional arguments - as many "HTTP request" libraries do. If you have control over these calls you can pass custom
replacer
argument for JSON.stringify() and custom
reviver
argument for
JSON.parse()
and manually handle Int64 values using a data type different from default number.

What could this "different" type be?

Built-in BigInt64Array type

Built-in BigInt type. Note, that it has limited math operations support.

3rd-party libraries.

2️⃣ Decimal

This approach implies using special Decimal numeric type that allows to store fractional decimal numbers accurately with the specified precision (maximum precision differs in different databases).

Most languages (
JavaScript,
PHP,
Go,
Python,
Java,
C#,
C++
) have built-in support or 3rd-party libraries for handling this data type.
SQL databases offer own standard Decimal
type.

Note that there are 2 different types of decimal type implementations:

Decimal128 with limited number of significant digits (MongoDB, C#).
So-called BigDecimal type of arbitrary (or big enough) size (SQL Decimal, BigDecimal in Java) with different internal representations.

As in the case of BigInt, the binary representation can differ:

2 decimal digits in each byte (base 100, like it is done in databases).

BigInt with exponent in a manner similar to base 2 floating point values.

Array or string of single decimal digits.

The performance concerns described above for BigInt are relevant for Decimal types as well.

The main advantages of using Decimals comparing to BigInts are:

No major overhead if values are already stored as Decimals in the database (you'll do it anyway even with BigInts). Also, formatting BigInt values requires computations similar to those within Decimal type.
More intuitive representation of the values simplifies the business logic and formatting numbers to a human-readable format.
Changing minor units can be done by altering the precision of the decimal column in the database; you don't have to rescale the values.
Value serialized to string naturally includes the information about precision. If you change the precision, it cannot be misinterpreted by external systems as in the case of BigInts.

As in the case of BigInt values, all serialization goes through decimal strings for compatibility between different libraries. Of course, serializing in the form of mantissa + exponent can be more efficient for performance-sensitive applications.

However, you still need to keep track of minimal accountable units. Limiting the precision of Decimal in the database requires remembering about precision reserve for intermediate operations and accumulators.

Bad choice

Float, Double

The first rule here is "never use floating-point data types for storing money amounts". Humans expect money calculations to be made in base 10, but floating-point arithmetic uses base 2 representation that can lead to results that are not expected in financial sense. The common example to illustrate the problem is 0.1 + 0.2 != 0.3. This rule is relevant for both programming languages and databases.

Even though decimal representation also can't store all amounts precisely (e.g. 1/3 - read bonus part at the end),
it's expected by people, financial institutions and regulations.

SQL Server MONEY

This proprietary type of Microsoft SQL Server stores amounts as Int64 internally and
is not recommended
to use.

Bonus: rational numbers

Rational number is a number that can be expressed as the fraction A/B. In most applications, their use is not necessary, however, for some use cases, they are the only option for obtaining the desired precision.

I can imagine a game or an expenses splitting application in which you need to take 1/3 of 10 $, save it (to a database, message queue, or simply to memory) and later multiply it by 30. Using decimals (both Decimal type and BigInt type with the number of cents) can't provide exact precision - the result will never be exactly 100. But using rational numbers, you can save the intermediate value as numerator = 10, denominator = 3 and later do simple arithmetic to get the value 100/1.

Operations on non-decimal units, such as time and distance units, historical non-decimal currencies, calculating probabilities, etc., may be candidates for introducing rational numbers when absolute precision is required.

There is standard Fractions library in Python, and 3rd-party libraries in other languages:
JavaScript,
PHP,
Go,
C#,
C++.

Unfortunately, the only database I know to be supporting rational numbers is PostgreSQL with pg_rational extension. Storing rational numbers in separate "numerator" and "denominator" columns limits the possibilities of math calculations directly in the database.

👏 Thank you for reading

Any comments, criticism, and sharing of your own experience would be appreciated!

The main workflow

Cardinal — Tue, 15 Mar 2022 10:31:08 +0000

We have prepared all the needed composite actions and workflows, and we are finally ready to create the main workflow that triggers the entire pipeline.

.github/workflows/publish-release-on-tag.yml :

name: Release and publish on tag
on:
  push:
    tags:
      - '*.*.*'
  workflow_dispatch:
jobs:
  build-release-publish:
    if: github.ref_type == 'tag'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - uses: cardinalby/export-env-action@v1
        with:
          envFile: './.github/workflows/constants.env'
          expand: true

      - name: Look for an existing release
        id: getRelease
        uses: cardinalby/git-get-release-action@v1
        continue-on-error: true
        with:
          tag: ${{ github.ref_name }}
        env:
          GITHUB_TOKEN: ${{ github.token }}

      - name: Build, test and pack to zip
        id: buildPack
        if: steps.getRelease.outcome != 'success'
        uses: ./.github/workflows/actions/build-test-pack

      - name: Create Release
        id: createRelease
        if: steps.getRelease.outcome != 'success'
        uses: ncipollo/release-action@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          draft: 'true'

      - name: Upload zip asset to the release
        if: steps.getRelease.outcome != 'success'
        uses: actions/upload-release-asset@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          upload_url: ${{ steps.createRelease.outputs.upload_url }}
          asset_path: ${{ env.ZIP_FILE_PATH }}
          asset_name: ${{ env.ZIP_FILE_NAME }}
          asset_content_type: application/zip

      # Should trigger build-assets-on-release.yml
      - name: Publish release
        if: steps.getRelease.outcome != 'success'
        uses: eregon/publish-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.WORKFLOWS_TOKEN }}
        with:
          release_id: ${{ steps.createRelease.outputs.id }}

      - name: Publish on Chrome Webstore
        uses: benc-uk/workflow-dispatch@v1
        if: "!contains(github.event.head_commit.message, '[skip chrome]')"
        with:
          workflow: publish-on-chrome-web-store
          token: ${{ secrets.WORKFLOWS_TOKEN }}
          wait-for-completion: false

      - name: Publish on Firefox Add-ons
        uses: benc-uk/workflow-dispatch@v1
        if: "!contains(github.event.head_commit.message, '[skip firefox]')"
        with:
          workflow: publish-on-firefox-add-ons
          token: ${{ secrets.WORKFLOWS_TOKEN }}
          wait-for-completion: false

The workflow can be triggered by pushing ..* tag or by workflow_dispatch event.
To perform the work we need a tag to create a release for, that's why we add if: github.ref_type == 'tag' condition for the job to prevent running on branches.
We use git-get-release-action to find a release for the tag. continue-on-error: true prevents the job from failing if release not found.
If a release not found, we:
- Call build-test-pack composite action to build zip file.
- Call release-action to create a draft release. This doesn't trigger on: release event.
- Call upload-release-asset to upload zip asset to the release (to be used by publish-on-chrome-web-store and publish-on-firefox-add-ons workflows later).
- Call eregon/publish-release to publish the draft release. This triggers on: release event and build-assets-on-release workflow.
We use benc-uk/workflow-dispatch action to asynchronously dispatch:
- publish-on-chrome-web-store workflow (if the commit message doesn't contain [skip chrome] text).
- publish-on-firefox-add-ons workflow (if the commit message doesn't contain [skip firefox] text).

👏 Thank you for reading

Finally, we are done! You can try out your first deployment by pushing a new ..* tag to the repo (don't forget to check the extension version in manifest.json).

Any comments, critics and sharing your own experience would be appreciated.
You can find a real example of the described CI/CD approach in my Memrise Audio Uploader extension.

Don't let Google refresh token expire

Cardinal — Tue, 15 Mar 2022 10:26:17 +0000

In this part we will finish setting up workflows related to publishing the extension on Google Web Store and create the workflow that isn't shown on the workflow diagram because it stays aside and isn't included to the main pipeline.

Dealing with Google API credentials we should be aware of the fact that, according to Google's guide, refresh token (which we use in the workflow from the previous part) might stop working if it has not been used for six months. And I can assure you, it happens 😞.

To solve the issue we can use a great option GitHub offers called "schedule event" for workflows.

.github/workflows/touch-google-refresh-token.yml :

name: Touch google token
on:
  schedule:
    - cron:  '0 3 2 * *' # At 03:00 on day-of-month 2
  workflow_dispatch:
jobs:
  fetchToken:
    runs-on: ubuntu-latest
    steps:
      - uses: cardinalby/google-api-fetch-token-action@v1
        with:
          clientId: ${{ secrets.G_CLIENT_ID }}
          clientSecret: ${{ secrets.G_CLIENT_SECRET }}
          refreshToken: ${{ secrets.G_REFRESH_TOKEN }}

Here I assume that you have already obtained and have added values required for Google API access to secrets (as it described at the previous post).

Once a month GitHub will run our workflow and perform the single step: fetching access token using the credentials that we have already added to secrets (see the previous post). It will prevent refresh token from invalidation.

One note here: GitHub will suspend the scheduled trigger for GitHub Action workflows if there is no commit in the repository for the past 60 days. The cron based triggers won't run unless a new commit is made. Probably, you can use the same trick (committing to the repo once a month by schedule) to circumvent this limitation.

Publish on Chrome Web Store

Cardinal — Tue, 15 Mar 2022 10:20:01 +0000

In this part we are going to create the workflow that will be responsible for publishing the extension
on Chrome Web Store. This part is going to be a bit tricky comparing to the others.

🧱 Prepare

❶ To set up Google Publish API access you need to obtain clientId, clientSecret and refreshToken from Google. These articles can help you to do that:
* Using the Chrome Web Store Publish API
* How to generate Google API keys

🔒 Add to the repository secrets:

G_CLIENT_ID
G_CLIENT_SECRET
G_REFRESH_TOKEN

❷ You should find out the ID of your extension on Chrome Web Store. Normally, it is shown on your Developer Dashboard. It's not private information, but I prefer to store it in secrets.

🔒 Add to the repository secrets:

G_EXTENSION_ID

❸ Also, we have to create a new personal access token with write access to the repo. The automatically provided token e.g. secrets.GITHUB_TOKEN can not be used, GitHub prevents this token from being able to fire the workflow_dispatch and repository_dispatch event.

🔒 Add to the repository secrets:

WORKFLOWS_TOKEN

❹ The last preparation step is creating an environment in the repository settings. It's main purpose is providing a wait timer to delay a workflow run (details will be explained below). Let's call the environment 12hoursDelay and set its wait timer equal 720 minutes (12 hours).

publish-on-chrome-web-store workflow

On the one hand, the workflow will be similar to publish-on-firefox-add-ons workflow (described in the previous part): it is also triggered by workflow_dispatch event (emitted manually or by publish-release-on-tag) and retrieves zip asset in the same way.

On the other hand, it has its own complications because of peculiarity of Webstore publishing. Also, we are going to include the additional job for downloading published crx file from Webstore and attaching it to the release (if any).

The publishing process consists of 2 steps: uploading a new version and publishing the extension. What is the peculiarity I'm speaking about?

It's a Webstore behavior in the case when you are trying to upload a new version shortly after a previous one was uploaded to the store. In this case API call completes with an error and with the status called IN_REVIEW (only one uploaded version can be in processing at the time). Unlike other errors, it doesn't mean that something is wrong with our extension and if we try uploading later it will succeed.

That's exactly what we are going to do, it only remains to find a good technical solution to do that.
If you are interested in learning about existing approaches, please read my "GitHub Actions: implementing deferred steps" post. Here we will
use "Dispatched workflow calling itself + wait timer" approach to repeat a new version uploading after 12 hours delay (number was chosen arbitrary).

Let's observe the workflow diagram and start with creating the workflow file:

As you can see, the main idea is the following:

We try to upload a new extension version.
If it succeeded, we proceed to publishing the extension and downloading the published crx file.
If it didn't succeed due to IN_REVIEW error, we dispatch the workflow with 12 hours delay to repeat it later. We will limit attempts number by incrementing an attemptNumber and passing it as an input to the workflow.

.github/workflows/publish-on-chrome-webstore.yml :

name: publish-on-chrome-web-store
on:
  workflow_dispatch:
    inputs:
      attemptNumber:
        description: 'Attempt number'
        required: false
        default: '1'
      maxAttempts:
        description: 'Max attempts'
        required: false
        default: '10'
      environment:
        description: 'publish-on-webstore job environment'
        required: false
        default: ''
jobs:
  # We will add 2 jobs here:
  # publish-on-webstore:
  #   ...
  # download-published-crx:
  #  ...

Defined workflow_dispatch event has 3 inputs that can be specified at the time of dispatching the workflow. You will see their usage in the following job. The first time the workflow is triggered, inputs will have their default values.

publish-on-webstore job

  publish-on-webstore:
    runs-on: ubuntu-latest
    environment: ${{ github.event.inputs.environment }}
    outputs:
      result: ${{ steps.webStorePublish.outcome }}
      releaseUploadUrl: ${{ steps.getZipAsset.releaseUploadUrl }}
    steps:
      # Validate the inputs and increase the attemptNumber if less than maxAttempts
      - name: Get the next attempt number
        id: getNextAttemptNumber
        uses: cardinalby/js-eval-action@v1
        env:
          attemptNumber: ${{ github.event.inputs.attemptNumber }}
          maxAttempts: ${{ github.event.inputs.maxAttempts }}
        with:
          expression: |
            {
              const 
                attempt = parseInt(env.attemptNumber),
                max = parseInt(env.maxAttempts);
              assert(attempt && max && max >= attempt);
              return attempt < max ? attempt + 1 : '';
            }

      - uses: actions/checkout@v2

      - uses: cardinalby/export-env-action@v1
        with:
          envFile: './.github/workflows/constants.env'
          expand: true

      - name: Obtain packed zip
        id: getZipAsset
        uses: ./.github/workflows/actions/get-zip-asset
        with:
          githubToken: ${{ secrets.GITHUB_TOKEN }}

      - name: Fetch Google API access token
        id: fetchAccessToken
        uses: cardinalby/google-api-fetch-token-action@v1
        with:
          clientId: ${{ secrets.G_CLIENT_ID }}
          clientSecret: ${{ secrets.G_CLIENT_SECRET }}
          refreshToken: ${{ secrets.G_REFRESH_TOKEN }}

      - name: Upload to Google Web Store
        id: webStoreUpload
        continue-on-error: true
        uses: cardinalby/webext-buildtools-chrome-webstore-upload-action@v1
        with:
          zipFilePath: ${{ env.ZIP_FILE_PATH }}
          extensionId: ${{ secrets.G_EXTENSION_ID }}
          apiAccessToken: ${{ steps.fetchAccessToken.outputs.accessToken }}
          waitForUploadCheckCount: 10
          waitForUploadCheckIntervalMs: 180000    # 3 minutes

      # Schedule a next attempt if store refused to accept new version because it
      # still has a previous one in review
      - name: Start the next attempt with the delay
        uses: aurelien-baudet/workflow-dispatch@v2
        if: |
          steps.getNextAttemptNumber.outputs.result && 
          steps.webStoreUpload.outputs.inReviewError == 'true'
        with:
          workflow: ${{ github.workflow }}
          token: ${{ secrets.WORKFLOWS_TOKEN }}
          wait-for-completion: false
          inputs: |
            { 
              "attemptNumber": "${{ steps.getNextAttemptNumber.outputs.result }}",
              "maxAttempts": "${{ github.event.inputs.maxAttempts }}",
              "environment": "12hoursDelay"
            }

      - name: Abort on unrecoverable upload error
        if: |
          !steps.webStoreUpload.outputs.newVersion &&
          steps.webStoreUpload.outputs.sameVersionAlreadyUploadedError != 'true'
        run: exit 1

      - name: Publish on Google Web Store
        id: webStorePublish
        uses: cardinalby/webext-buildtools-chrome-webstore-publish-action@v1
        with:
          extensionId: ${{ secrets.G_EXTENSION_ID }}
          apiAccessToken: ${{ steps.fetchAccessToken.outputs.accessToken }}

environment: ${{ github.event.inputs.environment }} sets the environment for the job providing a required delay (via "wait timer" of the environment).
We use js-eval-action as a generic JS code interpreter to validate the inputs and calculated the incremented attemptNumber. It is accessible as result output of the step.
As usual, at the beginning of each workflow we check out the repo and export env variables from constants.env file.
After calling get-zip-asset composite action we expect to have zip file with packed and built extension at env.ZIP_FILE_PATH path.
We call google-api-fetch-token-action to retrieve Google API access token that is needed for "upload" and "publish" steps.
We use webext-buildtools-chrome-webstore-upload-action action to upload a new version to Webstore. continue-on-error: true flag allows us not to fail immediately, but perform the next "dispatching" step and examine the error after it in a separate step (checking the action's outputs).
We use aurelien-baudet/workflow-dispatch action to dispatch the workflow in case of IN_REVIEW error:
- workflow: ${{ github.workflow }} points to the current workflow file
- token: ${{ secrets.WORKFLOWS_TOKEN }} makes use of the personal access token we created at the preparation step
- inputs is JSON containing values of inputs for workflow_dispatch event:
  - attemptNumber is an incremented workflow input value read from the validation step.
  - maxAttempts is the workflow input value passed without changes.
  - environment is the name of the environment that will be used to delay the execution.
"Abort on unrecoverable upload error" step complements the "upload" step validating errors and fails the job if we can't proceed with publishing because the new version was not uploaded. The case when the same version has been already uploaded (sameVersionAlreadyUploadedError output indicates that) is the exception - we still can publish it.
Finally, we call webext-buildtools-chrome-webstore-publish-action action to publish the extension.

Finally, we are done. Keep in mind that in the sake of simplicity I omit some details and description of optional inputs of the used actions. There are a lot of things to tune up. Please, read the documentation for corresponding actions to learn more.

In the next part we are going to solve one small but annoying issue with Google API refresh token expiration.

System testing of GitHub Actions

Cardinal — Tue, 08 Feb 2022 21:25:36 +0000

It's the last and the shortest part of the series. Testing the whole action as a black box can be done in 2 ways (as far as I can see).

github-action-ts-run-api again

Use the same tool as for integration test, but run tests against the whole action.

Use Act tool

This approach implies that you should create special testing workflows that can be naturally run on GitHub Actions runner or can be run locally using Act.

If you need to debug an action on actual GitHub hosted runner, take a look at debugging-with-tmate action on the Marketplace.

Testing workflows can be located in the action repository and refer to the action using the local ./ path. It's especially convenient because each branch in the action repo will be tested against the version of the action stored in this branch.

Here is a fragment of the testing workflow for git-get-release-action:

- name: Get 1.1.1 release by releaseId
  id: getByReleaseId
  # referring to the action in the current repo
  uses: ./  
  with:
    releaseId: 41301084

- name: Check getByReleaseId step result
  if: steps.getByReleaseId.outputs.tag_name != '1.1.1'
  shell: bash
  run: exit 1

Another option is creating a dedicated repo for testing workflows. It can also contain commits, releases, issues and other GitHub objects that can be read and modified by the action without bloating your main repository.

The drawback of this approach is that you have to specify an exact version of the action in uses key of each step:

- name: Get 1.1.1 release by releaseId
  id: getByReleaseId
  # referring to the specific version (v1)
  uses: cardinalby/git-get-release-action@v1
  with:
    releaseId: 41301084

If you have multiple branches in the main repo and want to test them, you should create the same
branches in the tesing repo and change all uses keys to the appropriate version (action-name@myBranch).

👏 Thank you for reading

Any comments, critics and sharing your own experience would be appreciated!

If you are interested in developing own Actions, you can read my other posts:

Testing of Docker Actions

Cardinal — Tue, 08 Feb 2022 20:41:57 +0000

In this part I'm going to tell about approaches that can be used to test a Docker container Action.

Unit tests

An approach here depends on what programming language you use inside a container. Each of them has own testing libraries that can be used to test an application in the container.

If you use bare bash script, you can divide a single entrypoint.sh file into the several small scripts considering them as units and testing separately.

The following approaches can be used to isolate their effects:

Redirecting scripts output to a temporary file.
Interacting with the main script through environment variables.
Don't hardcode path of filesystem that is modified by scripts, use env variables for that. It will allow you to use temp directories and files during testing.

Take a look at bash_test_tools - it's an analog of JavaScript test frameworks for bash scripts and executables.

It's better to use an environment similar to the production one in tests. Thankfully, it's not a problem with Docker containers. Just create a separate Dockerfile that runs tests in the same environment that the main Dockerfile defines.

Integration tests

github-action-ts-run-api can run Docker actions locally on Linux with native Docker, on MacOS and Windows via Docker Desktop and on any CI where Docker is installed (only Linux GitHub-hosted runners have docker support at the moment).

You still have the same TypeScript API for running, preparing all inputs, environment and examining results of an action execution.

Actually, it looks more like a system test (because action is black-boxed and executed as a whole) run using TypeScript API instead of normal yml workflow syntax. If you want to perform an integration test of only of some part of your code, probably, you need to create a separate Dockerfile and use github-action-ts-run-api to run it.

Working with a file system in a Docker action follows the common principle: don't use hardcoded paths, rely on the environment variables provided by GitHub instead.

github-action-ts-run-api allows you to prepare contents of all these dirs and files before the tested action run and examine their contents after it has finished. By default, temporary directories and files are created and attached as volumes to the tested action container.Corresponding environment variables are set to point to the mounting points of these volumes.

Stubbing external services

Advice here is similar to the one given for JavaScript actions. Don't hardcode URLs of external services in your code:

For GitHub API use the dedicated GITHUB_API_URL, GITHUB_SERVER_URL, GITHUB_GRAPHQL_URL environment variables.
For other external services introduce custom env variables that can be set in tests. In they are empty action just uses real production URLs.

NodeJS server

As in the case of JavaScript actions testing you can easily create a stub HTTP server right in the test suite managed by NodeJS and pass a base URL of the created stub server through environment variables to the tested action.

In this case the base URL of stub server should be:

host.docker.internal for Docker Desktop
172.17.0.1 for native Linux Docker

Check out the example.

Docker container server

Since we test a docker container action it would be natural to spin up a stub HTTP server in a docker container that will run along with the tested docker action container.

The best way of doing it is to create a docker-compose.yml file that defines a service with the stub HTTP server and a named network:

version: "3.5"
services:
  fake-server:
    build:
      # Image with stub GitHub API HTTP server
      context: httpServerDir
    ports:
      - "80:80"
networks:
  default:
    # Assign a name to the network to connect 
    # the tested action container to it
    name: testNet

Execute docker compose up before testing the action container.
Attach the action container to the named network (testNet in the example) and pass a base URL of the created stub server through environment variables to the tested action (fake-server in the example).
Execute docker compose down after the action container finished.

There is convenient withDockerCompose() utility function in github-action-ts-run-api package for performing these steps:

await withDockerCompose(
    'path/to/docker-compose.yml',
    async () => {
        const target = RunTarget.dockerAction(
            'path/to/action.yml', 
            // network name defined in docker-compose.yml    
            {network: 'testNet'}
        );
        const res = await target.run(RunOptions.create()
            // service name defined in docker-compose.yml
            .setGithubContext({apiUrl: `http://fake-server:80`})
        );
        assert(res.commands.outputs.out1 === 'fake_response');
    });

Check out the example.

Remarks

🔻 Docker Desktop for Windows and MacOS behaves differently from native docker on Linux. Be aware!

🔻 Windows and MacOS GitHub hosted runners don't have installed docker.

Testing of JavaScript Actions

Cardinal — Tue, 08 Feb 2022 19:26:53 +0000

Let's talk about JavaScript GitHub Actions and approaches that we can apply on the different levels of testing.

Unit tests

From my point of view, unit testing of Actions doesn't have any differences from testing any other JavaScript code. In most of the examples of Actions available on the GitHub Marketplace authors don't care about writing testable code. But nothing prevents you from extracting abstractions and following The Dependency Inversion Principle which will allow you to easily mock dependencies (such as @actions/core , @actions/github, @actions/exec packages).

You can use any of JavaScript testing frameworks and mocking libraries. I can advise taking a look at actions-mocks package.

Integration tests

I'm not going to duplicate the documentation for the github-action-ts-run-api package here, but I want to mention how it can be used.

Test a separate JS function

You can import the main function of your action or the function that implements a part of action's logic and run it in the same process as testing code (but still have all required isolation).
It's up to you to decide what granularity of testing you need.

As in unit tests you can mock external services, but all outputs, inputs and environment will be handled by the package, no mocks here.

Check out the example of testing a simple function.

Remarks

🔻 process.exit(...) calls inside a function are not mocked and will lead to process termination without cleaning up test environment. Try to avoid them.

🔻 Keep in mind that require("@actions/github").context is cached inside the actions library which can cause troubles if you run multiple test cases. To get it around you can:

Use new (require("@actions/github/lib/context").Context)() instead.
Call jest.resetModules() (or its analog) after each test case run.

Test a JS file

It's executed in a child node process. It can be the main packed file of an action (specified in runs.main section of action.yml file) or one of source JS files. Normally, you pack a JS action to a single file using tools like
ncc before publishing.
It makes debugging difficult if you use run test agains the packed file.

Testing js files you can't directly mock classes and functions (which is ok for integration testing), but instead you should have the entire external services stubs.

Filesystem modifications

Working with a filesystem follows the common principle: don't use hardcoded paths, rely on the environment variables provided by GitHub instead.

github-action-ts-run-api allows you to prepare contents of all these dirs and files before the tested action run and examine their contents after it has finished. By default, temporary directories and files are created and corresponding environment variables are set to point to these temporary paths.

Stubbing external services

Don't hardcode URLs of external services in your code:

For GitHub API use the dedicated GITHUB_API_URL, GITHUB_SERVER_URL, GITHUB_GRAPHQL_URL environment variables.
For other external services introduce custom env variables that can be set in tests. In they are empty action just uses real production URLs.

It's easy to create a stub HTTP server right in the test suite and set these environment variables to localhost URLs.

Check out the example of stubbing GitHub API called using octokit library.

Testing of GitHub Actions. Intro

Cardinal — Mon, 07 Feb 2022 15:09:35 +0000

Introduction

In this post series I want to share my experience and approaches with testing of GitHub Actions. Not using them to test your application, but test actions itself. I will mostly talk about testing of individual actions, not workflows.

Individual actions (steps) are "bricks" that workflows are built from, and we can consider testing them as unit testing of workflows.

One of the problems of GitHub Actions as cloud-based service is that there is no out of the box way of test them locally. Also, support in developing tools is poor comparing to mainstream programming languages. These factors lead to the high errors rate and long feedback loop to find and fix these errors.

That's why I believe it's important to adapt best practices we use in software testing for GitHub Actions, and I'm going to share my vision in it.

Overview

In the first part I give a general information about GitHub Actions and testing levels. Then I formulate requirements for testing tools and tell about my choise.

If you want to see concrete recommendations and approaches, just jump to the next part.

Action types

At the moment, GitHub supports 3 kinds of Actions which I will refer to in this post:

Levels of testing and tools

🔸 Unit testing

A Unit is a smallest testable portion of system or application which can be compiled, liked, loaded, and executed.

Depending on the action type, "unit" notion may have different meaning. I will cover it in
"Docker actions" and "JavaScript actions" parts.

For composite actions, individual steps can be considered units. If you don't hardcode runs commands in steps, but extract them to the separate actions instead (thankfully, they can be saved locally in the repo), then the unit testing approach reduces to the testing of individual actions. That's exactly what this post is about.

🔸 Integration testing

In this testing phase, different software modules are combined and tested as a group to make sure that integrated system is ready for system testing. Integrating testing checks the data flow from one module to other modules.

To perform integration testing of a GitHub Action we need a tool that:

Runs locally and on CI runner (including GitHub runner).
Runs the whole action or its part.
Isolates running code and give testing code an access to action's inputs, outputs and environment.
Allows stubbing external services used by an action, such as GitHub API.

Let's list what exactly we expect from such tool:

Parsing action config (action.yml file)
Setting up action inputs and saved state.
Setting up environment variables: custom ones and service GitHub variables.
Setting up GITHUB_EVENT_PATH variable and faking JSON file with an event payload.
Faking command files and setting up correspondent env variables (GITHUB_ENV, GITHUB_PATH).
Faking temp and workspace directories (and corresponding RUNNER_TEMP and GITHUB_WORKSPACE variables)
Intercepting and isolating stdout and stderr output. It's important, because being run on GitHub runner our tests can interfere with actual commands of test workflow.
Parsing intercepted output and faked command files to extract commands issued by tested code.

I haven't found any handy solution that meet these requirements and it made me write my own TypeScript package for testing JavaScript and Docker actions called github-action-ts-run-api. It has well typed JavaScript API with reasonable defaults, can be used with any JavaScript test framework or alone and covers all listed requirements.

In the following parts of the post I'm going to tell about the testing techniques that become
possible with this package. For more code examples take a look at the package documentation.

🔸 System testing

System testing is performed on a complete, integrated system. It allows checking system’s compliance as per the requirements. It tests the overall interaction of components. It involves load, performance, reliability and security testing.

It can be debatable what to consider as system testing in case of GitHub Action.

Option 1

Testing the whole action behavior using the same tool as we use for integration testing, but exclude external services stubs if it possible.

Option 2

Testing action behavour in the workflow. The only existing solution for doing that locally is a great tool called Act.

DRY: reusing code in GitHub Actions

Cardinal — Fri, 04 Feb 2022 14:47:46 +0000

In this post I want to make a quick overview of the approaches of reusing steps of your workflow to avoid duplication of the same steps across different workflows or jobs.

🔸 Reusing workflows

The obvious option is using the "Reusable workflows" feature that allows you to call extract some steps into a separate "reusable" workflow and call this workflow as a job in other workflows.

🥡 Takeaways:

Reusable workflows can't call other reusable workflows.
The strategy property is not supported in any job that calls a reusable workflow.
Env variables and secrets are not inherited.
It's not convenient if you need to extract and reuse several steps inside one job.
Since it runs as a separate job, you have to use build artifacts to share files between reusable workflow and your main workflow.
You can call reusable workflow in synchronous or asynchronous manner (managing it by jobs ordering using needs keys).
A reusable workflow can define outputs that extract outputs/outcomes from performed steps. They can easily used to pass data to the "main" workflow.

🔸 Dispatched workflows

Another possibility that GitHub gives us is workflow_dispatch event that can trigger a workflow run. Simply put, you can trigger a workflow manually or through GitHub API and provide its inputs.

There are actions available on the Marketplace which allow you to trigger a "dispatched" workflow as a step of "main" workflow.

Some of them also allow doing it in a synchronous manner (wait until dispatched workflow is finished). It is worth to say that this feature is implemented by polling statuses of repo workflows which is not very reliable, especially in a concurrent environment. Also, it is bounded by GitHub API usage limits and therefore has a delay in finding out a status of dispatched workflow.

🥡 Takeaways

You can have multiple nested calls, triggering a workflow from another triggered workflow. If done careless, can lead to an infinite loop.
You need a special token with "workflows" permission; your usual secrets.GITHUB_TOKEN doesn't allow you to dispatch a workflow.
You can call run multiple dispatched workflows inside one job.
There is no easy way to get some data back from dispatched workflows to the main one.
Works better in "fire and forget" scenario. Waiting for a finish of dispatched workflow has some limitations.
You can observe dispatched workflows runs and cancel them manually.

🔸 Composite Actions

In this approach we extract steps to a distinct composite action, that can be located in the same or separate repository.

From your "main" workflow it looks as a usual action (a single step), but internally it comprises of multiple steps each of which can call own actions.

🥡 Takeaways:

Supports nesting: each step of a composite action can use another composite action.
Bad visualisation of internal steps run: in the "main" workflow it's displayed as a usual step run. In raw logs you can find details of internal steps execution, but it doesn't look very friendly.
Shares environment variables with a parent job, but doesn't share secrets, which should be passed explicitly via inputs.
Supports inputs and outputs. Outputs are prepared from outputs/outcomes of internal steps and can be easily used to pass data from composite action to the "main" workflow.
A composite action runs inside the job of the "main" workflow. Since they share a common file system, there is no need to use build artifacts to transfer files from the composite action to the "main" workflow.
You can't use continue-on-error option inside a composite action.

👏 Thank you for reading

Any comments, critics and sharing your own experience would be appreciated!

If you are interested in developing own Actions, I also recommend you reading "GitHub Actions Testing" series.

Releasing WebExtension using GitHub Actions

Cardinal — Wed, 08 Dec 2021 19:11:35 +0000

My Workflow

I'm the author of an open-source browser extension and I want to share the workflow that I have created for:

releasing the extension to Chrome Web Store
releasing the extension to Firefox Add-ons
packaging the extension into various artifacts for offline distribution (crx and xpi files)

I wasn't able to find any existing solutions that can meet these requirements and I have prepared a set of actions and a complete solution in form of several workflows to set up the whole process.

They are based on my practical experience of dealing with all difficulties of interacting with the extension stores and their undocumented behaviour.

Submission Category:

DIY Deployments

Yaml File or Link to Code

cardinalby / memrise-audio-uploader

Memrise Audio Uploader

Chrome extension which allows you upload sounds of words and phrases prononsiation from http://soundoftext.com to http://memrise.com course

Install from the stores

Chrome Web Store

Firefox Add-ons

To download the offline versions please go to the Releases page.

Screenshots

View on GitHub

the main workflow

Additional Resources / Info

I have posted a detailed manual here on dev.to that will help everybody to understand my workflow and build the similar one for their projects.
Another post related to the tricky way of using "delayed" workflows in GitHub Actions that was used in my workflow.

Publish on Firefox Add-ons

Cardinal — Wed, 08 Dec 2021 18:11:37 +0000

In this part we are going to create the workflow that will be responsible for publishing the extension
on Firefox Add-ons marketplace.

🧱 Prepare

First, you need to find out your extension UUID. You can find it on your extension's page at Add-on Developer Hub in the "Technical Details" section.

Next, follow the official documentation and obtain jwtIssuer and jwtSecret values required for accessing the API.

🔒 Add these values to secrets:

FF_EXTENSION_ID - UUID of your extension
FF_JWT_ISSUER
FF_JWT_SECRET

publish-on-firefox-add-ons workflow

The workflow will have the only trigger: workflow_dispatch event. It can be dispatched:

Manually specifying any commit or tag as workflow ref.
By publish-release-on-tag workflow after it has prepared a release with zip asset.

We will utilize already created get-zip-asset composite action to obtain packed zip that is needed for deploying the extension.

.github/workflows/publish-on-firefox-addons.yml :

name: publish-on-firefox-add-ons
on:
  workflow_dispatch:
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - uses: cardinalby/export-env-action@v1
        with:
          envFile: './.github/workflows/constants.env'
          expand: true

      - name: Obtain packed zip
        uses: ./.github/workflows/actions/get-zip-asset
        with:
          githubToken: ${{ secrets.GITHUB_TOKEN }}

      - name: Deploy to Firefox Addons
        id: addonsDeploy
        uses: cardinalby/webext-buildtools-firefox-addons-action@v1
        continue-on-error: true
        with:
          zipFilePath: ${{ env.ZIP_FILE_PATH }}
          extensionId: ${{ secrets.FF_EXTENSION_ID }}
          jwtIssuer: ${{ secrets.FF_JWT_ISSUER }}
          jwtSecret: ${{ secrets.FF_JWT_SECRET }}

      - name: Abort on upload error
        if: |
          steps.addonsDeploy.outcome == 'failure' &&
          steps.addonsDeploy.outputs.sameVersionAlreadyUploadedError != 'true'
        run: exit 1

As usual, at the beginning of each workflow we check out the repo and export env variables from constants.env file.
After calling get-zip-asset composite action we expect to have zip file with packed and built extension at env.ZIP_FILE_PATH path.
We pass its path along with other required inputs to webext-buildtools-firefox-addons-action action to publish the extension. We use continue-on-error: true flag to prevent the step from failing immediately in case of error and validate the result at the following step according to our preferences.
Examining addonsDeploy step outputs we can find out the reason of its failure. If it failed because the version we try to publish is already published, we don't consider it as error.

Timeout notes

Also, the publishing action can sometimes fail with timeoutError == 'true' output. It means, the extension was uploaded but the waiting for its processing by Addons server was timed out. I didn't include a handling of this error to the workflow, but you can:

Specify longer timeout with timeoutMs input of webext-buildtools-firefox-addons-action action. Default timeout is 600000 ms (10 min).
Do not fail the job in the last step if steps.addonsDeploy.outputs.timeoutError == 'true'.
Just rerun the workflow after a while in the case of timeout. If the extension has been processed after the first run, the workflow will pass (with steps.addonsDeploy.outputs.sameVersionAlreadyUploadedError != 'true').

Build release assets

Cardinal — Wed, 08 Dec 2021 14:04:03 +0000

build-assets-on-release workflow

Let's create the first workflow that utilizes build-test-pack action and builds release assets for offline distribution once a release has been published.

.github/workflows/build-assets-on-release.yml:

name: Build release assets

on:
  release:
    # Creating draft releases will not trigger it
    types: [published]
jobs:
  # We will add 3 jobs here...

The workflow will have 3 jobs:

ensure-zip: Ensuring we have zip release asset.
build-signed-crx-asset: Building crx asset.
build-signed-xpi-asset: Building xpi asset.

ensure-zip job

The first job will find zip asset in the release or build it if not found:

  ensure-zip:
    runs-on: ubuntu-latest
    outputs:
      zipAssetId: | 
        ${{ steps.getZipAssetId.outputs.result || 
            steps.uploadZipAsset.outputs.id }}
    steps:
      - uses: actions/checkout@v2

      - uses: cardinalby/export-env-action@v1
        with:
          envFile: './.github/workflows/constants.env'
          expand: true

      - name: Find out zip asset id from the release
        id: getZipAssetId
        uses: cardinalby/js-eval-action@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          ASSETS_URL: ${{ github.event.release.assets_url }}
          ASSET_NAME: ${{ env.ZIP_FILE_NAME }}
        with:
          expression: |
            (await octokit.request("GET " + env.ASSETS_URL)).data
              .find(asset => asset.name == env.ASSET_NAME)?.id || ''

      - name: Build, test and pack
        if: '!steps.getZipAssetId.outputs.result'
        id: buildPack
        uses: ./.github/workflows/actions/build-test-pack

      - name: Upload "extension.zip" asset to the release
        id: uploadZipAsset
        if: '!steps.getZipAssetId.outputs.result'
        uses: actions/upload-release-asset@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          upload_url: ${{ github.event.release.upload_url }}
          asset_path: ${{ env.ZIP_FILE_PATH }}
          asset_name: ${{ env.ZIP_FILE_NAME }}
          asset_content_type: application/zip

At the beginning of each workflow we check out the repo and export env variables from constants.env file.
getZipAssetId step uses github.event.release.assets_url to get the release assets list and find zip asset id. It may not exist if the workflow was triggered by a release created by a user directly. Used js-eval-action is an action for executing general-purpose JavaScript code.
If it hasn't been found, we call build-test-pack composite action to build zip asset from the scratch, save it to env.ZIP_FILE_PATH and then attach to the release.
zipAssetId job output will have a value from either getZipAssetId step (if zip asset was found in the release) or or uploadZipAsset step if it has been built and uploaded in the job.

The following 2 jobs will run in parallel using zip asset provided by the job we just created.
This order is defined using needs: ensure-zip key in the jobs.

build-signed-crx-asset job

crx file is a distribution package of your extension. When you install an extension from the store, crx file gets transmitted and installed to your browser. But it can be also installed in offline mode manually or via automation tools. Read more about alternative extension distribution options here.

Actually, crx file is a kind of zip file that contains the extension dir. But it also contains some additional metadata required by Chrome browser. Namely, it's signed using a developer's private key.

🧱 Prepare

For this step you need to have a pem private key that is used for offline signing. You can use openssl to generate it.

🔒 Add the key to the repo secrets:

CHROME_CRX_PRIVATE_KEY - a string containing the private key.

Add the job to the workflow:

  build-signed-crx-asset:
    needs: ensure-zip
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: cardinalby/export-env-action@v1
        with:
          envFile: './.github/workflows/constants.env'
          expand: true

      - name: Download zip release asset
        uses: cardinalby/download-release-asset-action@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          assetId: ${{ needs.ensure-zip.outputs.zipAssetId }}
          targetPath: ${{ env.ZIP_FILE_PATH }}

      - name: Build offline crx
        id: buildOfflineCrx
        uses: cardinalby/webext-buildtools-chrome-crx-action@v2
        with:
          zipFilePath: ${{ env.ZIP_FILE_PATH }}
          crxFilePath: ${{ env.OFFLINE_CRX_FILE_PATH }}
          privateKey: ${{ secrets.CHROME_CRX_PRIVATE_KEY }}

      - name: Upload offline crx release asset
        uses: actions/upload-release-asset@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          upload_url: ${{ github.event.release.upload_url }}
          asset_path: ${{ env.OFFLINE_CRX_FILE_PATH }}
          asset_name: ${{ env.OFFLINE_CRX_FILE_NAME }}
          asset_content_type: application/x-chrome-extension

The job uses zipAssetId output from ensure-zip job to download the asset.
webext-buildtools-chrome-crx-action action uses the private key from secrets to build and sign crx file for offline distribution. This action doesn't interact with Chrome Web Store.

build-signed-xpi-asset job

Firefox also has options for self-distribution of add-ons. Firefox's own format of extension package is called xpi and it also has to be signed. Unlike Chrome's, this signing procedure is online: we have to ask Firefox server to do it for us.

🧱 Prepare

It's recommended to create a separate entity for the offline distributed extension on Add-on Developer Hub and use its id for signing xpi files. Otherwise, a new entity will be added for every build. You can find extension UUID in the "Technical Details" section of the created entity.

Next, follow the official documentation and obtain jwtIssuer and jwtSecret values required for accessing the API.

🔒 Add these values to repo secrets:

FF_OFFLINE_EXT_ID - UUID of the add-on entity created for offline distribution.
FF_JWT_ISSUER
FF_JWT_SECRET

Finally, add the following job to the workflow:

  build-signed-xpi-asset:
    needs: ensure-zip
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: cardinalby/export-env-action@v1
        with:
          envFile: './.github/workflows/constants.env'
          expand: true

      - name: Download zip release asset
        uses: cardinalby/download-release-asset-action@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          assetId: ${{ needs.ensure-zip.outputs.zipAssetId }}
          targetPath: ${{ env.ZIP_FILE_PATH }}

      - name: Sign Firefox xpi for offline distribution
        id: ffSignXpi
        continue-on-error: true
        uses: cardinalby/webext-buildtools-firefox-sign-xpi-action@v1
        with:
          timeoutMs: 1200000
          extensionId: ${{ secrets.FF_OFFLINE_EXT_ID }}
          zipFilePath: ${{ env.ZIP_FILE_PATH }}
          xpiFilePath: ${{ env.XPI_FILE_PATH }}
          jwtIssuer: ${{ secrets.FF_JWT_ISSUER }}
          jwtSecret: ${{ secrets.FF_JWT_SECRET }}

      - name: Abort on sign error
        if: |
          steps.ffSignXpi.outcome == 'failure' &&
          steps.ffSignXpi.outputs.sameVersionAlreadyUploadedError != 'true'
        run: exit 1

      - name: Upload offline xpi release asset
        if: steps.ffSignXpi.outcome == 'success'
        uses: actions/upload-release-asset@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          upload_url: ${{ github.event.release.upload_url }}
          asset_path: ${{ env.XPI_FILE_PATH }}
          asset_name: ${{ env.XPI_FILE_NAME }}
          asset_content_type: application/x-xpinstall

webext-buildtools-firefox-sign-xpi-action action uses the keys from secrets to build and sign xpi file for offline distribution. Practice shows that this step can take quite a long time to complete. timeoutMs input allows you to configure the timeout.
continue-on-error: true key of ffSignXpi step is used to prevent the step to fail immediately in case of error and examine an error at the next step.
At the next step we examine an error (if any) and fail the job, except the case where the error happened because this version was already signed. It's a peculiarity of Firefox Add-ons signing process: it doesn't allow to sign the same version twice. So we just suppress the error and don't fail the entire workflow and just skip the following "upload" step instead.