Forem: Hookdeck

How to Make Your AI Agent Get Webhooks Right (A Guide to Webhook Skills)

Phil Leggetter — Wed, 18 Feb 2026 15:27:46 +0000

When I ask my AI coding agent to set up webhooks from a new API (Stripe, Shopify, GitHub, whatever), the code it generates often looks fine until I run it. Then I hit signature verification failures, wrong raw body handling, or idempotency bugs that process the same event twice. Sound familiar?

I created webhook-skills to fix that. They're agent skills for webhooks: an open-source collection of provider- and framework-specific instructions and runnable examples that AI coding agents can load so they implement webhooks correctly the first time. In this guide I'll walk through how to use them, what's in them, and how you can contribute or request skills that are missing.

Why Agents Get Webhooks Wrong

At Hookdeck we process billions of webhooks a week, and we've seen every failure mode: signature mismatches from body parsing middleware, framework-specific gotchas that only show up in production, and so on. AI agents struggle with the same things. Their training data goes stale quickly. API versions change, security practices evolve, and the details that matter (raw body handling, middleware order, encoding) aren't in the model.

Research from PostHog on LLM code generation backs this up: the most reliable way to get correct code isn't the model's general knowledge; it's giving it specific, known-working examples to reference. Webhook skills are built to fill that gap.

How Webhook Skills Fix It

Webhook-skills is built on the Agent Skills specification, an open standard for packaging knowledge that agents can consume. In practice that means:

Runnable examples: complete, minimal apps the agent can reference and adapt, not just snippets.
Provider-specific guidance: Stripe's raw body requirement, Shopify's HMAC encoding, and other gotchas we see trip up developers.
Framework-aware implementations: how Next.js, Express, and FastAPI handle request bodies, middleware order, and async patterns.
Staged workflows: verify signature first, parse payload second, handle idempotently third.

When I ask my agent to "add Stripe webhooks to my Next.js app," it doesn't hallucinate a generic handler. It has the exact patterns for App Router body handling, preserving the raw body before verification, and pulling the webhook secret from env with the right naming. That's the difference.

How to Use Them

Install the skills

I use npx skills to add webhook-skills to my project. First I list what's available, then I install the skills I need:

# List available skills
npx skills add hookdeck/webhook-skills --list

# Install best-practice patterns (verify → parse → handle idempotently)
npx skills add hookdeck/webhook-skills --skill webhook-handler-patterns

# Install specific provider skills
npx skills add hookdeck/webhook-skills --skill stripe-webhooks
npx skills add hookdeck/webhook-skills --skill shopify-webhooks

I usually install webhook-handler-patterns plus the provider skill for whichever API I'm integrating. That gives my agent both the general flow and the provider-specific details.

How to prompt

Once the skills are installed, I prompt naturally. For example:

"Add Shopify webhook handling to my Express app."
"Set up Stripe webhooks in my Next.js app."
"Implement GitHub webhook verification in my FastAPI service."

The agent has the patterns; I don't need to spell out raw body or signature verification. It already knows.

Local testing (optional)

When I'm testing webhooks locally, I use the Hookdeck CLI. It gives me a public URL that tunnels to my local server and a UI to inspect and replay requests:

npm i -g hookdeck-cli
# or: brew install hookdeck/hookdeck/hookdeck

hookdeck listen 3000 --path /webhooks/stripe

No account required to get started. The skills work with or without Hookdeck; they're just complementary when you want to receive real webhooks on localhost.

What's Available (and What's Missing)

Right now webhook-skills covers the providers and frameworks we see most often:

Providers: Stripe, Shopify, GitHub, OpenAI, Resend, Paddle, ElevenLabs, Chargebee, and more.

Frameworks: Next.js, Express, and FastAPI.

Each provider skill includes signature verification, event handling guidance, common failure modes, and testing tips for local dev. Coverage isn't complete yet. If you need a provider we don't have or you want to contribute, see the next section.

How to Contribute and Ask for Skills

Request a skill: If you're integrating a provider we don't support yet, open an issue on GitHub with a title like "Skill request: [Provider] webhooks". Describe the provider and your framework if relevant. I use these to prioritize what to add next.

Contribute a skill: If you've built a webhook integration you're proud of, PRs for new providers or frameworks are welcome. The repo is hookdeck/webhook-skills. Check the existing skills for structure and open a PR.

API and platform maintainers: If you maintain a webhook-producing API and want AI coding agents to implement your webhooks correctly out of the box, I'd love a skill from you. Open an issue or PR and we can align on format and content.

Give It a Shot

If you've ever lost an afternoon to webhook signature verification or body parsing, try webhook-skills the next time you're wiring up an integration. Install the skills, prompt your agent, and you might find it finally does what you meant.

Links:

webhook-skills on GitHub
Hookdeck CLI (optional, for local webhook testing)

Have feedback or want to request or contribute a skill? Open an issue.

Introducing The Event Destinations Initiative

Phil Leggetter — Thu, 27 Feb 2025 13:57:21 +0000

The Event Destinations Initiative is a community effort aimed at creating a model for event interoperability between event producers and their consumers. This initiative focuses on providing a set of guidelines for managing event destinations, ensuring seamless integration, improved reliability, greater event delivery choice, and a much-improved developer experience.

What are Event Destinations?

Event Destinations are the endpoints where API platform events are delivered and consumed. Event destinations can include webhook endpoints, message queues, APIs, event gateways, and other services that handle event data. Webhooks are the most ubiquitous form of event destination, but the Event Destinations Initiative aims to establish a set of guidelines for API platforms to provide additional event delivery options offering more choice, improved reliability at scale, and an overall better developer experience.

What is the Event Destinations Initiative?

The Event Destinations Initiative is a collaborative project that aims to create a set of guidelines for defining, managing, and utilizing event destinations across platforms and services.

Key Features

Standardization: Establishing a common set of guidelines and best practices for event destinations.
Interoperability: Ensuring compatibility across different platforms and services.
Scalability: Providing solutions that can scale with the growing demands of modern applications.
Reliability: Enhancing the reliability and fault tolerance of event-driven architectures.

Benefits

Simplified Integration: Easier integration with commonly used services and platforms.
Improved Efficiency: Streamlined event publishing, routing, and ingestion.
Enhanced Reliability: More robust and fault-tolerant event handling for both the publisher and consumer.
Future-Proofing: A forward-looking approach that anticipates future needs and challenges.

What Supporters are Saying

In his book Flow Architectures, James Urquhart envisions a future where APIs are inherently asynchronous, with synchronous APIs being reserved only for scenarios where they are truly necessary.

The Event Destinations Initiative represents a significant step toward realizing that vision. Enabling API producers to adopt the most efficient and performant methods for sending and receiving asynchronous, non-blocking data is a development I wholeheartedly support—particularly when it is driven by a collaborative and open effort.

This direction strongly aligns with the vision we champion at AsyncAPI, and the synergy between these efforts is both promising and exciting. I am eager to see the Event Destinations Initiative succeed and contribute to the evolution and blending of API and EDA ecosystems.

-- Fran Méndez, Founder @ AsyncAPI Initiative

I've been working with (and often against) webhooks for years, both as a consumer and an implementer, always wishing that we as an industry could come up with something better. In my opinion, Event Destinations are exactly that.

-- Paul Asjes, Developer Experience Engineer @ ElevenLabs

Get Involved

We invite developers, architects, and organizations to join us in this initiative. By collaborating, we can build a more efficient and reliable event-driven ecosystem.

Visit eventdestinations.org to learn more and get involved.

Free ngrok alternative for async web dev - the Hookdeck CLI

Phil Leggetter — Mon, 29 Jul 2024 18:31:55 +0000

ngrok is great. It's been the go-to for many a web developer who wants to be able to expose a locally running web application to the public Internet for many years. And it's still a great solution. However, ngrok has changed in ways that have resulted in a much poorer developer experience and there are now better tools for some of the jobs that ngrok was originally the go-to for.

What's changed with ngrok?

ngrok used to be free and only required an account for "premium features". Now, you need an ngrok account no matter your use case.
There's a 1GB limit a month on data transfer (in all fairness, this should be more than enough for development)
The free ephemeral/random domains (the domain name changes every time you restart ngrok) have always been a bit of a pain. The first paid plan used to come with 3 subdomains but now only comes with one, making exposing multiple services at the same time more costly.

More broadly, Their product focus has shifted reasonably dramatically from supporting use cases such as:

Temporarily sharing a website that is only running on your development machine

Demoing an app at a hackathon without deploying

Developing any services that consume webhooks (HTTP callbacks) by allowing you to replay those requests

Debugging and understanding any web service by inspecting the HTTP traffic

Running networked services on machines that are firewalled off from the internet

To:

All-in-one API gateway, Kubernetes Ingress, DDoS protection, firewall, and global load balancing as a service.

So, ngrok has shifted focus away from supporting development use cases. Therefore, it's likely time to try an alternative tool that has been built to specifically support your use case.

The rest of this post will show how the Hookdeck CLI can be used to receive asynchronous events (a.k.a. webhooks) from third-party services such as Stripe, Shopify, or Twilio. If you're looking for a tool that solves the other use cases, take a look at the awesome tunneling repo.

What is synchronous web development?

As per the title of this post, the Hookdeck CLI is built to support asynchronous web development. By this, I mean requests, such as webhooks from an API service, are received and proxied onto the application running locally. However, the response from the local service is not passed back to the API service that made the request.

The Hookdeck CLI and the Hookdeck Event Gateway are built to support asynchronous messaging with event-driven applications and not the request/response paradigm. Therefore, you can't use the Hookdeck CLI to expose and serve a locally running website or request/response API. If you want to send the result of some work to another service, you should make a separate request to that service.

Using the Hookdeck CLI as a free alternative to ngrok for webhooks

The next section walks through installing the Hookdeck CLI, creating and running a local web server (which you can skip if you already have this running), creating a localtunnel using the CLI, sending test webhook events, and replaying webhooks using the Hookdeck Console.

This guide will use the Node.js runtime. If you prefer to use Python, you can run through a using the Hookdeck CLI with a Python server to receive webhooks guide.

Install the Hookdeck CLI

Via NPM:

npm i hookdeck-cli -g

On Mac:

brew install hookdeck/hookdeck/hookdeck

On Windows:

scoop bucket add hookdeck https://github.com/hookdeck/scoop-hookdeck-cli.git
scoop install hookdeck

There are also instructions for installing on Linux.

Create and run a localhost Node.js web server

Open your terminal and navigate to a directory for your project.

Install Express to use as the web server:

npm i --save express

Create a server.js file in the project root:

const express = require('express')
const app = express()

app.use(express.json());

const port = 3030;

app.post("/", (req, res) => {
    console.log({ webhook_received: new Date().toISOString(), body: req.body });
    res.json({ status: "ACCEPTED" });
});

app.listen(port, () => {
    console.log(`🪝 Server running at http://localhost:${port}`);
});

Run the server:

node server.js

Use the Hookdeck CLI to create a localtunnel

Run the following command in a new terminal window to create a localtunnel:

hookdeck listen 3030 my-webhook

The output will be similar to the following:

Dashboard
👤 Console URL: https://api.hookdeck.com/signin/guest?token={token}
Sign up in the Console to make your webhook URL permanent.

Sources
🔌 my-webhook URL: https://hkdk.events/wggd60fl5cxw20

Connections
my-webhook -> my-webhook-to-cli forwarding to /

> Ready! (^C to quit)

Test receiving a webhook with a cURL command

Run the following cURL command to act as an inbound webhook, replacing the URL with the Event URL from the Hookdeck CLI output:

curl -X POST https://hkdk.events/cnzpo680rdhejk \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, World!"}'

The cURL command output will be similar to the following:

{"status":"SUCCESS","message":"Request handled by Hookdeck. Check your dashboard to inspect the request: https://dashboard.hookdeck.com/requests/req_[id]","request_id":"req_[id]"}%

You will see the terminal running the Hookdeck CLI log the inbound webhook:

2024-07-09 19:06:46 [200] POST http://localhost:3030/ | https://console.hookdeck.com/?event_id={id}

You will also see the Python server log the inbound webhook:

{
  webhook_received: '2024-07-25T10:08:22.945Z',
  body: { message: 'Hello, World!' }
}

Trigger a test webhook using the Hookdeck Console

Open the Console URL from your terminal in your browser.

Choose a Sample Webhook Provider from the list of Example Webhooks. For example, Stripe.

Select a Sample Webhook Type from the list on the right. For example, invoice.created.

Click Send.

The Hookdeck Console will show the test webhook has been triggered. You can also inspect the webhook payload and the localhost web server response.

You will see the terminal running the Hookdeck CLI log the inbound webhook:

2024-07-09 19:06:46 [200] POST http://localhost:3030/webhook | https://console.hookdeck.com/?event_id={id}

You will also see the Python server log the inbound webhook:

{
  "id": "evt_1MhUT7E0b6fckueStwy86nfu",
  "object": "event",
  "api_version": "2022-11-15",
  "created": 1677833999,
  "data": {
    ...
  },
  "type": "invoice.created"
}

Replay a webhook

To replay the webhook, go to the Hookdeck Console and click on the Resend to destination button.

Receive a webhook from an API platform

Copy the Event URL from the Hookdeck CLI output. You can also find the same URL in the Hookdeck Console.

Go to the API provider platform, such as Resend, Twilio, Shopify, or Stripe, and register the Hookdeck URL as the webhook URL with the provider.

Trigger a webhook from your selected API provider to see a log entry in the Hookdeck Console.

You will also see the webhook logged in the Hookdeck CLI:

2024-07-09 19:45:57 [200] POST http://localhost:3030/webhook | https://console.hookdeck.com/?event_id={id}

And by the Node.js server running in your local development environment:

{
  "to": "{to_number}",
  "from": "{from_number}",
  "channel": "sms",
  "message_uuid": "461c9502-3c2f-4af9-8862-c5a8eccf6cfe",
  "timestamp": "2024-07-09T18:45:56Z",
  "usage": {
    "price": "0.0057",
    "currency": "EUR"
  },
  "message_type": "text",
  "text": "Inbound SMS from the Vonage API",
  "context_status": "none",
  "origin": {
    "network_code": "23415"
  },
  "sms": {
    "num_messages": "1",
    "count_total": "1"
  }
}

And that's it!

You have successfully received a webhook on your local development environment using the Hookdeck CLI. You also inspected the webhook payload and server response, and replayed a webhook using the Hookdeck Console.

Learn more about Hookdeck

This guide demonstrated how to use the Hookdeck CLI as a replacement for ngrok for local asynchronous web development. The Hookdeck CLI is free to use, and you don't need a Hookdeck account.

If you're interested in finding out more about Hookdeck including features such as transformations, and filtering, benefitting from functionality like configurable retries, and generally using Hookdeck as your reliable inbound webhook infrastructure, head to hookdeck.com

Add Webhook Verification, Queueing, Filtering, and Retry Logic to Any Vercel Deployed Endpoint

Phil Leggetter — Mon, 06 May 2024 18:38:18 +0000

Today we're excited to announce the Hookdeck Vercel Middleware. Vercel middleware that convert any Vercel application endpoint into an asynchronous endpoint with authentication, filters, queueing, throttling, and retry logic.

Every month, we deliver hundreds of thousands of asynchronous events to Vercel application endpoints, so we want to make debugging and building with webhooks and other asynchronous messaging use cases on Vercel even easier.

The Hookdeck Vercel Middleware requires minimal effort (install + simple configuration) and a DX that fits into a code-first workflow. So, within a few minutes, you've converted a Next.js route, for example, into a reliable and feature-rich asynchronous endpoint.

Along with the simple setup, I believe this solves a real problem with numerous use cases. The main one is handling webhooks (Stripe, Shopify, Twilio, etc.) at a scale where you really don't want to miss a webhook.

Here's a 5-minute video of our CEO and co-founder, Alex, installing, configuring, and deploying a Next.js app:

You can see the quickstart instructions (along with the middleware source) in the Hookdeck Vercel Middleware GitHub repo

There's also example Next.js app with a deployed demo.

Let us know what you think.

Shopify Webhooks Tutorial: How to Use the Shopify API

Fikayo Adepoju — Thu, 16 Dec 2021 19:46:54 +0000

Introduction

Shopify webhooks help build integrations with external applications that need to respond in real-time to events taking place in a Shopify store. You can create and manage Shopify webhooks using the Shopify admin dashboard. Although easy to use, the admin interface does not allow developers to perform automated tasks on their webhooks. Instead, users have to log into the admin interface and manually perform webhook-related tasks.

As a solution to this drawback, Shopify opened up its webhook API. This allows programmatic interaction with Shopify webhooks, thereby expanding the scope of what is possible with the technology.

In this article, I will walk you through how to set up authentication with the Shopify API, and then use the Shopify webhooks API to create and view your webhooks. You will also use a locally running API to receive and log your webhooks.

Let's get right into it!

Prerequisites

To follow along with this tutorial, you will need a couple of things:

A Shopify store
The ability to create Shopify private apps (you may not have this permission if you're not the direct owner of the store you're working with)
Node.js installed on your system (if you already have a local API that you prefer to use, that is fine as long as you make sure it exposes a POST endpoint and listens on a known port)
Hookdeck CLI installed (you can find details on installing the tool here)

Creating a Shopify store app

The first step to accessing and using the Shopify webhooks API is to create a Shopify store app. There are 3 types of Shopify apps: public, custom, and private. To learn more about the different types of Shopify apps and how to create them, read the documentation pages here.

To access the Shopify API, we are going to create a private app. Private apps are built to work exclusively for your Shopify store (unlike public apps that can be used by any Shopify store and are listed on the Shopify App store). Private apps also allow you to access your store's data directly using the Shopify API.

To create a Shopify app, click Apps on the side menu of the admin interface. Just below the main section on the Apps page, click the Manage Private Apps link and then click the Create private app button.

On the app creation page, enter the name for the application you are about to create (in this example, Purchase Log API) and enter your email. Next, in the Admin API section, expand the Show Inactive Admin API and give Read and write access to the following resources...

Order editing
Orders

...as shown below:

Once the permissions are set, scroll down and select the latest API version (at the time of this writing, the latest version is 2021-10).

Click Save to complete this process. You will be prompted with a pre-save message informing you that an API key will be generated to provide access to the store's data.

Click Create app on this prompt and you will be taken to the app page. On this page, you will find the following credentials in the Admin API section:

API key (this is your username when using Basic authentication)
Password (this is your API access token)
Example URL (a sample URL format for performing basic authentication using a username and password)
Shared Secret (used for verifying webhooks, more on this later)

With these credentials, you have all you need to authenticate with the Shopify API.

Authenticating with the Shopify API

Now that you have your authentication credentials, let's discuss how you will go about making authenticated calls to the Shopify API. It is important to note that the different types of Shopify apps do not use the same authentication strategy.

While public and custom apps use OAuth and session tokens, private apps use basic authentication. In this section, I will only be discussing authentication for private apps.

There are 3 ways you can authenticate with the Shopify API using credentials from your private app.

1) Username and password combo

If your client supports basic authentication, you can include authentication credentials in your URL. This is done by prepending username:password@ to the hostname in the URL as shown below:

https://{username}:{password}@{shop}.myshopify.com/admin/api/{version}/webhooks.json

username: Your API key.
password: Your private app password.
shop: Your Shopify store subdomain string, e.g. hookdeck-store.
version: The version of the API you're using, e.g. 2021-10.

Any request made to the Shopify API using this URL is automatically authenticated.

2) Using an authorization token

Some clients do not support basic authentication as used in the above method. You can go around this issue by using the Basic {token} value in the Authorization header. The token value is derived by joining your store app's API key and password with a colon (:) and encoding the resulting string in base64. The Authorization header can then be sent, as shown below:

Authorization: Basic NDQ3OGViN2FjMTM4YTEzNjg1MmJhYmQ4NjE5NTZjMTk6M2U1YTZlZGVjNzFlYWIwMzk0MjJjNjQ0NGQwMjY1OWQ=

3) Using the X-Shopify-Access-Token header

The last strategy for authenticating private apps is to send the X-Shopify-Access-Token header, along with your request, to the Shopify API. The value of this header is your private app's password, which is your access token. The header is sent as follows:

X-Shopify-Access-Token: {my_app_password}

This is the strategy we will be using for all our API calls in this tutorial.

Now that you know how to authenticate with the Shopify API, let's start creating and receiving Shopify webhooks.

Setting up and receiving Shopify webhooks using the Shopify API

As described earlier, we will create a webhook using the Shopify API. Then, we'll spin up a local API to receive and log part of the webhook payload.

Before we begin, let's walk you through the steps necessary to achieve this webhook logging setup.

Clone the demo API and run it locally
Use the Hookdeck CLI to generate a webhook URL that points to the API endpoint
Create a webhook using the API
Test your webhook by triggering the subscription event
Inspect your webhook and webhook logs
Verify the webhook payload

Now that you're caught up, let's begin.

Clone the demo API

Clone the application repository for the API by running the following command:

git clone --single-branch --branch shopify-webhooks https://github.com/hookdeck/nodejs-webhook-server-example.git

Navigate to the root of the project and install the required dependencies by running the following commands:

cd nodejs-webhook-server-example
npm install

When the installation completes, you can then run the Node.js server with the following command:

npm start

This will boot up the API application and print a message to the screen indicating that the API is now running and listening for connections on port 1337.

We are using two endpoints in this project.

/log-shopify-webhook: This is the endpoint that will be receiving the Shopify webhook and logging it into an in-memory database. It logs a simple object containing a subset of the information from the webhook payload.
/fetch-webhooks-logs: This endpoint can be called to retrieve a collection of the logged webhook data.

Generate the webhook URL using the Hookdeck CLI

The next step is to use the CLI to generate a webhook URL that points to the running API application. To do this, run the following command (note: replace the port 1337 value with the port number on which your API is running if you're not using the sample project):

hookdeck listen 1337

This command starts an interactive session where the CLI collects information about the endpoint you're about to create. Below are the questions and the answers you should supply to each question. Ensure to hit the Enter key after each answer.

What source should you select? Ans: select **Create new source* (this prompt does not show up if you have not created any previous connections)*
What should your new source label be? Ans: type the text Shopify
What path should the webhooks be forwarded to (i.e.: /webhooks)? Ans: type /log-shopify-webhook (if you're using your own custom server, replace this value with your endpoint)
What's the connection label (i.e.: My API)? Ans: type Purchase Log API

With this information, the CLI will begin the process of generating the URL and once it's done, you will see the URL printed to the screen and the CLI indicating that it is ready to receive requests.

Note: If you're running the CLI in guest mode, you will need to use the guest Login URL link in the console to access the dashboard. Copy and paste this into your browser to begin a guest login session.

Create a Shopify webhook using the API

Now that you have your webhook URL, let's create the webhook. Shopify's webhook API can be accessed at https://{shop}.myshopify.com/admin/{version}/webhooks.json (remember to replace the values in {} with yours).

To create a webhook, you need to send a POST request to this endpoint containing data. You can use any HTTP client of your choice (Postman, Curl, etc.). In this tutorial, I am using ReqBin because it is accessible and easy to use.

For the webhook you are creating in this tutorial, you must subscribe to the orders/create. This event is fired when a new order is created in your store.

The data to send along with your webhook creation request should be structured as follows:

{
    "webhook" : {
            "topic" : "orders/create",
            "address": "{webhook_url}",
            "format": "json"
    }
}

As seen in the above snippet, the format to receive the webhook is set as json, so remember to replace {webhook_url} with the webhook URL generated on the Hookdeck CLI session.

Open https://reqbin.com and paste the API URL in the address field, then set the request method to POST. For authentication, add the X-Shopify-Access-Token field in the Headers section as shown below (replace MY_ACCESS_TOKEN with your Shopify app password):

The header will now be reflected in the details under the Raw section. Lastly, click the Content section and paste in your request object. See below a screenshot of the request setup:

Now hit the Send button to create your new webhook. You should get back a successful response, like you can see below:

This indicates that you have successfully created a webhook for the orders/create topic on your Shopify store.

Other actions that can be performed on your Shopify webhooks using the API include updating a webhook, fetching a single webhook, and deleting an existing webhook. For more information on how to perform these operations, visit the Shopify webhooks API documentation.

Test your webhook

With a locally running log API set up to receive webhooks, and a webhook subscription created via the API, the time has come to test your setup.

To receive a webhook for the subscribed topic, you need to trigger the topic's event by creating a new order. You can create a new order easily on your Shopify admin interface. Below, I am placing an order for a Ben Simmons 76er's jersey:

Once the order is created, Shopify will trigger a webhook for the orders/create event.

This will be received in your Hookdeck CLI session as shown below:

Inspect your webhook and webhook logs

The Hookdeck CLI helps us easily inspect the details contained in a Shopify webhook by providing a URL to an event page that breaks down the webhook data. This URL is the last item on the webhook entry, as shown in the CLI output above. Copy this URL and load it in your browser, and you will see an event page where you can view details like the webhook headers in the Headers section below:

You also have a browsable representation of the webhook payload in the Body section:

Finally, let's check to confirm that the locally running API is indeed logging the webhook payload. Still on your browser, navigate to the /fetch-webhooks-logs endpoint of the locally running API, and you will see a display similar to the one below (ignore the first object):

As seen, you have been able to log the webhook ID, the total number of items in your order, and the time the webhook was delivered.

Verify your Shopify webhooks

Shopify sends you an X-Shopify-Hmac-Sha256 header that can be used to verify that the payload contained in the webhook is from Shopify and has not been spoofed or tampered with.

This header is an encrypted version of the payload and is computed using the private app's shared secret, the webhook payload, and the HMAC cryptographic algorithm.

To verify your webhook on your external API, you compute the same encrypted version of the webhook payload using the shared key and HMAC, then compare the result with the value sent in the X-Shopify-Hmac-Sha256 header. If there is a match, then the payload is valid and it originated from Shopify. If not, then it's suspicious and should be ignored.

You can find this verification logic in the server.js file as a commented middleware function validatePayload, shown below:

const sigHeaderName = 'x-shopify-hmac-sha256';
const sigHashAlg = 'sha256';
const secret = "xx-xx-x";

.....

/* function validatePayload(req, res, next) {
    if(req.method == "POST"){
        if (!req.rawBody) {
            return next('Request body empty')
        }
        const body = req.rawBody;
        const hmacHeader = req.get(sigHeaderName);
        //Create a hash based on the parsed body
        const hash = crypto
            .createHmac(sigHashAlg, secret)
            .update(body, "utf8", "hex")
            .digest("base64");
        // Compare the created hash with the value of the X-Shopify-Hmac-Sha256 Header
        if (hash !== hmacHeader) {
            return next(`Request body digest (${hash}) did not match ${sigHeaderName} (${hmacHeader})`)
        }
    }
    return next()
}
app.use(validatePayload); */

For more information on security considerations for your webhooks, check out our security checklist.

Conclusion

In this article, you have learned and demonstrated how to perform Shopify webhook operations using the webhook API. Being a programmatic interface, the Shopify API allows you to run automated tasks that can, for example, update and fetch webhooks on the fly. You can perform these tasks directly in your code or within CI/CD pipelines to tailor Shopify webhooks towards your application/infrastructure needs.

Happy coding!

Tutorial: How to Use Shopify's Admin Dashboard to Create Webhooks

Fikayo Adepoju — Thu, 16 Dec 2021 19:30:46 +0000

Introduction

In a previous article, we looked at how Shopify webhooks provide a one-way communication medium for Shopify to send notifications and data about events taking place in your store to an external application. In this article, we will take a more hands-on approach to learning and understanding how Shopify webhooks work.

Each Shopify store comes with an administrative interface to manage the store. Webhooks can also be created and managed using this interface. In this tutorial, we are going to create a webhook, test it, and later delete it through Shopify's admin interface.

Read this tutorial if you are more interested in using Shopify's webhook API.

Ready? Let's get started.

How Shopify webhooks work

As stated earlier, Shopify webhooks help you integrate external applications with the Shopify store by creating a channel through which Shopify sends notifications for events happening in the store to the external application.

A webhook is set up on Shopify by subscribing to a topic on your store. A topic is an event that can take place on a Shopify store resource e.g, a Cart can be updated. Cart in this example is the resource while updated is the event, and this topic on Shopify is represented as Cart/update.

When a webhook subscription has been created, an HTTP request is triggered each time an event for that topic occurs. This HTTP request is sent to an endpoint on your external application. This endpoint is known as the Webhook URL and is supplied when creating a webhook on Shopify.

Shopify requires that webhook URLs use the secure HTTPS protocol and the POST request method.

Your external application receives the webhook with its payload at the webhook URL endpoint and triggers any custom processes in response to the webhook.

Tutorial: Logging Shopify cart events

In this tutorial we will explore logging, a common use case for Shopify webhooks, as engineers often use webhooks to feed information into an external logging system. These logs are mostly integrated with tools that help visualize the webhook data.

Here's a quick summary of the steps we need to take to achieve the Shopify webhook logging setup:

Clone the workflow logger API.
Generate a webhook URL that will be used to create a webhook on Shopify.
Set up a webhook on Shopify.
Trigger and inspect the webhook.
View the webhook logged by the API.

Now let's look at what you need for the project.

Project requirements

To set up our demo logging system, we are going to use a local Node.js API with an endpoint that receives and logs a portion of the webhook payload.

To follow along with this tutorial, it's required that you have the following items set up:

Have Node.js installed on your system, version 12 or greater is fine (if you already have a local API that you prefer to use, that is fine. Just make sure it exposes a POST endpoint and listens on a known port).
Hookdeck CLI installed (you can find details on installing the tool here).
An active Shopify store with access to Shopify apps (Shopify apps are required for the webhook authentication step, but more on this later).

Cloning the demo API

Clone the application repository for the API by running the following command:

git clone --single-branch --branch shopify-webhooks https://github.com/hookdeck/nodejs-webhook-server-example.git

Navigate to the root of the project and install the required dependencies by running the following commands:

cd nodejs-webhook-server-example
npm install

When the installation completes, you can then run the Node.js server with the following command:

npm start

This will boot up the API application and print a message to the screen indicating that the API is now running and listening for connections on port 1337.

We are using two endpoints in this project.

/log-shopify-webhook: This is the endpoint that will be receiving the Shopify webhook and logging it into an in-memory database. It logs a simple object containing a subset of the information from the webhook payload.
/fetch-webhooks-logs: This endpoint can be called to retrieve a collection of the logged webhook data.

Generating a webhook URL using the Hookdeck CLI

We now have a locally running server that contains an endpoint to receive our webhook. However, this endpoint is not accessible to the public, thus it cannot be targeted by Shopify webhooks.

A mechanism is required to route our webhooks to the locally running API endpoint, and this is where the Hookdeck CLI comes in. The Hookdeck CLI is built specifically for working with webhooks, and it helps tunnel external HTTP requests into your local development environment by providing you with a publicly accessible HTTPS URL. You can also inspect the headers and payloads of your webhooks, as you will see later on in this tutorial.

Visit Hookdeck's CLI documentation to install and set up the tool on your operating system. For macOS users, you can run the following command to install the CLI tool:

brew install hookdeck/hookdeck/hookdeck

If you're using the Windows operating system, use the following command to install the CLI tool:

scoop bucket add hookdeck https://github.com/hookdeck/scoop-hookdeck-cli.git
scoop install hookdeck

Once the setup process is complete, the next step is to use the CLI to generate a webhook URL that points to the running API application. To do this, run the following command (note: replace the port 1337 value with the port number on which your API is running if you're not using the sample project):

hookdeck listen 1337

What source should you select? Ans: select **Create new source* (this prompt does not show up if you have not created any previous connections)*
What should your new source label be? Ans: type the text Shopify
What path should the webhooks be forwarded to (i.e.: /webhooks)? Ans: type /log-shopify-webhook (if you're using your own custom server, replace this value with your endpoint)
What's the connection label (i.e.: My API)? Ans: type Purchase Log API

With this information, the CLI will begin the process of generating the URL and once it's done, you will see the URL printed to the screen and the CLI indicating that it is ready to receive requests.

Creating a webhook subscription using the Shopify admin interface

Now that you have your webhook URL, the next step is to create a webhook subscription on Shopify. On your Shopify admin dashboard, navigate to Settings → Notifications → (scroll down to) Webhooks. Click the Create webhook button and fill the webhook form as follows:

Event: Cart Update
Format: JSON
URL: The Webhook URL printed to your Hookdeck CLI session screen
Webhook API version: Select the version marked **Latest**

See below an example of the filled form:

Click Save to complete the webhook creation process. You will then see your newly created webhook displayed in the Webhooks list as shown below:

Testing your webhook

Great! You now have your webhook logging system set up, and it's time to put it to the test. Conveniently, Shopify provides a way to send a test webhook notification, which you can do by clicking the Send test notification button on your webhook row as displayed in the previous image.

Click this button for your Cart update webhook subscription to send a test webhook. Once sent, you will see an entry in your Hookdeck CLI session like below:

As seen in the above image, the status of the request is 201, indicating that a new resource has been created on the logger API.

To view details of the webhook, copy the event URL, which is the last item on the logged webhook in the CLI. Load this URL in your browser and you will see a screen similar to the one below:

The webhook event page provides you with in-depth details regarding the webhook you just received. Starting from the top section you can see some metadata provided by Hookdeck, shown below:

Next is the Headers section. Here, you can access all the headers that came with the webhook and inspect their values:

For the webhook payload, the Body section contains a browsable object representing the webhook payload. You can expand this object to inspect all the properties contained in the data that came in the payload:

Viewing the webhook logs

So, we have successfully received a webhook on our logging application. Let's now check the application logs to confirm that our webhook is indeed being logged. To have an appreciable amount of webhooks logged, click the Send test notification button a few more times.

Then, on your browser, navigate to the /fetch-webhooks-logs endpoint of the locally running API, and you will see a display similar to the one below:

As seen in the above image, we have 3 webhooks logged (the first entry is a test entry already contained in the database). We have been able to log the webhook ID, the total line items in the cart, and the time the webhook was triggered for each webhook. Now, it's time to verify.

How to verify Shopify webhooks with HMAC Signature

Webhooks create a communication medium between your external apps and Shopify, so you need to make sure that only Shopify is able to use this medium to speak to your app. Attackers often take advantage of the open-ended nature of a webhook URL to send requests containing malicious content to the webhook endpoint.

Fortunately, in the X-Shopify-Hmac-Sha256 header sent with each webhook, Shopify sends an encrypted version of the payload. This encrypted value is created using your Shopify app's shared secret key and the HMAC cryptographic algorithm.

You can get a Shopify shared secret for your store by creating a Shopify application in the Apps section. The shared key is contained in the Admin API section of the app created.

When you receive a webhook, you can compute the same encrypted version of the payload using the shared key and HMAC, and then compare the result with the value sent in the X-Shopify-Hmac-Sha256 header. If there is a match, then the payload is valid and it originated from Shopify. If not, then it's suspicious and should be ignored.

You can find this verification logic in the server.js file as a commented middleware function validatePayload, shown below:

const sigHeaderName = 'x-shopify-hmac-sha256';
const sigHashAlg = 'sha256';
const secret = "xx-xx-x";

.....

/* function validatePayload(req, res, next) {
    if(req.method == "POST"){
        if (!req.rawBody) {
            return next('Request body empty')
        }
        const body = req.rawBody;
        const hmacHeader = req.get(sigHeaderName);
        //Create a hash based on the parsed body
        const hash = crypto
            .createHmac(sigHashAlg, secret)
            .update(body, "utf8", "hex")
            .digest("base64");
        // Compare the created hash with the value of the X-Shopify-Hmac-Sha256 Header
        if (hash !== hmacHeader) {
            return next(`Request body digest (${hash}) did not match ${sigHeaderName} (${hmacHeader})`)
        } 
    }
    return next()
}
app.use(validatePayload); */

For more information on security considerations for your webhooks, check out our security checklist.

How to delete Shopify webhooks

Deleting webhooks from your Shopify store is simple and straightforward using the admin interface. Simply click the bin icon next to your webhook on the webhooks list:

Shopify will prompt you for to confirm to ensure that you really want to take this action. Note that once a webhook is deleted, it cannot be recovered again.

Conclusion

The Shopify administrative interface provides easy-to-use tools to create and manage your store webhooks. In this tutorial, you have learned and demonstrated how to use the Shopify admin interface to set up a webhook, test it, and then delete the subscription.

If you like doing things programmatically, or want to set up automated tasks like scheduled cron jobs to interact with Shopify webhooks and other store resources, Shopify also provides an API that you can call. We have detailed how you can authenticate with Shopify's API and use it to set up your webhooks in this article.

Happy coding!

Shopify Webhooks Best Practices

Fikayo Adepoju — Thu, 16 Dec 2021 19:16:04 +0000

Introduction

In this series, we've explored how to get started with Shopify webhooks, and shared step-by-step tutorials on how to create webhooks using both the Shopify admin dashboard and Shopify API. Now, we'll conclude the series by looking at some important best practices for building resilience and reliability into Shopify webhooks in a production environment. We want reliable Shopify webhooks because sudden spikes in your Shopify webhook traffic can overload your API and cause it to shut down if the capacity of your API server is exhausted. Shopify can also send duplicate webhooks that could potentially cause inconsistencies in your application database.

In this article, we will share and suggest some key engineering practices that will help make your Shopify webhooks resilient to the usage pressures common in production environments.

Shopify webhooks features

Before getting into best practices for Shopify webhooks, let's take a look at its features. Understanding the strengths and weaknesses of Shopify webhooks and how they operate helps engineers design the most efficient solutions to mitigate any issues that might arise when using them.

Feature	Related Best Practice	Related Best Practice
Webhook Configuration	Admin API and Shopify admin (REST or GraphQL)	Integration development
Webhook URL	Allow one per subscription	Integration development
Response Format	JSON or XML	Integration development, troubleshooting and testing
Request Method	POST	Integration development, troubleshooting and testing
Browsable Log	Delivery metrics on your partner dashboard	Troubleshooting and testing, failure recovery
Hashing Algorithm	SHA256	Security and verification
Manual Retry	Not available	Failure recovery
Automatic Retry Logic	Exponential, 19 times over 48 hours	Failure recovery
Alert Logic	Shopify sends an alert for each failure right before it deletes any webhooks	Failure recovery
Timeout period	5 seconds	Scalability with asynchronous processing

Now that you're familiar with the features of Shopify webhooks, let's go ahead and break down solutions to issues that may arise in production environments.

Shopify webhooks best practices checklist

Local troubleshooting and testing

Properly testing your Shopify webhooks helps you avoid all the performance issues that arise due to buggy code. For example, let's say your endpoint on an external accounting application assumes that Shopify will send the total price for a store order as an integer value (e.g. 10). Meanwhile, Shopify sends the price as a floating-point number (10.15). Your accounting application then forcefully coerces the floating number into an integer value which does not reflect the true cost of the order, and will lead to inconsistencies in the accounting data.

You need to test your Shopify webhook endpoint to ensure the following:

The endpoint exists (to avoid 404 errors)
The endpoint does not perform a redirect (Shopify sees a redirect as an error)
The logic on the endpoint is bug-free and has the desired impact on your application
You return a proper status code for successfully processing the webhook (2xx range of HTTP status code) and for failure (4xx, 5xx range of HTTP status codes)

One drawback in testing Shopify webhooks is that webhooks cannot be received in a local development environment because they require a publicly accessible URL. Fortunately, tools like the Hookdeck CLI exist to solve this problem by tunneling your Shopify webhook requests to your local environment.

Asynchronous processing to ensure reliability

When Shopify sends a webhook, it waits for a response from the receiving application. This waiting period elapses after 5 seconds. Thus, if Shopify does not receive a response within the waiting period, Shopify assumes that the webhook delivery has failed.

As a best practice, you need to respond to Shopify as fast as possible. This practice is recommended on Shopify's webhook documentation. That said, it might not be possible to respond so quickly under certain conditions, such as:

1) Long-running processes

Your webhook can trigger a time-consuming task, such as file encryption, database queries, or computations that take long periods to complete. It may be difficult, and sometimes impossible, to send a response within the 5-second limit in this situation.

2) Multiple webhooks

Let's assume that the time it takes your application to process a single webhook is 3 seconds; you would be able to return a response within the 5-second limit. However, if your application receives 3 webhooks at the same time, the time required becomes 9 seconds and the second webhook will not complete before it times out.

Imagine you have a Shopify store that sells flowers, and you process an average of 10 orders daily. You have set up a webhook that targets an external delivery service when a customer places an order. Your delivery service comfortably handles your daily average load and then suddenly, Valentine's Day season rolls by, and the number of orders for flowers on your store surges to an all-time high of 300 orders per day.

You start receiving 300 webhooks daily on your delivery service, which causes the server resources for the application to be used up quickly and drop webhooks due to service failure. As webhooks begin to drop, many customers that have ordered and paid for their flowers won't have their flowers delivered.

So, how do we mitigate these often unavoidable situations? One of the standard solutions is to boost processing speed by expanding the server hardware capabilities. You can also adopt horizontal scaling techniques to distribute your webhook workload.

The above solutions are good, but the best way to respond to Shopify as quickly as possible is by processing the webhooks asynchronously.

With asynchronous processing, Shopify does not need to wait for your application to finish processing a webhook before it receives a response.

One of the ways to implement asynchronous processing for Shopify webhooks is by placing a message broker between Shopify and your application. The message broker ingests webhooks from Shopify and immediately sends a response back to Shopify. The broker then routes the webhooks to your application at a rate that your application can handle.

This way, your Shopify webhooks never time out, and your application does not run out of server resources.

Verifying your incoming webhooks

The next best practice we'll look at is security. The HTTPS endpoint used by your applications to receive Shopify webhooks is publicly accessible, which means that any client can call it.

This open-ended nature of your webhook URL is a huge security flaw, as attackers can spoof webhook requests to your application for various malicious purposes. You need a way to prevent your webhook endpoints from receiving requests that are not from Shopify.

Let's imagine that you have a Shopify store that sells basketball jerseys, with a webhook set up to message a delivery service about the order details for the jersey to be delivered to the customer's address. Without verification, an attacker can spoof order webhooks to your delivery service webhook URL and get free jerseys anytime they want.

Fortunately, Shopify provides a way to help us with this. Shopify sends a hashed version of the body of the payload in the X-Shopify-Hmac-Sha256 header (if you're using Ruby on Rails or Sinatra, the header is HTTP_X_SHOPIFY_HMAC_SHA256 ). The value of this header is a base64 encoded string and serves as the digital signature for the webhook payload. This signature is signed using your Shopify app's shared secret. For more information about Shopify app secrets and how to generate one, check the docs page.

To verify the payload on your endpoint handler, you need to re-compute the signature using your Shopify secret, the webhook payload, and some HMAC algorithm functions. Once you have your version of the payload signature, you can then compare it with the one sent in the X-Shopify-Hmac-Sha256 to confirm a match.

Below is an example of the verification process written in Node.js:

const getRawBody = require("raw-body"); //install raw-body from npm
const crypto = require("crypto");
const secret = MY_SHOPIFY_SECRET;

app.post("/webhook", async (req, res) => {
//Extract X-Shopify-Hmac-Sha256 Header from the request
  const hmacHeader = req.get("X-Shopify-Hmac-Sha256");

//Parse the request Body
  const body = await getRawBody(req);
//Create a hash based on the parsed body
  const hash = crypto
    .createHmac("sha256", secret)
    .update(body, "utf8", "hex")
    .digest("base64");

  // Compare the created hash with the value of the X-Shopify-Hmac-Sha256 Header
  if (hash === hmacHeader) {
    console.log("Webhook source confirmed. Continue processing");
    res.sendStatus(200);
  } else {
    console.log("Unidentified webhook source. Do not process");
    res.sendStatus(403);
  }
});

It is also worth mentioning that you should store your Shopify secret in an environment variable, especially in production environments. The entire process of verifying your webhook breaks down if an attacker can get hold of your Shopify secret.

Idempotency in webhook processing

Idempotency helps prevent the negative impact of processing a webhook more than once. Let's imagine that we need to deduct the price of a purchased item from a customer's wallet (the wallet application is external to the Shopify store). A charge is deducted from the wallet when we receive a webhook confirming the purchase. If this process is done more than once for a single purchase, the customer's wallet is now in a state that is inconsistent with their purchase history.

Situations like this negatively impact the integrity of your application's data, and in extension, the Shopify store itself.

While some activities, like updating the status of an order, are by nature idempotent, others like the one described above are not. You need a mechanism built into your webhook processing operations to ensure that no webhook is processed more than once.

Shopify helps to achieve this by sending the unique identifier for a webhook in the X-Shopify-Webhook-Id. With this identifier, you can track webhooks that have already been processed and skip them to avoid duplicating their impact on your application.

Dealing with delayed webhooks

On rare occasions, Shopify might send a webhook late (sometimes up to a day late). If the data in the payload is very critical to your application, you should verify how recent the data is. Stale data has the potential to cause state inconsistencies within your application, thereby compromising the database.

For example, think of an external inventory system where the quantity of an item is updated (i.e. deducted) after a successful purchase on Shopify. If the item is no longer available (quantity = 0 in the inventory system) and you have a late purchase webhook coming in, the current quantity in the inventory will either be negative, undefined, or default to zero based on the business logic implemented. However, this new inventory value does not reflect the actual state of the item quantity.

Shopify does its best to send the most recent data with a webhook even when the webhook delays. As an engineer, you know that assumptions can easily lead to bugs, so you want to make sure you're verifying how recent the webhook data is despite the promise by Shopify.

One way to implement this verification is by comparing the timestamp of the webhook to the current time. If the timestamp on the webhook is behind the current time, you can call the Shopify API to request the current data. This check ensures that you're getting the most up-to-date data for the Shopify resource.

Implementing reconciliation jobs

This best practice falls under the housekeeping category. Webhooks are sent to notify your external application and send real-time data about an event on a Shopify resource. However, Shopify advises that you shouldn't solely rely on webhooks because, and this is very rare, the webhook for a particular event might not be sent.

An example of a scenario where a reconciliation job is required would be if you had a Shopify store selling shirts and an external accounting application integrated with your store using webhooks.

Let's assume that you need to balance your accounts in the external accounting application at the end of each business day. A reconciliation job can be scheduled to run 30 minutes before you query your transaction history to compute your balances. This job can be used to pull all purchase information from your Shopify store and synchronize it with your accounting application so that you have all the data you need up-to-date.

Thus, to mitigate the issue of unsent Shopify webhooks, you should implement reconciliation jobs to synchronize your application data with Shopify. These reconciliation jobs are scheduled tasks that periodically fetch data from Shopify for specific resources.

Reconciliation jobs should be set up for sensitive data within your application; these are data that need to be up-to-date at certain periods while your application is running.

Conclusion

Building reliability and resilience into your infrastructure ensures that your setup can withstand the pressures of production environments. In this article, we have looked at some best practices to follow to ensure that our Shopify webhooks function without issues. This list is not set in stone, and you're encouraged to take more precautions that help prevent your application integration with Shopify from falling short of expectations.

Happy coding!

Getting Started with Shopify Webhooks

Fikayo Adepoju — Thu, 16 Dec 2021 19:04:22 +0000

Introduction

Shopify is a great e-commerce SaaS application that offers a lot of flexibility and in turn helps developers create custom integrations. By using the API, merchants can programmatically interface their store with their custom applications. On top of the API, Shopify has introduced webhooks, a way to transfer event data from stores to custom integrations in real-time.

In this article, we will take a tour of Shopify webhooks to learn what they are all about, how to set them up, and how to find out more information on them.

What is a Shopify webhook?

Shopify webhooks are a one-way communication system. When an event occurs for a subscription to a Shopify topic (e.g. orders/create), an HTTPS request is triggered. This HTTPS request is then sent to an endpoint on the receiving application.

A Shopify topic is an event that can take place on a Shopify resource. In the orders/create example stated above, the resource isorders, and create is the event.

To understand things better, let's use a simple example to describe the webhook communication process.

A Shopify merchant that sells jerseys wants to send notifications for jersey orders on their Shopify store to an external delivery service. This delivery service is responsible for processing the delivery of the jersey to the customer.

The Shopify merchant goes into their Shopify admin dashboard (or uses the API) to create a subscription for the orders/create topic by submitting an endpoint on an external application where the webhook is to be sent. A customer visits the store and places an order. Once this order is created, the event for the orders/create topic is fired. Because there is a webhook subscription for this topic, an HTTPS request is immediately triggered to the external application's endpoint.

A Shopify webhook request consists of different parts. The webhook headers contain metadata about the webhook, and the payload contains the actual Shopify event information. The payload can be sent in either JSON or XML format. The format your webhook payload is sent as can be configured on Shopify's admin interface or through the Shopify API.

It is important to note that to receive webhooks, Shopify requires a publicly accessible HTTPS endpoint. Tunneling solutions like Hookdeck's CLI and Ngrok provide a way to generate an HTTPS endpoint that channels webhook requests to your development environment.

Supported Shopify webhook events

Shopify webhooks help applications that want to stay in sync with Shopify by updating internal states or executing code based on an event that occurs in a store. Applications that are often integrated with Shopify using webhooks include invoicing systems, accounting apps, and mailing/notification services, among other things.

Merchants can integrate Shopify webhooks to monitor sales, update inventory, perform accounting operations and synchronization, and also perform many more auditory and sales management processes. Making the right integration involves knowing the right event to subscribe to, so let's look at Shopify webhook events and how we can make use of them.

An event is fired when an action for a Shopify topic (e.g. cart/update topic, which means a Shopify cart is updated) takes place on Shopify. This action can be performed by the store admin, a customer, or Shopify itself.

Shopify supports a lot of events that webhooks can subscribe to. These events are namespaced by the resource that fires the event; for example, we have the Orders/closed and Orders/cancelled events belonging to the Orders resource.

Most e-commerce integrations focus on the Orders events as they generate webhooks for events involved in the process of purchasing items from the Shopify store. There are also events for resources like Cart, Checkout, Customer, InventoryItem, etc.

Let's take a look at some of these resource events, as well as example scenarios where they can be used in our application workflow.

Cart events

These events are fired when a customer creates or updates items in their shopping cart. The two available cart events are carts/create and carts/update. One potential use case for these events is to query an external inventory to check if an item is still in stock. You can also use these events as ping signals to an external inventory application in order to sync the number of items in the inventory with the number of items on your Shopify store.

Order events

Order events are fired when a Shopify order is created, updated, paid for, or deleted. Events under this category include: orders/cancelled, orders/create, orders/delete, orders/edited, orders/fulfilled, orders/paid, orders/partially_fulfilled, and orders/updated. Below are some scenarios where you can use webhooks for these events:

Reverting a payment through an external payment platform when a customer cancels an order on your Shopify store (orders/cancelled)
Notifying a delivery service once a customer pays for an item they ordered from your Shopify store (orders/paid)
Locking items that have been ordered in an external inventory application to ensure that preceding orders do not assume the wrong availability of an item or a wrong quantity of items (orders/create)

Dispute events

These are fired when a customer creates or updates an issue (complaint or comment) relating to the service they got from your store on Shopify. The two events under this category are disputes/create and disputes/update. One use case for these events is to create and update issues on an external issue tracker like Gorgias. Services like Gorgias have more customer support features built-in than what is available on Shopify.

To effectively take advantage of what Shopify webhooks have to offer, it is important to have a good knowledge of the Shopify events that you can subscribe for. This knowledge will help you build the proper workflows for webhooks within your application or third-party integrations.

It is also important to know that not all Shopify resources generate events. For a detailed list of Shopify events, check out the documentation page.

Creating a webhook using the Shopify admin interface

Now that you have a good understanding of Shopify webhooks and how they work, and you have chosen them as a solution to a project you're working on, let's look at how to create one in the Shopify admin interface. You can also follow along if you just want to get some practical knowledge on setting up webhooks on Shopify.

The easiest way to set up a webhook on your Shopify store is through the admin interface. To locate the admin interface, navigate to https://[MY_STORE_ADDRESS].com/admin. From the admin interface, go to Settings > Notifications > Webhooks. Click the Create Webhook button and fill the webhook form by selecting an event, the payload format, and entering your application endpoint (i.e. the endpoint on the application where you will be receiving your Shopify webhooks) into the URL field as shown below:

Ensure you select one of the stable API versions then click the Save button.

You will now receive a webhook request on the endpoint you supplied whenever the selected event occurs in your Shopify store.

For a more detailed hands-on tutorial on using the Shopify admin interface to create and manage your webhooks, check out this article.

Using the Shopify webhooks API

Webhook subscriptions cannot only be created from the Shopify Admin interface — you can also create and manage your webhooks dynamically by calling the Shopify webhooks API! The Shopify API comes in two forms: the very common REST API and a GraphQL variant.

The REST API consists of endpoints that you can use to create a webhook, retrieve webhooks, modify a webhook and also remove a webhook. The GraphQL API also consists of queries and mutations that mirror the same functionalities you get from the REST API.

To use the Shopify webhooks API, you need to be authenticated. Shopify also provides a good number of client libraries for languages like PHP, Node.js, and Python that make the developer experience a lot better.

For more information and a detailed analysis of using the Shopify API to manage your webhooks, check out our tutorial on using the Shopify webhooks API.

Finding information on the Shopify webhooks documentation

The Shopify documentation contains a great deal of information on webhooks; however, it can be a bit cumbersome moving around tons of documentation pages to find the exact information you are looking for. I have made a simple list below to help you easily find enough information to start using Shopify webhooks as a professional:

Resource	Why you should check it
Overview	If you are looking for more information on Shopify webhooks, this overview will be useful in helping to deepen your understanding of Shopify webhooks.
Shopify Webhook Configuration	This page contains a simple list of steps to take when configuring your webhook workflow.
Setting up your endpoint to receive webhooks	If you're setting up a Shopify webhook for the first time, this page takes you through a step by step process showing how to get your webhook set up properly. You learn how to prepare your endpoint to receive webhooks, how to test your webhook connection, and also how you can verify your webhook source.
API documentation	If you are looking to go down the API route, this page furnishes you with all the information about available endpoints, requesting an response format, supported events and more.
API versions	Over the years, different versions of the Shopify webhooks API have been released. If you're working on a project that uses older versions of the API, this page helps you manage different API versions in your code, and also teaches you how to upgrade to the latest version.

Conclusion

A large amount of activities on the internet are e-commerce related, and Shopify plays a huge part in that space. Shopify may have taken care of a lot of the basic requirements of an e-commerce store, but the platform's capacity for extensions makes it one of the most preferred choices for setting up an online store.

In this article, you have learned about Shopify webhooks and how they enable you to build custom integrations with the platform. This way, you can complement any missing feature in Shopify or substitute it with a better or preferred alternative.

Happy coding!

Troubleshooting GitHub Webhooks

Fikayo Adepoju — Mon, 01 Nov 2021 13:43:58 +0000

Introduction

In a previous article, I demonstrated the use of GitHub webhooks in a simple tutorial using a sample API that logged webhook events. In this article, we will learn how to debug common issues with GitHub webhooks using the same sample project. This time, I have introduced some bugs into the project that we are going to learn how to debug.

Troubleshooting GitHub webhooks: Requirements checklist

To begin there are a few things you need to have or set up, including:

A GitHub repository
Node.js installed on your system to run the sample project
A publicly accessible URL to the API endpoint. We will use the Hookdeck CLI to achieve this. To install the Hookdeck CLI, check out this page.
Clear visualization of error messages. The Hookdeck CLI will help us with event pages where error details can be inspected.
A text editor for editing code

With this setup, you have an environment in which you can conveniently troubleshoot your GitHub webhooks.

Troubleshooting GitHub webhooks with Hookdeck

Cloning and running a demo API

The sample API we will use is available on the Hookdeck GitHub repository. Follow the instructions to set it up.

Clone the repository by running the following command:

git clone --single-branch --branch github-webhooks-debugging https://github.com/hookdeck/nodejs-webhook-server-example.git

Navigate to the root of the project and install the required dependencies by running the following commands:

cd nodejs-webhook-server-example
npm install

When the installation completes, run the Node.js server with the following command:

npm start

This will boot up the API application and print a message to the screen indicating that the API is now running and listening for connections on port 1337.

We are using two endpoints in this project:

/log-github-webhook: This is the endpoint that will be receiving the GitHub webhook and logging it into an in-memory database. It logs a simple object containing a subset of the information from the webhook payload.
/fetch-webhooks-logs: This endpoint can be called to retrieve a collection of the logged webhook data.

Getting a GitHub webhook URL

The next step is to use the CLI to generate a webhook URL that points to the running API application. To do this, run the following command:

hookdeck listen 1337

What source should you select? Ans: select Create new source
What should your new source label be? Ans: type the text GitHub
What path should the webhooks be forwarded to (i.e.: /webhooks)? Ans: type /log-github-webhook
What's the connection label (i.e.: My API)? Ans: type My Github Response Server

With this information, the CLI will begin the process of generating the URL and once it's done, you will see the URL printed to the screen and the CLI indicating that it is ready to receive requests.

Note: You will need to use the guest Login URL link in the console to access the dashboard. Copy and paste this into your browser to begin a guest login session.

Confirming webhook delivery

To begin receiving webhooks, you need to set up a webhook on your GitHub repository.

Go to Settings → Webhooks. On the Webhooks page, click on the Add webhook button on the top right-hand corner.

On the webhook form displayed, paste the webhook URL generated by the Hookdeck CLI into the Payload URL field.

Fill the remaining fields as follows:

Content type: Select application/json so that you can receive the payload as a JSON object.
Secret: You can leave this blank, but for this tutorial enter 1234ABCD.
SSL Verification: Leave this as the default option, Enable SSL verification.
Which events would you like to trigger this webhook: This is the point where you subscribe for a GitHub event on your repository. For this tutorial, select the Just the push event option as we are only subscribing to the push event. You can either subscribe for all events or a subset of the events using the other two options.
Active: Leave this checked to receive event details when the GitHub webhook is triggered.

See the complete webhook form below:

Click the Add webhook button to complete the process.

With this setup, anytime you push code to your repository, a webhook request will be fired to the specified webhook URL.

Immediately after you complete registration for a webhook in the step above, a ping webhook request will be fired to your webhook URL. Because we are using the Hookdeck CLI, we would already see the ping webhook request logged on the terminal where the Hookdeck CLI is running, as shown below:

This confirms that we are successfully receiving our webhook request. However, did you notice the status code indicated on the webhook request? Yeah, a 404.

Let's take a look at how to deal with this error in the next section.

Troubleshooting GitHub webhook "not found" error

Our first webhook attempt resulted in a 404 error on the destination endpoint. This tells us that even though we are successfully receiving our webhooks, it cannot locate the endpoint specified, or the endpoint does not exist. 404 errors can be fixed, most of the time, by checking for typos or misspellings in the specified route name. Worst case scenario, the route truly does not exist and needs to be created.

Use the event page link displayed on the CLI to go to the event page, where you will see a page similar to the one below:

Scroll down to the red status code badge under the Attempts section and click on it to reveal details about the error on the right-hand side of the screen, as shown below:

This helps us see the actual response from the running server, and it also confirms that the specified endpoint cannot be found.

The endpoint specified for the webhook to hit is /log-github-webhook, which can be found in the routes.js file at the root of the project directory.

router.post("/log-github-webhoo", async function(req, res) {
  //console.log(req.body);

  const payload = req.body;

  let webhook_info = {
    repo : payload.repository.name,
    author : payload.sender.login,
    time : payload.head_commit.timestamp
  }

  const save_webhook = await req.db
  .collection("webhooks")
  .insertOne(webhook_info);

  res.status(201).send({
    message: "Webhook Event successfully logged"
  });
});

Taking a close look at the route handler, you will notice that there is a typo just at the end of the endpoint name: a missing 'k'. Simply add the missing character and save the file.

You will need to restart the Node.js server to allow the changes to take effect. Shut down the server using Ctrl + C and restart it with the npm start command.

Now, fire a new webhook to your endpoint by pushing a commit to your GitHub repository. Once this is done, check your Hookdeck CLI session for the new webhook received. You will see an entry similar to the one below:

As seen from the above webhook received, we have been able to clear the 404 error. However, we now have a new error: a 401 error. Let's deal with this in the following section.

Troubleshooting GitHub webhook error 401

HTTP 401 errors indicate that an unauthorized action is being performed. Before making any assumptions, let's inspect the server response using the event's page. Copy the event page link from the Hookdeck CLI session and load it in your browser, and you will see a screen similar to the one below:

Scroll down to the red status code badge under the Attempts section and then click on it. You will see the server response details on the right-hand side as shown below:

Reading the response message, we can see that we are failing the security check on the webhook payload. Because we defined an API secret, GitHub sends the X-Hub-Signature-256 header. This header contains an encrypted version of the real payload and has to be validated against the unencrypted payload received by using the secret key.

This is a security check to prevent attackers from replacing the actual payload with a malicious one with the intention of corrupting our API.

Because we know that we are receiving the actual payload from GitHub since we set everything up and triggered the push event, we know it's not an attacker's payload causing the 401 error. Something else must be wrong with our setup.

Let's take a look at the validation logic on our API. This can be found in the server.js file, as shown below:

//Validate payload
function validatePayload(req, res, next) {

    if(req.method == "POST"){
        if (!req.rawBody) {
            return res.status(400).send({
                message: "Request Body empty"
            });
        }

        const sig = Buffer.from(req.get(sigHeaderName) || '', 'utf8')
        const hmac = crypto.createHmac(sigHashAlg, secret)
        const digest = Buffer.from(sigHashAlg + '=' + hmac.update(req.rawBody).digest('hex'), 'utf8');

        if (sig.length !== digest.length || !crypto.timingSafeEqual(digest, sig)) {
            return res.status(401).send({
                message: `Request body digest (${digest}) did not match ${sigHeaderName} (${sig})`
            });

        }
    }

    return next()

}
app.use(validatePayload);

There are a couple of variables being used here in our code that are worth checking out to confirm that they are referencing the appropriate values. These are:

sigHeaderName: This represents the signature header name sent by GitHub.
sigHashAlg: This is the algorithm used for the encryption.
secret: This is the API secret set on our GitHub webhook.

These values are set on lines 10-12 in the server.js file. Lets take a look at these definitions:

const sigHeaderName = 'X-Hub-Signature-256';
const sigHashAlg = 'sha256';
const secret = "XXXXXX";

After observing these values, have you been able to catch the error? If the secret is not the same as the one we set in our webhook form, the validation will always fail.

GitHub does not allow you to view the API secret you set in your webhooks a second time — you can only reset it. This is why you need to make sure that you remember the value or store it somewhere secure.

For this practice, we know we set the secret as a simple 1234ABCD string. In real-world applications, you want to set a more complicated secret and reference it from an environment variable in your code.

Change the value of the secret to the right value, save the file, and restart the server. Now trigger another webhook by pushing a new commit to your GitHub repository. You will see a webhook entry similar to the one below on your Hookdeck CLI session:

Finally, we have a successful operation on our API. Load the event page, which should display a page similar to the one below:

Now click on the green status badge below the Attempts section to show the server response. The server returns a successful message, as shown below:

This time, when you visit the endpoint /fetch-webhooks-logs, you will see a collection like the one below:

Troubleshooting GitHub webhook "invalid HTTP response 400" error

Another error that can occur with GitHub webhooks is a 400 HTTP error, which indicates a bad request. One thing to note when working with GitHub webhooks is that GitHub is your client. Your application, or whichever system (SaaS apps, CI/CD server, etc.) that you're integrating GitHub webhooks with, is the server.

Thus, when you see a bad request, it means that your server is not receiving what it expects from GitHub.

The first thing you want to check is the format in which you're sending your webhook payload. GitHub allows you to set the Content-type to either application/json or application/x-www-form-urlencoded .

Make sure that the value you configured in your webhook form matches the content type your server is expecting.

Sometimes, 400 errors arise from empty payloads being sent by GitHub. If your server expects a payload to be sent, make sure that the Active option on your GitHub webhook form is checked.

Conclusion

Just like any other architectural component of an application, webhooks run into errors. Debugging is considered part of the workflow when developing applications or setting up application infrastructure. As errors are inevitable, knowing what to look out for when a certain error occurs and having the right tools to debug the situation eases the pains experienced in troubleshooting.

You also save time and push out solutions faster!

GitHub Webhooks Tutorial

Fikayo Adepoju — Thu, 21 Oct 2021 18:44:27 +0000

Introduction

As one of the biggest code hosting platforms available today, GitHub provides numerous ways for developers to integrate with the platform. One of those integration options is GitHub webhooks.

In this article, we are going to explore GitHub webhooks. We will look at what a GitHub webhook is, and how to set it up on our GitHub account. Further, we will explore a sample API that receives and logs webhook events. This API will run on our local development environment, thus we will be receiving the webhooks on our local machine. However, if you have a local API you would rather use, feel free to follow along with the article.

What is GitHub Webhooks?

GitHub Webhooks gives developers the ability to integrate with the GitHub platform.

Developers can subscribe to events taking place on GitHub, such as push events that occur when code is pushed to a repository, and respond to the event with an action.

The response action can be anything: making a database entry in an external application, sending an email, or triggering a deployment process, just to name a few. Actions you can perform in response to events on GitHub are limited only by your imagination.

When a subscribed event occurs on GitHub, GitHub triggers an HTTP POST request to the defined destination. This destination is specified as an HTTP URL, which is the endpoint to be called whenever the event occurs. This is known as the webhook URL. The endpoint handler at the webhook URL handles the action to be performed in response to the event.

Along with the request, GitHub also sends the payload of data about the event that triggered the webhook. Information contained in this payload can be used for the action on the endpoint.

Tutorial: How to set up GitHub Webhooks

To begin the tutorial, let's take a look at the steps involved:

Clone the sample Node.js API for receiving GitHub webhooks on your development machine
Generate a webhook URL using the Hookdeck CLI
Register for a webhook on GitHub
Receive and inspect GitHub webhooks locally
Make some commits and view logs

Clone a demo API server for receiving GitHub webhooks

As described earlier, GitHub webhooks need to be sent to an endpoint on your API.

If you already have a server running with a POST endpoint to receive the webhook request, you can skip this section. All you need to do is to take note of the port that your local API is running and also the endpoint to receive webhooks. Substitute these values as you follow this exercise.

If you do not have a local API, you can use a Node.js sample API available on the Hookdeck GitHub repository. Follow the instructions below to set it up.

Clone the repository by running the following command:

git clone --single-branch --branch github-webhooks https://github.com/hookdeck/nodejs-webhook-server-example.git

Navigate to the root of the project and install the required dependencies by running the following commands:

cd nodejs-webhook-server-example
npm install

When the installation completes, you can then run the Node.js server with the following command:

npm start

This will boot up the API application and print a message to the screen indicating that the API is now running and listening for connections on port 1337.

We are using two endpoints in this project.

/log-github-webhook: This is the endpoint that will be receiving the GitHub webhook and logging it into an in-memory database. It logs a simple object containing a subset of the information from the webhook payload.
/fetch-webhooks-logs: This endpoint can be called to retrieve a collection of the logged webhook data.

Get the webhook URL for GitHub

Endpoints on locally running servers cannot receive GitHub webhooks; a publicly accessible endpoint is required and this is where the Hookdeck CLI comes in. The Hookdeck CLI is built specifically for working with webhooks, and helps tunnel external HTTP requests into your local development environment by providing you with a publicly accessible HTTPS URL. You can also inspect the headers and payloads of your webhooks, as you will see later on in this tutorial.

Visit Hookdeck's CLI documentation to install and set up the tool on your operating system. For macOS users, you can run the following command to install the CLI tool:

brew install hookdeck/hookdeck/hookdeck

If you're using the Windows operating system, use the following command to install the CLI tool:

scoop bucket add hookdeck https://github.com/hookdeck/scoop-hookdeck-cli.git
scoop install hookdeck

hookdeck listen 1337

What source should you select? Ans: select Create new source
What should your new source label be? Ans: type the text Github
What path should the webhooks be forwarded to (i.e.: /webhooks)? Ans: type /log-github-webhook (If you're using your own custom server, replace this value with your endpoint)
What's the connection label (i.e.: My API)? Ans: type My Github Response Server

With this information, the CLI will begin the process of generating the URL and once it's done, you will see the URL printed to the screen and the CLI indicating that it is ready to receive requests.

Note: You will need to use the guest Login URL link in the console to access the dashboard. Copy and paste this into your browser to begin a guest login session.

Set up webhook on GitHub

Next, you will need to subscribe to a GitHub webhook. Navigate to your repository of choice (this should be a repository you own) and go to Settings → Webhooks. On the Webhooks page, click on the Add webhook button on the top right-hand corner.

On the webhook form displayed, paste the webhook URL generated by the Hookdeck CLI into the Payload URL field.

Fill the remaining fields as follows:

Content type: Select application/json so that you can receive the payload as a JSON object.
Secret: You can leave this blank, but for the purpose of this tutorial enter 1234ABCD.
SSL verification: Leave this as the default option of Enable SSL verification.
Which events would you like to trigger this webhook? This is the point where you subscribe for a GitHub event on your repository. For this tutorial, select the Just the push event option, as we are only subscribing to the push event. You can either subscribe for all events or a subset of the events using the other two options.
Active: Leave this checked to receive event details when the GitHub webhook is triggered.

See the complete webhook form below:

Click the Add webhook button to complete the process.

With this setup, any time you push code to your repository, a webhook request will be fired to the specified webhook URL.

Receive and inspect GitHub webhooks and payload

Here we can see that we have received a POST request with a 201 status code indicating that a new entity was created on our server; in other words, a webhook object was logged.

To view details of the webhook request, its headers, and payload, copy the Hookdeck dashboard URL printed along with your webhook request details on the CLI and load it in your browser.

You will see a page similar to the one below:

On this page, you can go through and inspect all the data that came with your webhook request. For example, the Headers section is expanded in the image below:

You can also view all that is contained in the webhook request payload by expanding the Body section as shown below:

With this setup, you get full and clear visibility into your webhook requests and you will be able to monitor and quickly troubleshoot your webhooks when any failure occurs.

View Application webhook logs

To check that the webhook information is being logged, make a few commits to your GitHub project. You should see webhook event entries in the terminal where your Hookdeck CLI session is running. Now visit the endpoint /fetch-webhooks-logs, where you should see a couple of log entries similar to the image below:

As seen above, we are logging the repo, author, and timestamp of the commit. Ignore the first entry as this is simply a test entry so that the application database does not start empty.

GitHub webhooks verification

A publicly accessible endpoint, just like the one being used as your webhook URL, can receive requests from anyone. This situation makes your endpoint vulnerable, as ill-intentioned individuals or bots can easily spoof requests imitating GitHub to dump irrelevant data into your system or manipulate your application.

You should verify the requests hitting your webhook URL to ascertain that they are from GitHub. GitHub helps you achieve this using the token secret we set in the form in an earlier step.

GitHub uses this token to create a hash signature with each payload. This hash signature is included with the headers of each request as X-Hub-Signature-256.

With this signature, you can validate your payloads. GitHub uses the HMAC algorithm to compute the hash and it is the same algorithm you will use to implement the validation on your server.

At the moment, our project is not enforcing this check but the code can be found commented out as shown below:

//Validate payload
/* function validatePayload(req, res, next) {

    if(req.method == "POST"){
        if (!req.rawBody) {
            return next('Request body empty')
        }

        const sig = Buffer.from(req.get(sigHeaderName) || '', 'utf8')
        const hmac = crypto.createHmac(sigHashAlg, secret)
        const digest = Buffer.from(sigHashAlg + '=' + hmac.update(req.rawBody).digest('hex'), 'utf8');

        if (sig.length !== digest.length || !crypto.timingSafeEqual(digest, sig)) {
            return next(`Request body digest (${digest}) did not match ${sigHeaderName} (${sig})`)
        }
    }

    return next()

}
app.use(validatePayload); */

From lines 10 to 12, you can see the validation variables defined, as shown below:

const sigHeaderName = 'X-Hub-Signature-256';
const sigHashAlg = 'sha256';
const secret = "1234ABCD";

The secret variable is the key we set when creating our webhook. In a production environment, these values should be kept in environment variables.

You can uncomment the payload validation middleware to ensure that your webhook payload is validated against the API secret set.

For more information on securing your GitHub webhooks, you can check out the GitHub documentation here.

How do I disable a GitHub Webhook?

At the moment, there is no setting on a GitHub webhook configuration that disables a webhook. Some will suggest that you uncheck the Active parameter but this only stops the delivery of the webhook payload, it may not prevent the event itself from being fired.

The only way you can be sure you have disabled a webhook is by deleting it completely from your GitHub Webhooks page. This may seem a bit extreme if you only want to disable it temporarily.

With Hookdeck, you can use the Pause feature on the dashboard to temporarily stop your webhooks from being delivered. When you're ready to start receiving them once again, you can unpause the webhook.

Conclusion

Webhooks are a fantastic offering by GitHub to help developers extend their integrations with the platform to more custom use cases. This enables the development of more robust applications that take advantage of events occurring on GitHub.

In this tutorial, you have learned how to set up and receive GitHub webhooks on your API endpoints. The tasks you can perform in response to these events are only limited by your imagination.

Happy coding!

Top 5 Use Cases for GitHub Webhooks

Fikayo Adepoju — Wed, 06 Oct 2021 20:28:09 +0000

Introduction

In our previous article, we looked at how to get started using GitHub webhooks.

However, what are the actual real-world applications of GitHub webhooks? In what ways can this technology deliver value to our applications? These are the questions we will be trying to provide answers to in this article by taking a look at some of the most common real-world applications of GitHub webhooks.

Why are GitHub webhooks important/useful?

Before we dive into the various use cases of GitHub webhooks, let's discuss why GitHub webhooks are a useful piece of technology. For over a decade now, GitHub has served as a central hub for storing, sharing, and collaborating on code-based projects.

It can be argued that GitHub houses the code for most of the software applications, frameworks, plugins, and any code-based technology that we use today. Thus, it is important that such a platform should open doors to integrations with other external systems that rely on the code hosted on it.

GitHub webhooks provides one such integration opportunity by allowing external applications to subscribe for events taking place on GitHub. These external applications can then receive notifications and information about their subscribed events, and trigger workflows in response to these events. This way, subscribed applications can deliver more value to their users (especially through the automation of tasks). The beauty of GitHub webhooks is that they are not restricted to any use case; you are only limited by your own imagination.

In the following sections, we will take a look at some of the most common scenarios in which application engineers have employed the use of GitHub webhooks to give you an idea of the flexibility that is possible.

Triggering continuous integration build pipelines

One of the most common applications of GitHub webhooks is the triggering of continuous integration pipelines to run tests on code being pushed to a branch. Team members working on a project send pull requests from feature branches to the main branch. Before these feature branches are merged to the main branch, it's considered best practice to run the code to be merged through tests.

You can set up a continuous integration (CI) pipeline on a CI server (Jenkins, CircleCI, etc.). Then subscribe for push and pull_request events on GitHub to fire webhooks to trigger your CI server to run tests on the branch.

This will prevent defective code from being merged to your main branch when errors are detected in the CI build and the build fails.

Automatically deploying applications to destination servers

For web projects, it is important for development teams to deliver their code to application servers where users and/or testers can access it. Whether you need to deliver your web app to a staging or production environment, a continuous delivery pipeline can be set up to move your code to the hosting environment.

However, this deployment pipeline needs a trigger to tell it when to ship your code to the host. Developers can simply subscribe for a push event on the main branch (for deployment to production servers), or any deployment branch. When the push event occurs, a webhook is sent to trigger the deployment pipeline to ship the code to the hosting environment.

CI/CD platforms like CircleCI have API endpoints to trigger a deployment pipeline. You can simply point your webhook to this endpoint to trigger a deployment to your servers.

Sending notifications to team collaboration channels like Slack and Discord

GitHub tries its best to send email notifications for commits and pull requests on repositories. However, development teams spend most of their time on collaboration platforms like Slack and Discord.

Thus, the most effective way to communicate activities happening on code repositories to team members, especially code reviewers and deployment approvers, is via these collaboration channels. This ensures that anyone who needs to take immediate action gets the notification on time. This can also help facilitate discussion on commits and issues, which allows the team to stay in sync with activities taking place on the repository.

Discord allows developers to generate a webhook endpoint URL that can be submitted to GitHub when subscribing to a webhook. This allows you to get instant notifications on subscribed events in your Discord channels. You can follow this GitHub gist to see how GitHub webhooks can be used to channel event details to Discord channels.

When subscribing for webhooks to be sent to discussion channels, you can subscribe to as many events as you're interested in. There are no rules as to what you should have sent to you.

Updating external issue trackers

GitHub includes an Issues tab on each repository where users of the application and code collaborators can submit issues observed. Discussion threads can then be created on each issue to share information about resolving the issue.

However, some external applications do a better job at handling issues than GitHub. These issue trackers often possess more features for dealing with code issues and helping developers collaborate better on a project.

Some of these issue trackers, like Jira, have built-in support for GitHub to follow up on issues on a repository. Others, like Pivotal Tracker and MongoDB Realm, provide you with a webhook URL that you can add to GitHub to subscribe to the issues events, issue comment event, commit comment event, and any other event related to issues tracking.

Automating workflows on external processes

Another interesting use of GitHub webhooks is in triggering workflows on external applications and processes. This use case is custom, and always dependent on what the application engineer wants to achieve.

It can be as simple as logging activities taking place on GitHub for audit purposes, or triggering database servers to backup data before a release is made. Project management applications can subscribe to events like issues and pull_request to keep track of the project.

Customer support applications can subscribe to the issues event to collate complaints, comments, and issues that users are reporting on GitHub.

If there is a process you need to automate in your custom application, database servers, SaaS applications, or email servers in response to any activity on GitHub, you can achieve this automation with GitHub webhooks.

Conclusion

The simplicity of GitHub makes it easy to integrate into a wide range of external applications and processes. Almost every application supports HTTP, and since webhook requests are simple HTTP requests carrying a payload of event information, they can easily communicate with a lot of online applications. In this article, we have looked at some of the common scenarios where GitHub webhooks are deployed.

GitHub webhooks use cases are not limited to what we have seen in this article, but having these examples at hand will help you better understand how to use GitHub webhooks to their full potential.

If you are interested in building an integration with GitHub webhooks, we recommend reading our tutorial for GitHub webhooks to kick start your project!

Happy coding!

Getting Started with GitHub Webhooks

Fikayo Adepoju — Wed, 29 Sep 2021 16:23:06 +0000

Introduction

GitHub is arguably the largest remote code hosting service in the world. It can also be argued that it is the most-used platform by developers around the world for working on both personal and team code-based projects. One of the attributes contributing to GitHub's widespread adoption is the ability to integrate the platform into development workflows, OAuth applications, CI/CD processes, and many more use cases.

In this article, we take a look at one of the strategies for integrating GitHub into external workflows and applications, known as GitHub webhooks.

What are GitHub webhooks?

Just as webhooks are being provided by many SaaS and PaaS systems for the purpose of integration, GitHub webhooks give developers the ability to integrate with the GitHub platform.

Developers can subscribe to events taking place on GitHub, for example Push events that occur when code is pushed to a repository, and respond to the event with an action.

The response action can be anything, such as making a database entry in an external application, sending an email, triggering a deployment process, and more. Actions you can perform in response to events on GitHub are only limited by your imagination.

When a subscribed event occurs on GitHub, GitHub triggers an HTTP POST request to the defined destination. This destination is specified as an HTTP URL, which is the endpoint that is to be called whenever the event occurs. This is known as the webhook URL. The endpoint handler at the webhook URL handles the action to be performed in response to the event.

GitHub also sends the payload of data about the event that triggered the webhook along with the request. Information contained in this payload can be used for action to be taken on the endpoint.

What are the differences betweem GitHub webhooks and actions ?

GitHub Actions is a built-in CI/CD tool for GitHub and allows developers to automate the testing and deployment of code. When new code is pushed to a GitHub repository, GitHub actions can be set up to test the code, merge it to the main repository when tests pass and also deploy the application to a destination server.

GitHub webhooks, on the other hand, are just plain subscriptions to events that take place on the repository and can be used to trigger different actions asides from CI/CD workflows.

You can use GitHub webhooks to achieve the same CI/CD operations found in GitHub Actions, however, GitHub is not responsible for the CI/CD operation. You will need to connect your CI/CD triggers to the necessary event in your repository with GitHub webhooks and use an external CI/CD server.

GitHub actions also require a configuration script to set up, whereas GitHub webhooks do not require a script.

How do I use webhooks on GitHub?

Webhooks can be configured on a GitHub organization account, a specific repository, or a GitHub App. A webhook is connected to a GitHub event type and you can configure webhooks for more than one event. Once set up, webhooks will be sent each time a subscribed event is triggered.

Here are the steps to set up a webhook on GitHub:

Navigate to the repository you want to subscribe to events on
Go to Settings → Webhooks
Click on Add webhook
Fill in the webhook form specifying your webhook URL
Subscribe to all, or only the events you're interested in
Click the Add webhook button to complete the process

See below an example of a webhook setup on GitHub.

Immediately after the webhook is created, GitHub sends a ping webhook to your endpoint. This is to test that your endpoint is live, and you can also use it to verify the expected reaction on the webhook URL.

You can create up to 20 webhooks for each organization, repository, or GitHub app you're interested in.

Webhooks can also be configured via the GitHub API through authenticated sessions.

Types of GitHub webhooks events

As mentioned earlier, each webhook is tied to an event that takes place on GitHub, so let's look at the types of events that GitHub supports. GitHub supports over a dozen event types but except in very peculiar cases, your interest will be focused on a few common ones.

Some of the common events supported by GitHub are:

Push Event: This event is triggered when one or more commits are pushed to a repository branch or tag. This is one of the most common events developers subscribe to, as a lot can be done in response to a Push Event. You can trigger tests in continuous integration pipelines, send alerts to code reviewers on your team to review the new commit, and even trigger deployments to staging and/or production servers.
Pull Request Event: This event is triggered for any activity that involves Pull Requests, for example when a Pull Request is opened, assigned, labeled, or closed, etc. The type of activity is specified in the action property of the webhook request payload.
Issues Event: Any activity related to issues causes this event to be triggered. Just like the Pull Request Event, the type of activity is specified in the action property of the payload. The issue itself is contained in the issue property. One common use case of this event is in updating an external issue tracker.
Create Event: This event is fired when a repository user creates a new branch or tag in the remote repository.

For a full list of events supported by GitHub, their description and payload properties, check out the documentation here. Familiarity with the supported events will help you get the best out of GitHub webhooks.

While subscribing for events on a repository, you might be tempted to just subscribe for every single event available. This is not advisable, as your GitHub webhooks rate limits can easily be exceeded. Your endpoint will also be burdened with a large number of irrelevant events.

Finding information on the GitHub webhooks documentation

GitHub has well-documented and exhaustive material on webhooks. However, sometimes you just need to get started quickly and you want to navigate to the information you need as fast as possible.

Below is a list of the most common pages you will be interested in:

Conclusion

GitHub webhooks further expand the range of integration that developers have with the platform. Also, they are easy to set up and highly reliable. With GitHub webhooks, you can subscribe to events that trigger workflows in your external applications, like your custom application servers, Zapier, G Suite, CI/CD servers, etc.

In this article, we dipped our toes in the water that is GitHub webhooks. In subsequent articles, we are going to be taking a closer look at using GitHub webhooks, securing them, and best practices you should be aware of.

Happy coding!