Forem: Rogier van den Berg

Building a Battery-Powered Pomodoro Timer with Deep Sleep

Rogier van den Berg — Sun, 04 Jan 2026 11:21:41 +0000

The Pomodoro Technique is simple but effective: 25 minutes of pure focus, followed by a 5-minute break. Back in 2013, I built my first custom timer to automate this. Now, over a decade later, it's time for a serious upgrade.

In this post, I'll show you how I converted my old bulky design into a modern, ultra-low-power version using an OLED display and Deep Sleep technology. I'm sharing the code for both the ESP32 and the energy-efficient Seeed XIAO nRF52840, including a custom 3D-printed case!

The History: Version 2013

My original project was built around a classic Arduino Uno and a 7-segment display. All put together in a 3D-printed box, held together with tape as printers then were not that good yet. While it looked cool, the wiring was a nightmare—I had to use shift registers (74HC595) to manage the pins, and power consumption was high, meaning it always had to be plugged in.

The code was functional but verbose, handling all the bit-shifting manually:

// The old way: Managing a 7-segment display through shift registers
byte chars[]={0xc0,0xf9,0xa4,0xb0,0x99,0x92,0x82,0xf8,0x80,0x90,0xff};
digitalWrite(latchPin, LOW);
shiftOut(dataPin, clockPin, MSBFIRST, chars[digit]);
digitalWrite(latchPin, HIGH);

It worked, but it wasn't elegant—and certainly not portable.

The Upgrade: Version 2026

The new version does exactly the same thing, but smarter.

What's Changed?

Display: A crisp I2C OLED screen (needs only 2 wires for data!).
Power: Battery support, and thanks to "Deep Sleep," the microcontroller turns off completely when the timer finishes. One press of the button wakes it up instantly.
Audio: A triumphant melody plays when your session is done.
Memory: The timer remembers your last setting, even after sleeping!

The Power Advantage

Here's the magic of Deep Sleep:

State	ESP32 Current	nRF52840 Current
Active (counting)	~40mA	~5mA
Deep Sleep	~10µA	~1µA

With the nRF52840, a small 500mAh LiPo battery could theoretically last for months in standby, waking up only when you press the button.

What You Need

Component	Description
Microcontroller	ESP32 (DevKitC) (accessible, pin headers, ready to prototype), Seeed Studio XIAO ESP32-C6 (cheap!), or Seeed XIAO nRF52840 (best for battery life)
Display	0.96 inch OLED Display (I2C SSD1306)
Button	Simple push button
Audio	Passive Piezo buzzer
Power	USB cable or a 3.7V LiPo battery (ideal for the XIAO)
Case	3D printed (files in the repository)

Wiring Guide

The wiring is straightforward. Here is the pinout for both board options:

Component	ESP32 Pin	XIAO nRF52840 Pin
OLED SDA	GPIO 33	D4
OLED SCL	GPIO 32	D5
Button	GPIO 25 (to GND)	D1 (to GND)
Buzzer (+)	GPIO 27	D2
Buzzer (-)	GND	GND

Both the button and buzzer connect to ground on the other side. The button uses the internal pull-up resistor, so no external resistor is needed. I soldered the battery directly to the board in the final product, as the XIAO supports battery charging.

The Case (3D Print)

I designed a compact housing to fit everything neatly. The design includes:

Slot for Seeed Studio XIAO nRF52840, with recessed USB port
Cutout for the OLED display
Button hole
Battery compartment

The board is clamped in place by a bracket and a pin from the top of the enclosure, the battery rests inside a compartment and the display hangs in a small gutter, with lips behind the screen coming from the top of the enclosure. Together with a snap fit this keeps everything neatly in place.

Download the STL files: Available in the enclosure/ folder of the GitHub repository.

The Code

I made two versions, first based on ESP32 for prototyping, later with the nRF52840 in the final product, to have a more power efficient solution.

Both versions share the same logic:

Wake up → Show last selected duration
Button press → Cycle through 5, 15, 25, 45 minutes
Wait 2 seconds → Auto-start countdown
Count down → Display remaining minutes
Finish → Play melody, show "KLAAR!" (Dutch for "Ready!" 🇳🇱), go to sleep

The key difference is how they sleep and remember settings.

Option A: ESP32 Deep Sleep

The ESP32 uses RTC_DATA_ATTR to store variables in RTC memory (survives deep sleep) and esp_deep_sleep_start() to enter low-power mode:

// Store setting in RTC memory - survives deep sleep!
RTC_DATA_ATTR int countdownSetting = 2;

void goToSleep() {
  display.ssd1306_command(SSD1306_DISPLAYOFF);
  esp_sleep_enable_ext0_wakeup(buttonPin, 0);  // Wake on button press
  esp_deep_sleep_start();
}

Option B: Seeed XIAO nRF52840

The nRF52840 uses System OFF mode for even lower power consumption. It stores settings in the GPREGRET register:

// Store setting in GPREGRET - survives System OFF!
NRF_POWER->GPREGRET = countdownSetting;

void goToSleep() {
  display.ssd1306_command(SSD1306_DISPLAYOFF);

  // Configure button to wake from System OFF
  nrf_gpio_cfg_sense_input(
    digitalPinToPinName(buttonPin),
    NRF_GPIO_PIN_PULLUP,
    NRF_GPIO_PIN_SENSE_LOW
  );

  NRF_POWER->SYSTEMOFF = 1;  // Enter System OFF mode
}

When the nRF52840 wakes up, it's essentially a reset—the code starts fresh from setup(). We check GPREGRET to restore the previous setting.

Getting Started

Clone the repository:

   git clone https://github.com/rogiervandenberg/pomodorotimer.git

Install the Arduino libraries (via Library Manager):

Adafruit GFX Library
Adafruit SSD1306

Install the board package:

ESP32: Add the ESP32 boards URL in Arduino IDE preferences
nRF52840: Follow the Seeed XIAO BLE guide

Upload the appropriate sketch to your board
Print the case and assemble everything

Lessons Learned

After 13 years, some things have changed dramatically:

I2C is supercool: Managing shift registers was fun in 2013, but modern I2C displays with clean libraries make life so much easier.
Deep Sleep changes everything: The ability to run on battery for months opens up entirely new use cases. My little clock is now fully portable!
One button is enough: Constraint breeds creativity. The single-button interface forces simplicity.

Conclusion

This project proves that sometimes the best gadget is the one that does exactly one thing well. No notifications, no apps, no distractions—just a timer and a button.

The full code is available on GitHub: rogiervandenberg/pomodorotimer
The 3D enclosure is available on Makerworld

Happy building and stay focused! 🍅

Revolutionize Your Unit Testing with Testcontainers and Docker

Rogier van den Berg — Fri, 31 May 2024 09:07:17 +0000

Recently, when working on a project I ran into a situation in which I needed to handle a complex object/data structure. This object had to be able to be saved into a database (dispersed over multiple different tables), retrieved, and reconstructed exactly as intended.

In order to verify things were built as intended, I created Unit Tests that would write and read this object into my database. But I did not want to use my development database for this.

Why? The traditional approach of using a development database for testing often leads to several issues: interference with development work, data corruption, inconsistent results, and overlapping tests affecting each other.

But what if there was a way to conduct these tests in a controlled, isolated, and repeatable environment, using a real database? 🤔

Enter Testcontainers!

Why Testcontainers?

Testcontainers is a very neat open source framework/project I just discovered. It enables developers to create unit tests using throwaway, lightweight instances of e.g. a database running in Docker containers.

This approach ensures that every test runs in a fresh, isolated environment, eliminating the problems associated with using a shared development database. No more conflicting tests, no more corrupt data, and consistent, reliable results every time you run your tests.

And it works for multiple languages, like Go, Python and Javascript and it supports many external dependencies, like Postgres (as I'm using in my case), MySQL, Git, several Cloud providers, Redis, Elastic and many more 🤯

Example

Look at the following code that I copied from the documentation of Testcontainers:

const { Client } = require("pg");
const { PostgreSqlContainer } = require("@testcontainers/postgresql");
const { createCustomerTable, createCustomer, getCustomers } = require("./customer-repository");

describe("Customer Repository", () => {
    jest.setTimeout(60000);

    let postgresContainer;
    let postgresClient;

    beforeAll(async () => {
        postgresContainer = await new PostgreSqlContainer().start();
        postgresClient = new Client({ connectionString: postgresContainer.getConnectionUri() });
        await postgresClient.connect();
        await createCustomerTable(postgresClient)
    });

    afterAll(async () => {
        await postgresClient.end();
        await postgresContainer.stop();
    });

    it("should create and return multiple customers", async () => {
        const customer1 = { id: 1, name: "John Doe" };
        const customer2 = { id: 2, name: "Jane Doe" };

        await createCustomer(postgresClient, customer1);
        await createCustomer(postgresClient, customer2);

        const customers = await getCustomers(postgresClient);
        expect(customers).toEqual([customer1, customer2]);
    });
});

This code is a test that tests a customer repository and does a couple of actions against a real database. The amazing thing is that this test is not mocking anything, but always runs with fresh and new real database in a Docker container! So no more mocks or conflicting tests and a clean database every time you run your tests.

Try it out yourself

I think Testcontainers make running specific test cases easier, while having optimal isolation, consistency (and thus reliability) and simplicity (and thus better to maintain and manage).

So, check out → https://testcontainers.com/

The location of iOS simulator builds has moved

Rogier van den Berg — Mon, 26 Sep 2022 18:29:00 +0000

TL;DR

iOS Simulator builds are not somewhere underneath ~/Library/Developer/Xcode/DerivedData anymore, but in ~/Library/Developer/CoreSimulator/Devices.

Simulator build for e.g. Meta/Facebook App review

If you build an iOS or Android app that integrates with the Facebook SDK, you need to have your app reviewed before being able to go live.

As part of this App Review process voor iOS, you have to submit a simulator binary package of your app to Facebook.

Part of the instruction is to run the app in your simulator, and then look into ~/Library/Developer/Xcode/DerivedData folder.

But, since XCode 14 this does not work anymore, and up until now the Facebook documentation is not up to date yet. 😱 So the whole day I've been looking for "Where does XCode (or Flutter command line in my case) drop my compiled app, that is run on my Simulator?".

As it occurs, that is ~/Library/Developer/CoreSimulator/Devices/<<hash of your device>>/data/Containers/Bundle/Application/<<hash of your app>>/<<name of target>>.app 🎉

Bonus; one-liner to get a Zipfile of the simulator build

How to get a zipfile of your simulator build, that you can send to Facebook right away?

Run your app in the emulator
Execute the following line in your shell:

ditto -ck --sequesterRsrc --keepParent `ls -1 -d -t ~/Library/Developer/CoreSimulator/Devices/*/data/Containers/Bundle/Application/*/*.app | head -n 1` ~/Desktop/your_app_$(date +"%Y%m%d_%H%M").zip

This will take the most recent app from underneath your CoreSimulator folder, puts it in the most small and optimized archive for Facebook and drops it at your desktop, while having a timestamp. Feel free to change this one-liner to your liking 😉

When having the zipfile on your desktop, it's smart to check if it's working before submitting to Facebook.

To test the Zipfile:

Extract the zip
Install ios-sim if not done yet: npm install -g ios-sim
Test your file with: ios-sim launch --devicetypeid com.apple.CoreSimulator.SimDeviceType.iPhone-14 ~/Desktop/Runner.app

Good luck! 👋

How to explain the delight of Open Source software contributions to a non-developer?

Rogier van den Berg — Thu, 17 Mar 2022 13:45:59 +0000

Being happy that my contribution to the Request Logger middleware of Go’s chi router got accepted and merged 🎉 I found out that it was difficult to explain the 'coolness' of this to my non-dev colleagues.

So I explained it as follows:

At our company we use open source software components to build our own products (think of them as pre made ingredients when cooking). But when doing so sometimes you run into something that is not yet possible/existing in one of these ingredients. 🤷‍♂️

How cool is it that you then can (instead of waiting or searching for an alternative) fix it yourself (make your spice ingredient a bit spicier and with multiple colors for instance 🌶 🌈) ánd give your additions back to the community to be available to the rest of the world? 🌍

That just happened, I change a piece of logging software to fix something I ran into myself, and gave it back! 🎉 #humblebrag https://github.com/go-chi/httplog/pull/8

It’s always a great feeling improving some software you use yourself and giving it back for others to use too!

Easy Google Cloud Logging from your Golang project in Google Cloud Run

Rogier van den Berg — Wed, 16 Mar 2022 09:37:02 +0000

In my journey of learning Go, for almost a day I have been searching and thinking in the wrong direction, and I want to prevent you from doing the same 😅

I am building a HTTP Service running in Google Cloud Run in Go and wanted an easy way to log stuff to Google Cloud Logging.

Cloud Run Logs

Cloud Run has two types of logs, and these are automatically sent to Cloud Logging:

Request logs: logs of requests sent to Cloud Run services. These logs are created automatically.
Container logs: logs emitted from the container instances, typically from your own code, when writing to stdout or stderr

Loggers

If you start searching on how to do logging, you will find many possibilities, this makes it very difficult to decide and/or even to know which one to use.

Even worse, because I'm making a HTTP service, I was searching in the direction of creating middleware for my chi router. 🤦‍♂️

KISS

But then it hit me: As Cloud Run already automatically takes care of the HTTP request logging, I only needed to log certain important 'events'. Thus, I could just write to os.stdout in the correct format.

And even better, there is a very easy to use log package for that:, having great benefits like no dependencies, uses ' severity ' to support the Cloud Logging filters.

The only implementation is

package main

import "github.com/apsystole/log"

func main() {
    log.Print("my message")
    // {"message":"my message","severity":"INFO"}
}

log.Error("my message")
 // {"message":"my message","severity":"ERROR"}

Bottom line: keep it simple, stupid ;)

Why I'm writing on dev.to from now on

Rogier van den Berg — Tue, 15 Mar 2022 09:39:37 +0000

As most developers having an own self made website, every time you think of blogging or technical writing, you end up creating a new website or optimization for yourself first. 😅 So did I originally; I was tweaking my website more than doing actual writing 🙄

Until at some point I decided to take a third party blogging tool, that focuses on writing and takes care of "the rest", preferably having the least amount of friction between an idea and a published post.

So I looked for a hosted solution...

dev.to / DEV

Enter DEV. It is an amazing platform to just start writing. To see if it would fit me, I wrote a couple of things here. The platform has a very simple layout, writing is quick and effortless and it takes no time to get your idea out there.

Furthermore it's free, it gives good SEO ranking and allows you to connect your privately owned site too with canonical URLs.

Finally, the organization behind DEV are sympathetic. They promote and facilitate many initiatives (e.g. International Women's Day, minorities in tech), seem very inclusive, open sourced their tech and have a merchandise shop!

Hashnode

But then, there is also Hashnode.

Some people on Twitter nudged me over to look at that. It has cool things like your own website (with custom pages!) on a custom domain. All for free too! So I created an account, created my site and started cross posting my blogs. They even have import tools to get started quickly with already written content.

But some things are not so good. For instance the overall navigation and "getting around": The difference between your profile (with a feed, about section, etc) and your website/blog for example. What to put where? It seems you have your "profile" and settings on the Hashnode platform, besides your website/blog that also has a profile with settings. Things are clunky and confusing.

Another thing I did not like was the speed. Every click takes about 3 seconds before the page is ready loading. It just is not snappy.

But, in the end you get your free custom domain, free website with custom pages, etc. etc. so I did not complain. Even better, if I want to in the future, I can easily use their Newsletter feature to add that to my Hashnode site too!

So I even almost decided to go for Hashnode as my default platform.

Who's reading?

But then, today, I saw some Analytics of exact same posts on both dev.to and Hashnode. dev.to is found and read way more than Hashnode. Sometimes almost 100 times better!

For instance:

A small piece about how to deploy Deno on Cloud run: 484 views on Dev.to, 1 on Hashnode.
When Rails 6 was new, something about creating a new project with Docker: 2886 views, 12 hearts and 3 comments on Dev.to. 52 views and no responses on Hashnode 🤷‍♂️
I made this boilerplate for a new Next.js project with Bulma and Typescript and wrote about it. Dev.to got 2031 views and 4 reactions. Hashnode just 21 views.

Conclusion

So, if I want to write something, not only for myself, but also for the world to be able to take an advantage of it, I'd better use dev.to.

And it gives me added benefits as well:

Loading of pages is nice and quick (as it should be)
There is one place to manage and discover content (not a separate own site, etc.)
No more brain clutter / decision fatigue with thinking about what customizations to make on my website (because that does not matter anymore)
Just posting on ONE spot → https://dev.to/rogiervandenberg

Am I missing things on DEV?

Well yes, it would still be nice to point my custom domain to my dev.to profile. Furthermore, an automated backup of my content to e.g. a github repo (as Hashnode has) would be great.

Happy writing! 👋

Run Postgres from Docker for easy development

Rogier van den Berg — Wed, 16 Feb 2022 08:18:10 +0000

Postgres is a great database server that you can install locally on your computer for development. However, there are various ways in which you can install PostgreSQL on your machine, e.g. by using the the installer, homebrew or postgres.app. But managing your local Postgres is a real pain. "What was the command again to stop/start the service?", "Do you need to have it always be running in the background (even when you are not working)?", "How to handle multiple versions next to each other?", "How to handle updates?"

→ What if I tell you, you can run Postgres in one command, in any version WITHOUT having to install it in your machine? WITH the benefit of persisting data? That would be a total game changer! 😃

Docker

Docker allows you to put software in a container and run it with a single command. You could make your own image that packs an operating system, your settings/preferences and your application into a container for easy deployment, but, you can also use a pre-build image to run things like a database engine!

Run a local Postgres with Docker

Let's say I need to have Postgres in the specific (old) version 9.6 and the databases should be persisted outside the container on my local computer.

Install Docker

First, make sure you have Docker installed on your computer: https://docs.docker.com/get-docker/

Create a local directory to persist the data in

In my example, I'm going to write the database to my /tmp folder, but this can be anywhere.

mkdir /tmp/database

A better place would be a subfolder in your project, or in your home-dir.

Start Postgres with Docker

docker run --rm -it \
-p 5432:5432 \
-e POSTGRES_PASSWORD=password \
-v /tmp/database:/var/lib/postgresql/data:delegated  \
postgres:9.6

Let me break down what this does:

Run a container and clean up afterwards and allow the process to be interactive (so you can cancel it when you're done)
Allow connections to the Postgres inside the container on port 5432 from the outside of the container on port 5432
Set the default password for the postgres user to password
Sync all stored data of Postgres inside the container into our local folder /tmp/database (make sure this folder exists!). "Delegated" means that the container’s view of the file system will be authoritative, as we won't be changing the data from outside the container, so the database will be quick.
Use the Postgres 9.6 image

Connect!

Now, you can connect to localhost on port 5432, with default username postgres, password password and default database postgres.

A great and easy way to connect to your database is with the TablePlus app (for Mac, Windows, Linux and more!):

Stopping

If you're done working, just CTRL-C out of your database. The container will be cleaned up and Postgres is gone again. But, your data is still save on your own computer.

Next time you execute the above docker run command, Postgres will be fired up again; Really fast if it is still cached on your computer, otherwise quite fast by downloading the image again:

Next.js with Bulma in Typescript boilerplate

Rogier van den Berg — Tue, 03 Nov 2020 13:18:10 +0000

Recently I (re-)discovered Next.js, and I love it. And for making styles, I really like Bulma. And I also prefer to write my Javascript code in Typescript.

So, instead of needing to reinvent the wheel for every (toy) project, I decided to make myself Typescript boiler plate for Next.js with Bulma using Typescript to quickly kick off a new project.

Add syntax highlighting to 'cat' in your terminal

Rogier van den Berg — Tue, 30 Jun 2020 09:44:41 +0000

I use my terminal everyday, and frequently I am way faster with it then navigating around with my mouse in Finder.

Within your terminal the cat command is very useful to quickly peek inside a file, but when you're looking at source code, this is a bit difficult to read. I just thought "Wouldn't there be syntax highlighting for my terminal?". As it turns out, there is. 🎉

How to set it up

With Brew, install Pygments: brew install pygments
Next, add the following to your .zshrc/.bashrc file: alias cat="pygmentize -g"
Open a new terminal window, or source your .zshrc file: source ~/.zshrc

Now, when you e.g. run cat Dockerfile on a Dockerfile, things look very neat!

Create new Rails 6 project with Docker

Rogier van den Berg — Thu, 11 Jun 2020 15:59:13 +0000

If you want to set-up a new Rails project, but you don't want to go through the hassle of setting op Ruby, node, Yarn, Rails itself, etc. you could do it with Docker. That way you can really fast start a new project, without possible version conflicts, installs, etc. on your computer...

Let's go! 🚀

Prerequisite: Have Docker installed

If not, install Docker first

Create the directory for your project

Open a terminal, create a project folder somewhere and go into that directory:

mkdir my_rails_project ; cd my_rails_project

Setup your project

First, you need to start a container to open your folder with, to install Rails.
Run your container:

docker run --rm -v ${PWD}:/usr/src -w /usr/src -ti starefossen/ruby-node /bin/bash

Within the container, install Rails first:

gem install rails

After installing Rails, create you project. Because your container working directory is bound to you project folder, the Rails project will be created there:

rails new .

(mind the . at the end! This will install the project in the current directory)

After creating the project, close your container with exit. Your container will be closed and removed. And you'll end up with a shiny new Rails install, without having Rails, Ruby, etc. on your computer.

Create a Dockerfile

Next, to Run your project, create a Dockerfile file in the root of your project, and paste the following (or whatever setup you'd prefer) in that file:

FROM starefossen/ruby-node

# Set the workdir inside the container
WORKDIR /usr/src/app

# Set the gemfile and install
COPY Gemfile* ./
RUN bundle install

# Copy the main application.
COPY . ./

# Expose port 3000 to the Docker host, so we can access it
# from the outside.
EXPOSE 3000

# The main command to run when the container starts. Also
# tell the Rails dev server to bind to all interfaces by
# default.
CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]

Build you container

Build a shiny new Docker container, based on a Ruby + Node Docker Image, having your project in there, executed on launch:

docker build -t my_project .

Run your container

Execute:

docker run \
-p 3000:3000 \
-v $(pwd):/usr/src/app/ \
my_project

This runs your built container, forwarding port 3000 and connects the current folder (your project folder) to the one inside the container (so you can edit files while it is running).

Check it out

Open your browser en go to http://localhost:3000 Yay! You’re on Rails! 🚂

Log in to your container

If you want to go inside your container (to create controllers, views, models, etc.) run:

docker exec -it $(docker ps -q) /bin/bash

This will execute an interactive (so you can actually use it) /bin/bash inside the running container.

That's all for now! 👋

Image credit: Jason White

Deno in Docker on Google Cloud Run

Rogier van den Berg — Mon, 18 May 2020 18:19:50 +0000

This weekend I read a lot about Deno, "A secure runtime for JavaScript and TypeScript." It's built on V8 and Rust, by Ryan Dahl, the creator of Node.js and it would basically 'fix' many things that are 'wrong' about Node.js and would be a "fun and productive scripting environment".

It intrigued me, so I took a first look 😊

Playing around

After some playing and reading (this is an amazing source!) I figured it would be nice to have an easy starting point for a Deno project running inside Docker and also on Google Cloud Run.

Deno in Docker (on Cloud Run)

I wanted to make a concrete example of Deno, running on Cloud Run. So I build just that. A "Hello world" of Deno, with a Dockerfile (that pre fetches all dependencies) to easily run it anywhere + deployment to Cloud Run.

You can see the result here: https://github.com/rogiervandenberg/deno-docker-cloudrun Of course with an easy single click deployment button 🎉

Google Task queues on GCP with Google Cloud Functions

Rogier van den Berg — Fri, 21 Feb 2020 15:53:05 +0000

At our company Embrosa we have many microservices, that all handle one specific part of our platform. An example is that when a user profile is changed, some of this data is synchronized to other services, by using a Firestore triggered Cloud Function.

This looks like this:

We needed queueing

But, we found that in some situations (e.g. when bulk updating user profiles) the logic for syncing to our backend and 3rd party API's were causing errors (rate errors, too many connections, etc.).

When we would mass-update 100 profiles, 100 cloud functions would try to do their magic. And sometimes, the receiving
Service could not keep up. So we ended up with out-of-sync data as there was no retry (Cloud Functions are triggered once (in general)).

Therefore, we wanted to implement a queueing mechanism, that does 3 things:

Quickly offload the tasks to be executed, so triggered functions can be done quickly
Rate limit calling external services
Ensure that the other services are successfully called, or retried when failing.

Google Cloud Tasks

Luckily we ran into Google Cloud Tasks. With that you can execute, dispatch and deliver tasks asynchronously, while having rate and retry controls among other things. And since last year Cloud Tasks supports Creating HTTP Target tasks 🎉

Creating tasks is well documented, but what I missed in the documentation was:

How to send data from one (Typescript) Cloud function to receive in another
How to make authenticated requests, to keep calling my Cloud Functions secure

So, in this post I will guide you, step by step, how to setup a Task queue, a way to send JSON from one Cloud Function to another and how to make calling Cloud functions secure with authentication.

It will look like this:

I assume that you already know how to work with Google Cloud Functions or FireBase Cloud Functions (it is almost the same in this example).

Setup the queue

Make sure you have Cloud SDK and that you are logged in to your project.

Run the following to set up your queue:

gcloud tasks queues create my-queue

It will ask you to enable the Cloud Tasks API (Yes, we want that) and if you don't have any app engine or regions set up in your project, it will ask you to do so.

This is necessary as your Task queue is based on app engine settings.

If you want to, you can make some adjustments to how your queue behaves:

gcloud tasks queues update my-queue --max-concurrent-dispatches=5 --max-attempts=3 --min-backoff=5s

Check how your queue looks:

gcloud tasks queues describe my-queue

The 'enqueueing/sending function'

The function that creates the tasks can be normal Cloud Functions, triggered by anything, like GCS, HTTP, Firestore, PubSub triggers, etc.

For Typescript/Javascript you need to make sure you are using the Cloud Tasks Node.js Client: @google-cloud/tasks. On the NPM page there is an explanation on how to use the library but there are two important things to add to make authenticated calls:

OIDC token

GCP Services like Cloud Functions, etc. can automatically validate the inbound OIDC token. This allows for authentication and authorization through IAM policies for the user or service account associated with that token. We use this to securely trigger our receiving function over HTTPS without allowing it to be public.

When setting up the Cloud Function that will be triggered by the task queue, check for the Service account that it uses. Use that email address as the serviceAccountEmail to specify in setting the task on the queue. This is most probably 'PROJECT_ID@appspot.gserviceaccount.com'.

Content Type JSON

This seems logical once knowing, but if you omit to specify the correct headers, the receiving cloud function cannot use all data you send.

Example

The function setting data on the queue could be looking similar to this:

const { v2beta3 } = require("@google-cloud/tasks");
  const client = new v2beta3.CloudTasksClient();

  const serviceAccountEmail = "PROJECT_ID@appspot.gserviceaccount.com";
  const project = "PROJECT_ID";
  const queue = "my-queue";
  const location = "europe-west1";
  const url =
    "https://europe-west1-PROJECT_ID.cloudfunctions.net/requestQueuedUpdateBackend";

  const payload = {
    foo: "bar",
    key: "value"
  };

  const formattedParent = client.queuePath(project, location, queue);

  const task = {
    httpRequest: {
      httpMethod: "POST",
      url: url,
      body: Buffer.from(JSON.stringify(payload)).toString("base64"),
      headers: {
        "Content-Type": "application/json"
      },
      oidcToken: {
        serviceAccountEmail
      }
    }
  };
  console.log("Sending task:");
  console.log(task);
  const request = {
    parent: formattedParent,
    task: task
  };

  const [response] = await client.createTask(request);
  console.log(`Created task ${response.name}`);
}

Change PROJECT_ID into your project ID and take note of the OIDC token and the headers for setting the request to JSON.

If you execute a function with the above snippet, it will enqueue a task on 'my-queue'.

The 'triggered function'

The function that is triggered by the Task Queue will do the actual work. This function will be triggered over HTTP, and therefore must not allow unauthenticated invocations (to prevent abuse).

Furthermore, you want to be able to receive data from the sending function.

The code could be as simple as:

export async function myFunction(req: Request, res: Response) {

  const payload = req.body;
  console.log("Received: ", payload);

  res.status(204).end();
}

Because the incoming data is JSON, we have the original payload object again. So you can make use of payload.foo for instance (= "bar").

When this function is deployed, take a look at its details (click on the link in of your Cloud Function in the https://console.cloud.google.com/functions/ overview. Check if the Service account is indeed 'PROJECT_ID@appspot.gserviceaccount.com'.

Keeping it safe

If you look at the IAM Permissions of your project you can see there is a member with the name "Cloud Tasks Service Account" and the role "Cloud Tasks Service Agent". We only want this member to be allowed to call our Cloud Function. Its email address will be service-PROJECTNUMBER@gcp-sa-cloudtasks.iam.gserviceaccount.com → remember / copy that address.

In the overview of Cloud Functions, look for your cloud function that is triggered by the Task queue, check its checkbox and look at permissions.

In the block of "Cloud Functions Invoker" make sure there is NOT a member "AllUsers" (publicly accessible). Furthermore, add a member to your Cloud Function Permissions ('service-PROJECTNUMBER@gcp-sa-cloudtasks.iam.gserviceaccount.com') and give it the role "Cloud Functions Invoker" ("Ability to invoke HTTP functions with restricted access.").

That's it!

You now have set up 3 things:

A queue that dispatches tasks (receives tasks and calls Google Cloud Functions) with the help of Google Cloud Tasks.
A Cloud Function that enqueues a task onto that queue (the enqueueing/sending function)
A Cloud Function that receives the payload over HTTP from the queue, in a secure way.

At Embrosa we are using this in the following way:

And now, the "other services" are not breaking anymore when we have suddenly a surge in the amount of things done on the left side of the diagram, because Cloud Tasks is taking care of limiting the amount of calls and retrying if something might fail.

This post was published originally on my website