Forem: Paweł Lewtak

Improve communication and transparency with Architecture Decision Records (ADR)

Paweł Lewtak — Fri, 12 May 2023 12:29:55 +0000

Architecture Decision Record, or ADR for short, is a tool that will improve communication, increase transparency in your engineering department, foster collaboration, and make future maintenance easier. We at Supermetrics are using ADRs and have found these benefits for ourselves.

Having up-to-date and useful documentation is a constant struggle. It’s hard for software engineers to think about writing documentation when writing actual code is often more interesting. It’s even harder to keep that documentation up to date if there’s a production incident or a tight deadline for delivering a project. As a result, documentation might not exist or not be as useful as it could be. That’s where ADRs can be useful.

What is an Architecture Decision Record?

Simply put, an ADR is the sweet spot between having extensive, deep-dive documentation and no documentation at all. Think of it as minimal documentation that’s always up to date with just a little effort.

More specifically, an Architecture Decision Record is a document that captures the important decisions made during the development of any project. It’s about writing down your thoughts, the context around the changes you’re making, your assumptions at the time of writing, expected consequences, and the changes themselves. Michael Nygard coined the name ADR and created a handy template for creating them.

A template for ADRs

The original template, as suggested by Nygard, looks like this:

Title - May include a sequential number and/or date.
Status - The status of ADR, such as proposed, accepted, rejected, deprecated, or superseded.
Context - An explanation of why the change is needed.
Decision - A summary of the actual change being made.
Consequences - The anticipated positive and negative consequences of the change.

At Supermetrics, we’ve decided to extend the template with a few more items:

Impact - Will the change have a low/medium/high impact on the code base?
Contributors - Who contributes to preparing, reviewing, and approving the ADR?
Options considered - A summary of what other options were considered, if any.
Outcome - What do we expect the end result to be when this decision is fully implemented?
Metrics - How will we measure the success of this decision?
Roll-out strategy - What’s the plan for releasing this change to users?

Each development team is welcome to customize the template if they find it limiting. When using an ADR template, customize it to fulfill your needs and cover your use case.

When to write an ADR?

If you made a code change but didn't document and communicate it properly, you can hardly expect your teammates to know about it or understand it. As short-form documentation, an ADR makes getting everyone on the same page easy. It creates a common understanding of new changes before they happen.

So, prepare an ADR before you make a change that impacts another part of the system or how code is written. As a bonus, it'll help reduce the number of different solutions for the same problem, which often result from undocumented changes.

Nothing stops you from going back to old changes and writing ADRs about them. It’s even recommended, especially when different developers repeatedly ask the same questions about a part of your system. Next time, you can refer them to the ADR.

Where and how to store ADRs?

The original idea by Nygard suggests you should use Markdown for easier formatting and reading and a git repository for storing the .md files themselves— ideally in the same repository as the code. At Supermetrics, we decided to keep ADRs in our wiki Confluence because we have multiple products and many repositories. It’s easier to find and read ADRs if they’re all in the same place, especially if individual ADRs affect multiple products.

Benefit #1: easier documentation maintenance

As you can see from the template, the final ADR isn’t going to be a very long document. Depending on your template, it could be from one to a few pages of text. Nor should it be highly abstract or require deep-dive knowledge to decipher. The ADR is supposed to be a simple document that gives concrete, practical information about how a change has been thought out and executed. It’s not about something abstract, or about explaining some architecture that requires deep dive knowledge. Writing down the context, decision, and consequences is straightforward enough since a developer has to consider them anyway when making changes to code.

An ADR explains the thinking behind a change at the time when the change was made. Once approved, its contents don't change much. Later, if you find out that you made a poor decision, or if the requirements of the decision (the context) change, you'll prepare a new ADR and deprecate or supersede the older one, changing only its status. The ADR releases you from maintaining old documentation. If something changes in the code, you’ll simply create a new ADR for the alterations.

You can compare an ADR to a blockchain, where only new pieces are added, and you need to go through the whole chain of records to understand the current state of the system.

Benefit #2 - Insight into the past

Understanding the why behind past software design decisions allows you to better understand their outcome. ADRs can help you with that. Traditional documentation typically explains how different parts of the software are connected and how they communicate. What it fails to explain is why something was done a certain way.

An ADR is a kind of a brain dump, covering the context of a change and its potential effects, and considering alternative options. So, when a new person joins the team and asks, "Did you consider X?" or "Why was it done this way?" they can go and read the ADR.

If you use an ADR throughout your projects for all meaningful changes, you’ll notice that it becomes a changelog and a single source of truth for your software. There’ll be less speculation about past changes and less knowledge lost between silos. ADRs also make code ownership a real thing, making project handovers and learning from past mistakes easier.

Benefit #3 - improved transparency, collaboration, and communication

The ADR template we use at Supermetrics includes a list of collaborators, which ensures that any planned changes are discussed between at least a few people, enforcing communication and collaboration. Teammates may have differing opinions about how to solve a problem, but collaborating on the ADR makes it easier to find common ground and choose the best solution. It also helps us see more of the potential consequences of implementing a change.

Listing contributors also makes it easy for developers to discuss past changes and their results with change authors and plan future changes that would supersede the old ones. Not knowing why something was done in the past makes future changes in the same area very risky, so that context should be written down in an ADR and made easily accessible to all developers.

ADRs make communication easier by improving transparency and collaboration within and between development teams. To make it even better, follow these steps:

Use simple, easily understandable language in your ADRs.
Skip details on technical implementation — those aren’t meant to be part of an ADR.
Limit ADRs to a single decision or change.

No documentation is perfect

We think ADR is a good-enough form of documentation for architecture and software changes. While it’s not a silver bullet by any means, it’s simple to prepare and has great benefits. ADRs allow us to make decisions widely understandable within the Engineering team and course-correct later if needed while keeping a paper trail of all the changes.
These easy-to-read documents make it easier to onboard new developers to the inner workings of the system architecture and why it was built in a certain way.

Your turn

Give ADRs a try, and let us know what you think. Were they helpful? What information did you find essential to include in your ADR template? We’d love to learn from your experiences!

Learn more about how the Engineering team at Supermetrics works at supermetrics.com/careers/engineering.

How to organize an internal bugathon, clean up your backlog, and help charities in 8 steps

Paweł Lewtak — Fri, 28 Apr 2023 07:40:17 +0000

Bugathons can be a fun and collaborative way to clear up that backlog of issues you've been putting off. In this article, we'll share the lessons we learned from hosting a bugathon at Supermetrics and how we turned it into an opportunity to donate to charity.

Step 1: Identify your goals

At Supermetrics, we have three main reasons for hosting bugathons: We wanted to give our distributed team the opportunity to work closely with each other. We also wanted to spring-clean our backlog of issues in a fun way. Finally, we wanted to donate to charity, and the bugathon was the perfect excuse.

Step 2: Gather your team

First, decide who will take part. Maybe you'll involve the entire engineering department, and maybe it's just for one development team. Whichever the case, we recommend blocking off a day (or a few hours at least) in the participants' calendars for the bugathon so they can focus on it fully.

Bugathons are the best fun when you do them with the whole team in person. The Supermetrics Engineering team is distributed in various cities globally, but for our latest bugathon, we brought everyone together at our HQ in Helsinki. It was a great opportunity for everyone, especially our fully remote team members, to practice pair programming and work together face-to-face.

Of course, you can also run successful bugathons fully remotely. That's what we did during the pandemic — it was a joyful experience, too.

Step 3: Make it feel like a fun competition

We started our bugathon with a kick-off session where we briefly explained the objectives and rules of the event. Teams then picked a mascot and came up with a team name — both important elements in creating team spirit.

Before the teams spread around the office to squash their bugs, everyone joined the bugathon Slack channel to keep tabs on the competition and banter with rivaling teams. We've also found that having a screen displaying our progress toward the bugathon's goals creates a shared feeling of accomplishment. Including a timer on the screen can also boost the sense of urgency for teams to complete their tasks.

Step 4: Assign points to each story

If you make a competition out of it, you should have a system for grading the bug-fixers' accomplishments. In our system, each story was labeled with a specific color — bronze, silver, gold, or platinum — representing a different number of points. More importantly, each label represented a specific amount of money we'd donate to charity at the end of the day — but more about that in Step 8.

Step 5: Maintain quality standards

While the goal of a Bugathon is to fix as many bugs as possible, it shouldn't mean lowering your standards for quality. In our events, each change goes through a proper code review and automated tests, as with any other code we write. And if a story turned out not to be a bug after all, its points were still counted for the team. After all, figuring out a bug isn't a bug, or figuring out a game plan for resolving a bug is almost as important as fixing the bug itself.

Step 6: Vary the day with extra activities

Hunting for bugs can be fun, but doing it for hours can get mind-numbing. To keep things fun and engaging, we had some extra activities throughout the day. The team had a chance to bond over a catered and take a fun trivia quiz with questions about the company, its culture, and traditions. There was a chance to earn extra points for secret achievements. Some of those achievements were:

The first bug of the day.
More tests than actual code.
The team with most stories closed.
The best team work.
The least new code added.
The best photo taken with the team mascot.

We also like to have talks from experts inside and outside our team during our bugathons. In the past, we've had talks from various experts — once, we had Google coming to talk about Pub/sub — and this time, two of our own developers gave talks. One was on debugging, the other on the lessons learned by building a new software product in 70 days.

Step 7: Celebrate success

At the end of the bugathon, you should gather together, count the points, crown the winners, and celebrate the shared achievements. In our latest Bugathon, we identified and resolved nearly a hundred bugs. It was a fantastic achievement with a positive impact on our products and our customers

Step 8: Donate to charity

Last but not least, we encourage you to use bugathons as opportunities to donate to charity. You can assign the stories a monetary value based on the number of points and donate the pot to a charity of your choice. In our latest bugathon, we donated 8000 EUR to earthquake relief in Turkey and humanitarian aid in Ukraine. Not only did we have a great day of collaboration and fixing bugs, but we also positively impacted the world around us.

We believe organizing bugathons is a great way to bring your development team together, clear up the backlog, and give back to the community through charitable donations. We hope these steps help you run your own bugathon, and you'll see the same positive impacts as we have.

Learn more about dev life at Supermetrics: supermetrics.com/careers/engineering

Benefits of a Developer Experience (DevEx) team

Paweł Lewtak — Thu, 23 Feb 2023 13:41:26 +0000

Developer Experience (DevEx) is a concept similar to User Experience (UX). It focuses on the developer's perspective of how easy, efficient, and enjoyable it's to work with your product.

DevEx can be external or internal:

External DevEx focuses on a developer who uses your product as a consumer, by, for example, integrating with your API.
Internal DevEx, which this blog post focuses on, strives to create the smoothest possible experience for the engineers who develop your product.

Internal developer experience impacts the productivity of the Engineering team and how satisfied and engaged developers feel at work. It can encompass various elements, from the more technical, like clean code, good documentation, and efficient internal tooling, to more intangible things, like your engineering culture and the level of psychological safety in your team.

How investing in DevEx has helped us at Supermetrics

At Supermetrics, our DevEx team focuses on the technical side of things — it develops and maintains tooling and processes that make our developers' work easier and increase their productivity.

Here are some of our DevEx team’s recently closed cases to give you an idea of how they improve our tooling and processes:

Upgraded dependencies in shared repositories.
Researched and compared code quality tools to be used across teams.
Upgraded Docker container images.
Improved error handling in smtool.
Improved the performance of our package managers.
Drove the PHP version upgrade.
Added checks for required software’s versions and warned teams if it wasn’t up to date.
Shared settings for PhpStorm (configured debugger and other external tools).
Automated database migrations.

When the DevEx team owns our shared tools and scripts, developers can focus on their projects without worrying about who'll implement new features and whom to report new requests and bugs. A tight feedback loop between the DevEx team and developers allows the team to react to problems, needs, and improvement ideas quickly.

Onboarding

A good developer experience starts with a positive first impression, which is why onboarding is a key focus area for our DevEx team. Comprehensive documentation on how to set up your development environment is essential for a new hire to get up to speed quickly.

We've taken this one step further with our install_dev.sh script. It installs all the required software at the correct version and helps a new developer set up their local Docker environment, Git integration, common repositories, and even IDE configuration.

Talk about getting a running start! Before, it could take 1–2 days to set everything up. With the script, you'll be done in 2–3 hours at most, with little chance of human error. Considering how frequently we've welcomed new developers to our team lately, we appreciate the time saved.

Developer environment

Another time saver is when you can easily replicate whatever you do in a production environment in any other environment, including local. It means using a similar setup, the same software version (HTTP server, PHP, database), and the same version of packages. The DevEx team helps ensure that the local environment mirrors the production one, resulting in fewer discrepancies between the two and fewer bugs.

The DevEx team has built a handy tool that our developers use daily to manage their development environment — the smtool. It works for both macOS and Linux, automates the running and updating of Docker containers, and keeps repositories and IDE up to date. It started as a collection of Bash scripts, but we're now migrating them to Go.

My personal favorite is the smtool check command. It checks if the required software is up to date and all required shared repositories and local configuration are in place. It's the first command I use to look for the cause of issues with the local developer environment. Most common issues are handled automatically. Whether that's outdated software, lack of configuration or credentials, smtool will offer to solve it right away.

3rd party software and packages

Our Core Engineering Group tends to be the source of most packages that are used in most projects. By using Private Packagist and Dependabot we can handle them easily and ensure up to date versions are used.

New vulnerabilities are discovered every week, and new versions of popular packages are released. Keeping them all up to date is tedious work that takes time. With some automation, it's only a matter of a manual review and approval before changes are made. That's because Dependabot is a software that automatically looks for possible updates and prepares a PR with an update to the latest available version.

Error handling and documentation

Meaningful error messages in internal tooling are one way of making developers’ lives easier. Whenever smtool fails to do something, it tries to come up with a helpful error message. It links to a troubleshooting guide, prints a help page, or asks for credentials to set up missing configuration. This way, you don't have to type in or copy-paste long and complex commands.

If all else fails, we aim for logs to be useful enough to identify bugs quickly.

It's a crucial thing to have documentation for common issues. We tend to test our internal software and document any potential issues we find that can’t be automatically solved. That way our developers get a troubleshooting guide that’s helpful when we’re doing a backward incompatible change (like new PHP or database version).

Documentation is also a place where we keep good practices documented and a common coding style we’d like to follow.

Investing in DevEx is paying off for us

The desired outcome of devoting time and resources to improving Developer Experience is an engineering organization that works comfortably with the local environment and handles tasks efficiently. If working with the code or the environment causes frustration, confusion, or fatigue, it might be a sign of a bad Developer Experience.

We're only at the beginning of our DevEx journey, but the benefits to our developers' happiness and productivity have been encouraging. We're in it for the long run and excited to see where further emphasis on a good Developer Experience will take us!

If you’re interested in diving deeper into DevEx, here are our favorite resources on the topic

Learn more about Supermetrics as a workplace at supermetrics.com/careers/engineering.

Meet the Supermetrics Engineering Team

Paweł Lewtak — Fri, 27 Jan 2023 13:21:31 +0000

The Supermetrics products help businesses streamline their marketing data from siloed sales and marketing platforms into their go-to reporting, analytics, and warehousing tools — whether that's a spreadsheet, a data visualization tool, a data lake, or a data warehouse.

It sounds simple, but when you factor in the scale we operate on, it isn’t. Around fifteen percent of global advertising spend is reported through our products. Every month, we transfer about 2.5 petabytes of marketing data from more than a hundred sources through billions of queries.

It's a team effort to make all that happen. In this blog post, we'll look at how the Product Engineering organization is set up at Supermetrics, how it works, and which product areas each team owns.

100+ engineers, five engineering groups

In 2022, we introduced a new organizational structure with engineering groups. Previously, we had many small and separate teams, but now we're more structured, and each team has a clear set of responsibilities and product areas to focus on.

Currently, our teams are divided into five engineering groups:

The Core Group owns our backend foundation.
The Hub Group owns our new Supermetrics Hub product.
The Destinations Group builds products that transfer data to destinations like Google Sheets, Looker Studio, and data warehousing.
The Connectors Group develops third-party API integrations for data fetching.
The Infrastructure Group supports the rest of the engineering organization and makes their work flow more smoothly.

On top of these groups, we have several specialist teams servicing the entire organization, including Security, Developer Experience, Testing, and Architecture.

Each development team consists of people with various roles working together. Software Engineers and their Engineering Managers are paired with a dedicated Product Manager, and experts from fields like Test Automation, DevOps, and Product Design are on hand when needed.

How we work together

Each team has the autonomy to choose its ways of working. We share some working patterns to make internal communication and alignment easier and avoid siloing.

Two-week sprints with retro meetings and daily standups either as a meeting or via Slack.
We're agile and combine useful features from Scrum and Kanban to manage stories.
Jira is our tool of choice for keeping tabs on tasks, epics, and the like. We plan each epic and discuss these plans across the organization to keep everyone on the same page.
We prefer more communication over less. As the engineering organization grows and our products become more complex, teams must communicate with each other to avoid silos. We have an all-hands dev channel on Slack for discussing general technology issues and ideas.
While different teams are for different things, all engineers at Supermetrics can contribute anywhere in the codebase. If you see a bug on another team's domain, you're welcome to submit a pull request and get the appropriate team member to review it.

What technologies we use

Most of our backend code uses PHP, MySQL, and Redis. You might see some Go, Node.js, or Python, but that's rare. We use these technologies locally in Docker containers. Our staging and production rely fully on cloud services provided by Google and Amazon.

Our frontend is based mostly on React, TypeScript, and Google App Script. Some projects still use Vue.js. We use REST APIs for communication.

Without going into further details, here's a short list of other tools we use in our daily work: Jira, Jenkins, GitHub, PHPStan, Psalm, Jest, PubSub, PHPUnit, and Kubernetes.

Introducing our engineering groups and teams

Core Group

If the Supermetrics Engineering team is the Solar System, the Core Group is The Sun, and the product teams revolve around it. The Core Group is responsible for the backend foundation on which all Supermetrics products are built.

The Core team maintains our custom framework, fixing bugs and optimizing the code for better performance. The team owns the backend platform and provides services like data aggregation to all the other Engineering teams within Supermetrics. They also create reusable components for all our products.

Hub Group

Supermetrics Hub is a new key product for us. It’s the centralized interface where customers interact with a wide variety of different services, like managing, transforming, and exporting their data. They can also manage subscriptions, user accounts, and billing in the Hub.

The Hub Products team builds core features like data blending, data transformation, and data source connections, which are available for all Supermetrics products.

The Hub Workflow team focuses on developing tools for customer engagement, onboarding, and support.

The Platform Growth team owns the purchasing flow and the licensing system.

Connectors Group

A connector is an API integration that fetches data from various sources and processes it to match the user's request before sending it to a destination. The Connectors Group teams are responsible for these third-party API integrations and fetching data from those sources. The Connectors Group teams rely heavily on code designed by the Core Group.

The Connectors Engineering team mostly maintains and migrates APIs within existing coded connectors. We currently have over a hundred connectors, most of which are still coded. It means they're written using actual code (as opposed to Codeless Connectors, which we'll discuss next) and need to be maintained and adjusted as the user base grows.

The Connector Platform team builds Codeless Connectors — a new technology that allows us to build new connectors faster, not with code but with configuration. You can read more about this concept in our previously published tech blog post.

The Connector Specialist team uses the no-code configuration wizard built by the Connector Platform team to configure new Codeless Connectors. The Connector Specialists don't have to have coding skills. They only need to understand the API documentation and our configuration schema.

Product Destination Group

As its name suggests, the Product Destination Group focuses on building and maintaining the data destinations Supermetrics supports. We partner with and build functionality on top of widely used data analysis, visualization, and storage applications, like Microsoft Excel, Looker Studio, and Snowflake.

The user workflow is very similar regardless of destination. You connect to one or more data sources, select the metrics and dimensions you want to pull into the destination application, configure things like date range and filters, and then click to request data. But since all the destination applications are unique, the Product Destination Group's development teams are specialized to work with specific destinations.

The Google Sheets team develops our add-on for Google Sheets. If you use Google Sheets to gather stats from different sources, this product can save a ton of time, so you can focus on making sense of raw data. Recent releases include inline data refresh and triggers for refreshing data in set intervals.

The Excel/PowerBI team builds an add-on for Microsoft Excel. While the user experience might feel similar to what we do for Google Sheets, the code base is entirely different. The Excel add-on requires the Microsoft environment to run, while Google Sheets uses Google Apps Script. A part of this team also develops connectors for Microsoft's Power BI data visualization tool.

The Looker Studio team (formerly Google Data Studio) owns our extension to Looker Studio, which marketers use to create customizable visual reports and dashboards.

The DWH and API Products team works with huge volumes of data. Supermetrics supports various data warehousing and data lake platforms, like BigQuery, Snowflake, Google Cloud Storage, and Amazon S3. Users can create their own schemas to pull specific data from multiple sources and automate transfers to their preferred data storage.

Infrastructure Group

The Groups and teams mentioned above focus on their product areas. The Infrastructure Group, on the other hand, operates cross-functionally and supports the work of other engineering groups and teams. These teams are:

Security
Developer Experience
Architecture
DevOps

Plus: Team Labs

Team Labs is a scrappy startup-like team within our Engineering organization. It takes on new product ideas and organizes itself around greenfield projects. At the moment, the team builds our monday.com product and focuses on new product ideas like how our product is enhanced by generative AI and machine learning features.

Our technology and engineering organization have expanded over time

Each Group and team offers a unique perspective and distinct technical challenges. What they have in common is that engineers can truly influence not just their own work but the evolution of the Supermetrics products and their technical implementation. All our teams are also made up of wildly talented and helpful colleagues.

We continue to grow and are looking for talented people to join our Engineering team. Explore career opportunities on our Engineering careers page.