Forem: Roadie

Backstage Adoption: The Day 2 Problem

Jian Reis — Thu, 06 Feb 2025 14:13:46 +0000

Deploying Backstage is an huge milestone for any engineering team. Whether you’ve set it up to improve service discoverability, streamline onboarding with templates, or enhance governance, the possibilities it unlocks are transformative. But here’s the hard truth: deploying Backstage is just the beginning. What comes next - the Day 2 experience - is where the real work begins.

Day 2 with Backstage is about moving beyond the initial setup and figuring out how to turn Backstage into a tool your developers use as part of their daily or weekly workflows, and in time, come to rely on. Many teams start with specific use cases like service discoverability or software governance, but these don’t always translate neatly into widespread adoption. That’s because solving a single pain point doesn’t necessarily create long-term habits. To make Backstage stick, you need a strategic approach that aligns its features with your developers’ daily workflows. So, let’s explore why Backstage adoption often stalls, and how to set your organization up for sustained success.

Build it and they will might come

It’s easy to assume that once Backstage is deployed, adoption will naturally follow. After all, it solves critical pain points: creating a unified service catalog, automating repetitive tasks with templates, and introducing governance tools to improve software quality. But in practice, adoption requires much more than simply solving these problems. It requires building habits, demonstrating value, and integrating Backstage into the fabric of your engineering culture.

Strategy: treat Backstage like a product

The key to overcoming adoption challenges is to treat Backstage like a product, not just a one-off project. This mindset is at the heart of successful adoption and ensures that Backstage evolves over time to meet the needs of its users. Treating it as a product means committing to continuous improvement, guided by user feedback and measurable outcomes. That does mean that if you’re a platform engineer, you may need to put on a product manager hat, thinking strategically about what your users need, prioritizing features, and continuously communicating the value of Backstage to developers and leadership alike. This means balancing the technical work with a clear focus on solving problems and delivering value internally, which may represent a big change.

It needn’t be overwhelming though - start by engaging your most important internal users - your developers. Communicate with them to understand their workflows, pain points, and priorities. Backstage adoption is a collaborative effort, and the more involved your developers feel in shaping its direction, the more likely they are to embrace it. Use tangible metrics to track success, such as catalog completeness, daily active users, or ROI from key features like templates (engineering time saved for each template run, for instance). These metrics not only highlight areas for improvement but also help demonstrate value to leadership, ensuring sustained investment.

The ‘treat it like a product’ approach to a developer portal such as Backstage is really well articulated by Adam Rogal, who leads Developer Productivity and Platform at DoorDash. In the podcast episode “Bootstrapping a Developer Portal,” Rogal shares how his team built their developer portal, DevConsole, with a clear focus on delivering immediate value to their engineering customers. By engaging engineers early and iterating based on their feedback, the platform team ensured that DevConsole directly addressed real pain points. This approach not only drove higher adoption but also built trust with their internal users.

Rogal also underscores the importance of creating a community - not just of users but of contributors. By empowering teams to actively participate in shaping the portal, DoorDash fostered a culture of collaboration and continuous improvement. This communal effort allowed the portal to evolve alongside the organization’s needs, ensuring its long-term relevance and success.

By adopting this product mindset and prioritizing user engagement, platform teams can transform Backstage into an indispensable tool that enhances productivity and aligns with organizational goals. The experience at DoorDash serves as a powerful example of how this approach can drive both adoption and satisfaction.

Tactics for sustaining adoption

While adopting a product mindset gives you the strategic foundation for long-term Backstage success, tactics offer some actionable focus areas on a day-to-day basis. Let’s explore some key tactics to help drive and sustain adoption.

Automate catalog completeness

A rich and accurate catalog is the foundation of Backstage, but manually maintaining it is time-consuming. The best approach is automation. By using integrations with GitHub or AWS, you can automatically populate the catalog with metadata from your repositories or clusters. In our experience, organizations with the highest levels of catalog completeness (90% or more) have all made extensive use of automations such as templates and scripts, and entity autodiscovery and ingestion to reduce the manual effort involved.

Leverage templates for early wins

Templates are one of the easiest ways to showcase the value of Backstage. They can automate repetitive tasks, like setting up a new microservice or creating a CI/CD pipeline, saving developers significant time and ensuring software governance and best practice is baked in. The ROI here is often immediate and measurable - reducing a process from months to minutes is something both developers and leadership can get behind.

Measure and communicate value

Metrics are your best friend when it comes to adoption. Track data like catalog completeness, template usage (and time saved), and daily active users to understand what’s working and what needs improvement. Share these metrics with leadership to demonstrate the impact of Backstage and justify further investment.

Make adoption a cultural effort

Adoption isn’t just a technical challenge - it’s a cultural one. Evangelism plays a crucial role here. Host “lunch and learns,” demos, or office hours to show developers how Backstage can make their lives easier. Engage other engineering teams and create internal champions who can advocate for the platform within those teams. Adoption grows when developers see Backstage as part of their workflow, not an extra step.

Invest in custom plugins

While Backstage’s open-source plugins are a great starting point, one of Backstage’s biggest selling points is its near limitless extensibility through custom plugins. These plugins can address your organization’s unique workflows and challenges, creating a toolset that developers can’t find elsewhere.

Plan for Day 2 from Day 1

The real value of Backstage lies not in its deployment but in its adoption. To unlock this value, you need to approach Backstage as a product - gathering feedback, iterating on features, and aligning its capabilities with your organization’s goals. Adoption doesn’t happen overnight, but with a strategic mindset and sustained effort, Backstage can become an indispensable tool for your engineering teams.

So, as you set up Backstage, don’t just think about the first deployment. Think about what comes next - the Day 2 experience. Plan for it, invest in it, and you’ll set your organization up for long-term success.

What to think about when you’re thinking about an IDP

Jian Reis — Mon, 27 Jan 2025 16:17:51 +0000

Thinking about implementing an Internal Developer Portal (IDP)? You're in good company - Gartner believes 80% of of platform engineering teams will use IDPs by 2026. There’s a growing sense that every forward-looking technology organization should “have an IDP,” but without a clear rationale as to the "why", this mindset risks building something that (at best) is broad and shallow but that fails to address the real issues slowing your teams down. If you focus on too many things at once, you could end up with a platform that may look impressive but doesn’t actually help people do their jobs.

Instead, a successful IDP emerges when you carefully pinpoint and address the actual challenges developers face. Is discoverability a major and constant headache, with engineers spending hours trying to figure out who owns which service, or whether a certain piece of functionality already exists somewhere else? Are operations teams buried in tickets because developers need help every time they want a new environment or test database spun up? Do your devs waste time hopping between half a dozen interfaces to track deployments, check logs, and find documentation?

These are the signals to tune into when you’re thinking about building an IDP. Aligning your IDP closely with well-understood problems ensures every feature is both purposeful and valued. Equally important, it sets the stage for adoption. Developers and team leads won’t embrace a platform just because it’s there—they’ll embrace it if it truly solves their everyday pain points. By narrowing your focus to the challenges that matter, you can go deep on solutions that genuinely improve how your team works, rather than going wide and hoping something sticks.

Focus on the Three Big Challenges

1. Discoverability

The Challenge:

Have you ever seen teams re-implement a piece of functionality simply because they didn’t know it already existed somewhere else? Or maybe someone spends half a day sifting through old wikis, outdated Confluence pages, and random Slack threads looking for an API endpoint. That’s poor discoverability in action. It’s not just about knowing what services exist, but also understanding who owns them, what their dependencies are, and where the latest documentation or runbooks can be found. When this information is hard to find, it leads to frustration, wasted time, and sometimes unnecessary duplication of effort.

What to Prioritize and Implement:

A Service Catalog that brings together all your services, their owners, docs, performance data, and runbooks in one easily searchable location. Using something like Backstage, you create a living directory of what your organization offers internally, so developers spend less time hunting and more time building.

Measuring Impact and ROI:

Track how often teams ask questions like, “Do we have a service that does X?” or “Who’s responsible for Service Y?” If these inquiries drop significantly, you’ve hit a milestone. Similarly, if your onboarding times shrink—new hires who previously took weeks to understand the landscape now feel comfortable in days—that’s your IDP delivering tangible value.

2. Self-Service

The Challenge:

Ever see a feature get stuck in limbo because the developer can’t get the right environment spun up? Or watch an ops team drown under a pile of infrastructure requests that never seem to end? Without self-service capabilities, your velocity takes a hit. Developers have to wait on someone else’s schedule to get a test database or a staging cluster. By the time the resource is ready, the developer may have lost context or moved on to something else. Multiply that by all the teams and projects in flight, and it’s a huge drag on efficiency.

What to Prioritize and Implement:

Template-driven provisioning and a*automated pipelines* that let developers handle common requests themselves. Pre-approved templates can ensure that every provisioned environment adheres to best practices and security standards, so you’re not just removing a bottleneck—you’re also improving consistency and reliability.

Measuring Impact and ROI:

Start by noting how long it currently takes to get a new environment—maybe it’s three days. After introducing templates, see if you’ve brought that down to a few hours or less. Fewer infrastructure-related tickets and faster environment turnarounds mean your teams can maintain their momentum, delivering features and fixes quicker than before (never mind the increase in developer productivity).

3. Developer Experience (DX)

The Challenge:

Picture a developer’s daily routine: they log into one tool for CI/CD pipelines, another for metrics, another for logs, and a separate browser tab for documentation. This constant context-switching slows them down and increases cognitive load. Over time, this fragmented experience can lead to frustration and reduced morale. It’s not that your teams don’t have the tools—they might have too many, scattered across different interfaces with inconsistent user experiences and integration points.

What to Prioritize and Implement:

An IDP such as Backstage that has all the necessary plugins and integrations properly configured and working becomes a Single Pane of Glass that consolidates these critical elements. By giving developers one place to view logs, metrics, deployments, code reviews, and documentation, you’re streamlining their workflow and cutting down on wasted mental effort. Having an IDP that pulls in data and functionality from multiple sources, presenting it in a coherent, intuitive manner is a surefire way to improve developer experience for your internal engineering teams.

Measuring Impact and ROI:

Survey your developers before and after implementation. Ask how easy it is to find information, how often they switch tools, and how smoothly they can move from coding to testing to deploying. You can also keep an eye on DORA metrics—if the team starts shipping more frequently or fixing issues faster, your integrated interface may be part of the reason why.

Tying It All Together

These three challenges—discoverability, self-service, and DX—often feed into one another. Better discoverability saves teams from reinventing the wheel, reducing the complexity of what you need to maintain. Self-service capabilities speed up delivery, easing the workload on ops and freeing developers to move quickly. Improved DX lowers friction, streamlines daily workflows, and keeps developers happier and more productive. Together, these improvements create a virtuous cycle that helps your engineering organization move faster and build more resilient services.

With that as a baseline, your IDP can really step up and take things a step further. For example, Roadie’s Tech Insights can add another layer of value to your IDP by providing a data-driven, real-time view into the overall health and quality of your services. Rather than relying on anecdotal evidence or gut feelings, Tech Insights surfaces concrete metrics—like compliance scores, dependency health, and adherence to security best practices—within the same platform your developers already use. This makes it easier for engineering leaders to identify hot spots, measure improvement over time, and align investments with areas of greatest need.

Ultimately, you don’t implement an IDP just to say you have one. You implement it because you’ve identified specific, costly problems and want to solve them. Start by pinpointing the real issues—where are you losing time, where are developers frustrated, where are processes too complex or opaque? Then map each problem to a targeted solution—like a service catalog, a set of ready-made provisioning templates, or an integrated interface—and track the results. By focusing on the areas that matter most, you ensure that your platform isn’t broad and shallow, but narrow and deep—truly making a difference where your teams need it most.

Get the “why” and the “what” clear first. From there, your IDP will have real purpose, real value, and real staying power within your organization. It’ll be something people rely on, not just another system they’re forced to use.

Understanding the Backstage System Model

Sam Nixon — Thu, 12 Dec 2024 14:21:20 +0000

The Backstage Internal Developer Portal is, at its heart, a software catalog. As a catalog, Backstage relies on a structured System Model to represent and organize individual items, in order to make it easier to find the information development teams need. When you are setting up or running Backstage you’ll often want to tweak this Model (or make wholesale changes to it) to make it fit your organization.

In this blog we’ll explore the Backstage System Model and how you can extend it if you need to.

Why do we need a system model?

Catalogs require at least some structure. If you don’t have a common taxonomy for how to describe each element inside it then it lacks coherence, like a library with no labels on the shelves (or worse yet, contradictory labels). You could pour in all of your various repositories, components, gateways, resources and clusters into a catalog and it will closely resemble a giant blob of nothing.

In a Catalog, information needs to be sorted to have value. Decisions need to be made about what gets included and what does not, and you need an idea of what goes where - how things are categorized now and how they should be categorized in the future.

The Basics

The Backstage data model is made up of nodes ("entities") and edges ("relationships").

Entities

The Backstage data model is built around "entities." Entities are the core units within the Backstage catalog that represent various elements of your software ecosystem.

Each entity is defined via metadata (name, description, labels etc), spec (custom properties), and relations (connections with other entities). In OSS Backstage this information is often piped into Backstage via YAML files that adhere to Backstage's entity specification. Sometimes entities can also come from "Providers" which provide the entity from some source of truth (i.e. Users and Group entities from Okta)

This model allows teams to maintain a structured, discoverable Catalog by distributing the load across every team who owns part of the Catalog.

Friction Warning:

Backstage advocates for distributed ownership (i.e. each team owns the information in the Catalog that represents the software that it owns) so it can be tricky to update your model and change it over time. For example, if you wanted to replace a Kind all of the various teams would need to update their catalog files. To get around this, a lot of self-hosted Backstage users have built API-based methods for mass updates.

Kinds

Entities are grouped into Kinds. Kinds are like a aisle at a supermarket - everything within it is broadly cohesive and organised around similar principles.

Kinds have a schema and they require a processor to correctly ingest them into the Catalog.

You get some core Kinds out-of-the-box with Backstage, like:

Domain: Defines larger business domains, organizing systems and components
System: Higher-level abstraction representing a collection of components working together
Component: Represents deployable units like services, websites, or libraries)

Friction Warning:

In OSS Backstage you can extend existing Kinds or write new Kinds to include whatever you’d like, but you need to build or modify a processor each time. That means writing code.
You will also need consider the long-term impact of a new Kind. You’ll likely be supporting that Kind for a long time unless you want to deprecate it and force entities that use that Kind to fail.

Types

Kinds have Types, allowing grouping within these larger buckets.

Types can be defined on-the-fly. Nothing special is needed to make Types work, any team can create a new Type just by articulating it in their catalog-info.yaml file.

Friction Warning:

This can lead to a Cambrian explosion of Types, so you may want to introduce some constraint there. Validation of Types is common.
Annoying errors can creep into Types (i.e.Website and Wesbite) unless you’re validating them in some way.

Relationships

Relationships exist between entities to provide the connective tissue of the Backstage Catalog.

Each Kind has a preset series of permissible relationships that are built when the processor runs for that Kind.

For example, a simple Component might have some API relationships and dependencies defined:

```apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: artist-web
description: The place to be, for great artists
spec:
type: website
lifecycle: production
owner: artist-relations-team
system: artist-engagement-portal
dependsOn:
- resource:default/artists-db
dependencyOf:
- component:default/artist-web-lookup
providesApis:
- artist-api




## The Core System model

Out of the box, Backstage comes with a lot of built-in Kinds with attendant relationships so you can get started as quickly as possible. 

Some Kinds, like software templates and Locations are effectively atomic and compartmentalised away from other Kinds. The remainder are tied to how the Catalog is built and used to represented entities.

![Backstage System Model](//images.ctfassets.net/hcqpbvoqhwhm/2c89CI4rKHDNmWvCM2IHhM/37022568f4168c497956da8d9615511a/software-model-entities.drawio-3ce7f43dd236c3934209fde8f21a4d9e.svg)

They in effect represent "The Spotify Way" to model software. That’s not for everyone and won’t necessarily work perfectly for you. 

If that’s the case, you have two options:

- `Force it a little`: aka shoehorn your existing concepts into Spotify’s version. This works in a lot of cases, but is necessarily a compromise.
- `Re-model`: if that doesn’t do the trick, you need to get to work remodeling Backstage entity Kinds and types to fit your needs. Some can be done without code changes, but some need you to get your hands dirty.

# Going beyond the basics and extending the Backstage System Model

The Backstage framework is designed to be highly extensible, allowing you to modify or add new Kinds, Types, and Relationships based on the requirements of your organisation. 

That said, there are a few things you need to think about when extending the model:

### 1. No code extensibility

Backstage has flexibility baked on for a large degree of software definition. Using Types or built-in relationships handles for most situations when you want to model your software inside the Backstage System model. 80-90% of the time this will do the trick, but will often come with some degree of compromise. For example, let’s say you want to articulate `Value Streams` as a top level concept, but have to make do with `Value Streams` being a Type associated to the `Domain` Kind. It’s imperfect, but it’ll do in a pinch.

At Roadie, we evaluate and extend the System Model for our customers regularly. That works a lot of the time, but sometimes customers have niche requests that we don’t feel would benefit all our users. This is non-optimal. We want to customers the freedom to extend the model without talking to us or writing code. To achieve that we’re building a fully self-serve, no-code UI for dynamically generating Kinds and defining a system model that can be as arbitrary as you’d like: if you want a Kind called `purple-monkey-dishwasher` you should be able to have one. 

### 2. Extending the framework using code

Backstage is built around [providers](https://backstage.io/docs/features/software-catalog/external-integrations/) and [processors](https://backstage.io/docs/features/software-catalog/external-integrations/#custom-processors). Providers pull data in, processors manipulate and validate that data to build the Catalog entities and relationships. 

You can create wholly new providers to handle the ingestion of data from sources not currently handled by Backstage. The Backstage community has built  a lot of Providers over the years, but they may require tweaks to fit your specific use-case. For example, Roadie has rebuilt the GitHub provider to use webhook-based ingestion because the size of Catalog we habitually deal with break the GitHub rate limits

You can also modify processors for existing Kinds. For example to extend the list of allowed relationships between Kinds you need to tweak those processors.

You can also create wholly new processors to define new business logic or processes for manipulating and validating that data when you create a new Kind. Going back to the Value Stream example, now you can differentiate `Value Stream` from `Domain` and allow the Kinds to deviate usefully from one another. Maybe they each need different allowed relationships, or they’ll build their entities differently: the choice is yours.

### 3. Data

In the [out-of-the-box OSS Backstage model](https://backstage.io/docs/features/software-catalog/system-model/) the data for the system model comes from yaml files. This follows the GitOps model, where changes are made in git-tracked repositories and then ingested by other systems (in this case, the Backstage Catalog).

That means if you want to change or update your model you need to change all those files. That in turn means that opening PRs against every repos which contain a relevant yaml file. This is often a large undertaking, adding significant friction. That’s why most high-volume users of OSS Backstage have built API- and database-based mechanisms to do mass updates. Roadie has two: the Decorator UI and APIs to do a variety of different update patterns (idempotent updates to sync data from a source of truth into Backstage, or just pushing in whole entities via the Roadie Entities API).

# Levers to pull when extending the model

Below are some common methods for extending the Backstage data model:

### 1. **Custom Annotations**

Difficulty: Trivial

- **Why**: If you need to add metadata specific to your organization (like security labels, compliance levels, etc.), you can define custom annotations.
- **How**: Annotations are added as key-value pairs within the `metadata.annotations` field in your YAML definitions. These annotations can be used to enhance search functionality, create custom views, or provide additional context.
- **Example**: Adding `security-level: high` as an annotation for services that handle sensitive data allows you to quickly filter and prioritize compliance and monitoring for these services.

**References**:

- [Backstage Annotations Documentation](https://backstage.io/docs/features/software-catalog/well-known-annotations/#annotations): Documentation on creating custom annotations to extend metadata.



    ```yaml
    apiVersion: backstage.io/v1alpha1
    kind: Component
    metadata:
      name: fraud-detection-model
      description: "AI model for fraud detection"
      annotations:
        security-level: high

    ...
    ```



### 2. **Custom Types**

Difficulty: Easy

- **Why**: In cases where the existing entity types (Component, API, etc.) do not fit your specific resources, you can create custom entities.
- **How**: Define a new entity type in any valid catalog-info.yaml. This simple involves adding a new type to the `spec.type` in the YAML file.
- **Example**: Suppose you have machine learning models as a core resource in your project. You could define a new `model` type.

**References**:

- [Roadie Kinds and Types documentation](https://roadie.io/blog/kinds-and-types-in-backstage/) talks a lot about how to use Types without introducing problems



    ```yaml
    apiVersion: backstage.io/v1alpha1
    kind: Component
    metadata:
      name: fraud-detection-model
      description: "Machine learning model for fraud detection"
      annotations:
        security-level: high
    spec:
        type: model
      version: "1.0"
      trainingDataset: "transactions-v1"
      accuracy: "95%"
    ```



### 3. **Modifying Existing Kinds to Add Custom Relations**

Difficulty: Normal

- **Why**: Relationships between entities help you capture dependencies, ownership, and team structures within your catalog. If your use case involves additional relationship types, custom relations can improve representation.
- **How**: Modify the relevant processor for a given Kind to enable new types of relationships to be built for that kind. Then define relations within the `spec.relations` section of the YAML file.
- **Example**: Suppose you want to track models associated with data sources. You could create a custom relation `usesDataFrom`, linking ML models to the Resource entities that document data sources they rely on.

**References**:

- [Roadie Kinds and Types Documentation](https://roadie.io/blog/kinds-and-types-in-backstage/): Provides practical examples of defining and extending Kinds.



```yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: fraud-detection-model
  description: "Machine learning model for fraud detection"
  annotations:
    security-level: high
spec:
    type: model
  version: "1.0"
  trainingDataset: "transactions-v1"
  accuracy: "95%"
  relations:
  - type: usesDataFrom
    targetRef: resource:exampleorg/some-data-source
    target:
      kind: resource
      namespace: exampleorg
      name: some-data-source

4. Creating Entirely New Custom Kinds

Difficulty: Normal / Hard

Why: When the System Model cannot adequately encapsulate how you build software or the relationships between various parts of your organisation, you will need to build a custom Kind.
How: Write a new processor for that Kind and define a custom schema for that Kind. This ensures all entities adhere to required fields, valid types, and constraints, providing an additional layer of validation. Then add new catalog-info.yaml files for the new Kind to relevant resources, or modify existing catalog-info.yaml files.
Example: For the MLModel entity, you could create a new Kind to represent that in your System model. Using that new Kind you could then model relationships as version, trainingDate, and accuracy.

References:

Backstage JSON Schema Documentation: Explains how to define and enforce custom schemas.

Conclusion

Backstage is an extremely flexible framework for modelling software and once the building blocks and options are understood it’s simple enough to fully customise the model.

Useful links:

Backstage System Model: official docs and a good starter diagram for how entities in the Catalog interact.
Backstage Entities: official docs on the lifecycle of entities
Backstage Relationships: official docs on how relationships work inside Backstage
Modelling software in Backstage: Roadie blog from 2021 about how to model software in Backstage using the core system model. This still represents a great primer on the out-of-the-box system model and how you could use it.

Why Hybrid is Best for IDPs

David Tuite — Wed, 04 Dec 2024 12:01:18 +0000

The Internal Developer Portal (IDP) market is only 5 or 6 years old, but we’ve seen a huge amount of growth in that time, and many different options emerging.

According to Gartner, IDPs are reported as the most frequently piloted technology in their 2022 - 2024 Technology Adoption Roadmap Survey, and 75% of organizations with platform engineering teams will provide IDPs by 2026.

With this growth, 3 main categories of Internal Developer Portal products have emerged:

Open-source IDPs are exactly what you think. The code is open-source and you host and maintain it yourself.
Proprietary IDPs are typically software as a service startups funded by venture capital. You purchase access to them just like you purchase access to PagerDuty or GitHub Enterprise.
Hybrid IDPs are a combination of open-source and proprietary. They’re built on an open-source foundation, but come with commercial support and proprietary features.

In this article, I’m going to dive into each category one by one, and learn the pros and cons of each.

I’ll also explain why I believe the Hybrid model brings the best of both worlds. The community, extensibility and lack of vendor lock-on of the open-source model, alongside the low cost, ease of use, and fast deployment of the proprietary model.

Open-source

While technically not an IDP, and technically not the only open-source IDP, you can’t talk about open-source IDPs without the conversation focussing on Backstage.

Backstage was open-sourced in early 2020 by Spotify. It was a rewrite of their internal IDP that they’d been using for years. It’s not an IDP because it’s actually a set of TypeScript libraries that developers can combine together to build an IDP, but it has still captured a huge amount of attention in the IDP space.

When Backstage launched, it was often compared to Hygieia from Capital One, which predates it by almost 4 years. Even after Backstage appeared, a third contender emerged with a strong start. Ride-hailing company Lyft threw their hat into the ring when they released Clutch a few months after Backstage was open-sourced. Clutch describes itself as "an extensible platform for infrastructure management”.

in 2024, Clutch popularity seems to have stalled, Hygieia is deprecated, and the reality is that Backstage dominates the open-source IDP market. Although not a perfect metric by any means, GitHub stars show the difference in popularity of each tool.

Benefits of the open-source model for IDPs

In many ways the open-source model is perfect for building an Internal Developer Portal. IDPs are a window into all of the tools that developers use. They’re the venerable “single pane of glass”.

Developers use a lot of tools though, and the developer tool landscape is thousands of tools strong and growing. Even the Cloud Native Landscape alone has more than 1,000 tools in it’s ecosystem.

How can a single IDP integrate with and plug-in to all these tools? Well, by creating an open-source community who will help with the effort.

Huge community

This is exactly what Backstage has accomplished. The Backstage community is huge and engaged.

Backstage was the top end-user contributed CNCF project of 2023, with more than 4,000 contributions from end user companies.

Benefits of the open-source model for IDPs

Developers use a lot of tools though, and the developer tool landscape is thousands of tools strong and growing. Even the Cloud Native Landscape alone has more than 1,000 tools in it’s ecosystem.

How can a single IDP integrate with and plug-in to all these tools? Well, by creating an open-source community who will help with the effort.

Huge community

This is exactly what Backstage has accomplished. The Backstage community is huge and engaged.

Backstage was the top end-user contributed CNCF project of 2023, with more than 4,000 contributions from end user companies.
There are 17,000 people in the Backstage Discord channel, with dozens and dozens of questions being asked or answered every day.
Backstage has a deep partner ecosystem. Whether you’re a Fortune 50 enterprise or a Series C startup, there’s a partner who can help you deploy and use Backstage.
Backstage has official support from leading DevTools companies. Companies like Snyk, PagerDuty, Dynatrace and even AWS have released officially supported plugins for Backstage.

This vibrant community means that you can always get help with Backstage. The other benefit is the extensibility and the number of integrations available.

Huge number of integrations

The result of this large community is that Backstage is installed and adopted at thousands of organizations. These organizations use a diverse set of engineering tools, and many adopters have built and open-sourced Backstage plugins for these tools.

The Backstage plugins directory has more than 200 plugins available, so you can be fairly confident that there are plugins available for the tools you use.

Highly extensible

Underlying all these plugins is a framework which is designed from the ground up to be maximally extensible. The docs say:

Backstage is a single-page application composed of a set of plugins.

Our goal for the plugin ecosystem is that the definition of a plugin is flexible enough to allow you to expose pretty much any kind of infrastructure or software development tool as a plugin in Backstage.

Practically, this means you can customize and extend your Backstage install to your specific requirements. If you need to build your catalog in Gerrit rather than GitHub, then you can swap out the GitHub integrations and replace them. The same concept applies to most parts of Backstage.

By coupling this technical extensibility with a permissive Apache 2.0 license and a mature governance model, adopters can make wholesale changes to Backstage if they like.

The result is that many organizations have customized Backstage heavily. Take a look at this screenshot of Sunrise, the Backstage-based IDP that Zalando created, as an example.

This completely custom UI shows the progress of changes as they flow through CICD on their way to production. Very little of what you see here comes out of the box with Backstage. Instead, it’s wired together from the basic building blocks of Backstage plugins.

Lack of vendor lock-in

Because Backstage is open-sourced and backed by large end user companies and the CNCF, adopters can feel comfortable that it will be around for a long time, and will continue to meet their needs into the future.

The IDP market is still very early. The majority of the players have been founded in the past 5 years. We will likely see some consolidation of this market as players get acquired and wound down. Any customers of these companies will be forced to migrate to a different solution.

We’ve seen this play out in the CICD market already. There was a time when Travis CI was a dominant provider of continuous integration. In 2019, Travis CI was acquired by private-equity firm Idera. One month later, layoffs began. In early 2020, Travis CI stopped providing free support for open-source products, and in late 2020 the pricing changes began.

Companies can avoid this fate in the IDP market by building on an open-source project instead of a commercial vendor with it’s own data model and lock-in.

Drawbacks of the open-source model of IDPs

It’s expensive to build

Backstage is a larger undertaking than most people realize before they get into it. To quote Gartner in their Innovation Insight for Internal Developer Portals report:

Gartner inquiries indicate that Backstage implementations may require substantial effort in standing up the service.

In order to replicate something approaching the Sunrise portal mentioned above, companies should expect to allocate 2 to 5 engineers to the project for a number of years.

Don’t believe me? Zalando said it themselves in this presentation they did at the Autodesk Developer Productivity Summit.

All that I’ve showed you was done through the contributions of many teams, but the core team was three engineers and a engineering manager.

That team has been working on Sunrise since 2020.

And that’s where my team is focussed on, actually like, the discovery part of our application landscape, and where we actually decided to, a few years ago in the beginning of the Backstage era in 2020 when Backstage was released, to actually onboard ourselves into Backstage.

Let’s do the math. Assume an engineer or engineering manager costs $250k per year, therefore the team of 4 costs $1 million per year. 4 years have passed since 2020, so they’ve spent $4 million in total on their core Backstage team. Add in the “contributions of many teams” and we could easily be talking about a $6 million outlay.

Now you’re probably saying “yeah but we’re not as big as Zalando”, but trust me, even small companies will spend a half a million dollars pretty easily.

It requires uncommon skills

Backstage is typically owned and managed by platform teams or DevOps teams. These teams are probably skilled in YAML and kubernetes-native languages like Go.

Backstage is written in TypeScript. The backend runs on NodeJS, and the frontend is written in React.

Organizations who are considering self-hosted Backstage should consider whether or not they have the skills to develop in these languages. As mentioned above, Backstage is a set of libraries that users combine together to make a developer portal. That means you need to write TypeScript to make use of it. You need to understand frontend development, HTML and CSS to customize it. It may not make sene make sense to build up these skills in a Platform and Infrastructure organization because they will not be easily transferrable to other projects.

The second set of uncommon skills that a Backstage deployment needs are project management and developer relations. Once it’s live, you need to evangelize it. You have to go out to the rest of the organization and teach them about it and how to use it. Are your platform developers really going to want to do this work?

It’s got a long time to value

The typical Backstage rollout has a few phases. It starts with an initial deployment, which typically takes a few months to get a basic Backstage instance into production. Some parts of Backstage will work straight out of the box at this point. You’ll be able to use the scaffolder to create new software projects from predefined templates, for example.

Other parts of Backstage require an adoption phase. The software catalog is usually one of them. In order to have a complete software catalog, teams must first put their software into it.

In 2023, at BackstageCon North America, I spoke to the experiences of some Backstage adopters who had attempted this feat. I first explained that 60% of the Backstage adopters I had interviewed were attempting to populate their catalog by writing YAML files that contain metadata about their services (these are called catalog-info.yaml files).

I then shared some quotes from adopters who had followed this path

The catalog never really worked for us. We struggled with adoption and rate limiting. Half the info in the catalog was wrong when they first pushed adoption.
2,000 engineers, 2 years experience

The thing we struggled with the most was getting teams to create the YAML file. We spent 6 months actively encouraging teams to register their components with Backstage. After 1 year, only 30 or 40% of the active repos were there.
120 engineers, 1 year experience

Without a lot of product development and project management, this catalog population process takes a long time. There are many Backstage adopters who are at less than 50% catalog completeness after two years with the tool. Even Spotify admit that Backstage adoption rate often stagnates at 10%. You may have dreams of your software catalog becoming a single pane of glass for all of the software development in your organization, but it simply cannot fulfill its purpose if the software is not in there.

Proprietary

At the other end of the spectrum, there are a number of proprietary developer portals like Cortex and Port.

The concept of proprietary developer portals is largely the same as Backstage. They have similar features and target a similar user in the engineering organization. The main differences are in their extensibility. Users can’t edit the code of a proprietary developer portal, and there won’t be a large community who are producing open-source plugins.

Benefits of the proprietary model for IDPs

While a proprietary developer portal may never be as extensible as Backstage, they certainly do have some benefits.

Quicker to get started

Proprietary IDPs will mostly just work out of the box. You don’t have to plan sprints and do research and write code to get some value. They come with admin interfaces where you can set some properties, connect up your tools, and starting using the product straight away. This is likely a better experience for platform engineers who don’t have the time or expertise to deploy and customize Backstage from scratch.

When you do need to configure something, the documentation provided by a proprietary developer portal will likely be more comprehensive and up to date. They’re simply more motivated and able to produce high quality documentation than an open-source community. This makes setup faster and less confusing, and helps to prevent misconfigurations.

User-friendly interface

Proprietary IDPs bring their own user interface (UI). They’re developed from the ground up with design systems in place to help to ensure consistency across integrations and pages. They also have designers and UI engineers working to implement design practices in a coherent way.

While the Backstage core team have done a lot to put the tool in a good place, the reality is that Backstage’s default interface leaves a lot to be desired.

This reality is brought on by the combination of overly familiar Material UI libraries and community contributed code that can be hit-or-miss in terms of how it looks and feels.

Backstages plugins can be contributed by anyone, so there’s no guarantee that they will look good, or look consistent. It’s very likely that different plugins will work completely differently even though they are installed in the same Backstage instance.

More guidance

Proprietary IDPs are more opinionated in how they should be used. When you work with a vendor you should get access to solutions engineering and customer success resources who can help to share lessons from other customers.

Working with the open source can sometimes feel like being handed a box of car parts and asked to build a car. Sure you can figure it out eventually, but you’re going to make mistakes along the way, and it probably won’t be the most efficient path.

Having a partner to work alongside and share lessons is worth a lot, and helps to cut the time to value for your IDP project.

Drawbacks of the proprietary model for IDPs

Vendor lock-in

Proprietary IDPs have their own in-house APIs, their own data model, and their own way of doing things. Many of the options on the market are early stage startups. There’s a good chance some of them will disappear over the next few years. If you’re a customer, your IDP will go with them.

When they do survive, you’ll likely be forced to choose between paying up for price hikes, or completing an expensive migration project to a different solution.

Limited extensibility

It’s not possible to change the code of a proprietary IDP. If there’s something about the way it works that doesn’t suit the needs of your company, your best hope is to contact support and hope that they have the desire and the roadmap space to change it.

Most development organizations have some homegrown tools that have sprung up over the years internally. These tools only exist inside the company they were created, and no IDP vendor will have an out-of-the box integration for them. For these tools, you need to create your own plugins if you wish them to be part of your single pane of glass IDP. Some proprietary IDPs do support custom plugin development, but you’ll need to do so using only the concepts and libraries that the vendor provides, and the plugins you create will not be transferrable to any other solution.

Proprietary IDPs have fewer integrations than the massive Backstage community. Even the most well funded proprietary vendor has 48 plugins listed on their integrations page, less than a quarter of the number that are available for Backstage. No matter how big this vendor grows their team, they won’t be able to compete with the 500+ companies who have contributed to Backstage.

Limited speed of execution

The tech landscape is constantly shifting and evolving. Before serverless we had Kubernetes, before Kubernetes we had virtualization, before virtualization we had bare metal. In between and around these major platform shifts we had and continue to have a massive number of different types of serverless, containerization and virtualization options.

A developer portal is supposed to wrap all of these shifts and technologies in order to give the best possible coverage over your internal engineering landscape. Proprietary IDPs may struggle to capture all of these technology shifts into the future, leading to gaps in the portal that platform teams put in front of their internal users.

Hybrid

Last, but not least, we have the Hybrid model. This model takes an open-source foundation with hundreds of plugins and integrations available, and makes it available in an out-of-the-box format which is easy to get started with.

This is what Roadie is. It’s the only standalone IDP based on Backstage, and it offers the best of both worlds - extensibility and speed of execution.

Huge number of integrations

The vast majority of plugins that have been created for Backstage can work on Roadie. Our integrations library contains 70+ plugins and integrations today, and we’re adding more all the time, in line with customer requests. Once a new open-source plugin is created, it takes us about a day to make it available in Roadie.

The only reason we wouldn’t add a Backstage plugin are that it’s of such low quality that it doesn’t work or provide any value.

This means that Roadie customers can effectively choose from 200+ plugins and integrations that the Backstage community has created and use them on our platform. In fact, we’ve created some of the most popular plugins ourselves and open-sourced them.

Highly extensible

Roadie strives to support as much customization of our IDP as possible. In addition to theming and branding, we support a custom data model, custom Backstage plugins, custom proxies, layouts, scaffolder actions and on and on. You name it, we can probably let you customize it.

This means that you can customize Roadie to meet the specifics of your organization, improving adoption and facilitating ease of use.

No vendor lock-in

Roadie is API compatible with Backstage. The same software metadata schema that works on Backstage will work on Roadie. Custom frontend plugins written for Backstage will also work on Roadie. We’re just supporting and extending the same APIs.

This has two benefits for customers.

They can migrate off self-hosted Backstage onto Roadie very easily. Numerous organizations have already done this. Simply connect us to your source code management tool and we’ll automatically ingest any software metadata files you’ve created.
They can migrate back to self-hosted Backstage very easily. In fact, our Terms of Service dictate that we will give you your data back within 30 days of termination.

User friendly interface

Roadie’s interface is vastly improved compared to the barebones Backstage experience. We’ve worked with our users to streamline how people use the product. This means better out of the box defaults and easier customization.

Inexpensive

In many cases, Roadie is 5 times cheaper than self-hosting Backstage.

People tend not to realize this up front, but the most highly developed Backstage deployments in the world have cost millions of dollars. We talk to companies all the time who have built a team of 5+ engineers around Backstage and run those teams for 3+ years. The most expensive Backstage deployment we’ve seen cost $7.5 million dollars over 3 years.

Roadie doesn’t require such an extensive team to be built around it. We take away all of the operations and deployment effort, and much of the customer support that has to happen. We also give you out-of-the box features like Scorecards and Role-Based-Access-Control that you would otherwise have to build yourself to realize the potential of your IDP.

Lastly, we charge based on usage. You only pay for the engineers who write code which is tracked in the catalog. Everyone else gets to log in for free. This means you can ramp gradually throughout your org, rather than paying for the developement of the entire IDP up front.

Requires few skills

Roadie doesn’t require any TypeScript skills. You just drag and drop plugins where you want them, and configure the product in our administration panels. It’s dead simple. This is perfect for platform teams who are trained on cloud-native technologies. We even have no-code plugins that you can use to display lists or charts inside the UI without writing any code.

If you do need to make something completely custom and you don’t want to write a Backstge plugin yourself, we can sort you out with professional services to suit your needs.

Short time to value

We’ve built a ton of features into Roadie to help shorten the time to value. For example, we help teams build out their catalog quickly and we see customers reaching a high level of catalog completeness within 4 months.

We also provide customer success and guidance to help organizations effect change with their IDP. We take the lessons we’ve learned over the years and use them to guide you to success in as short a time as possible.

Conclusion

We’ve designed Roadie from the ground up with flexibility and easy adoption in mind. We believe engineering organizations should not waste their resources on undifferentiated heavy lifting projects like deploying an open-source IDP. At the same time, they shouldn’t be locked into a proprietary data model or limited in terms of what they can build or the integrations that are available.

It turns out that you can have your cake and eat it too. An Internal Developer Portal can be both easy to use and supported by a massive and growing open-source community. This means that you can focus on unlocking productivity in your engineering organization while knowing you have a rock-solid and flexible foundation beneath.

I’ve also written about the difference between IDPs and the IDP space in general over on The New Stack

Image by MSTORK from Pixabay

Migrating to Backstage’s New Backend: A Step-By-Step Guide

Miklós Kiss — Tue, 05 Nov 2024 09:08:05 +0000

Introduction

Migrating to Backstage’s new backend system is more than just an upgrade - it’s a step toward a more scalable, efficient, and streamlined platform. The new backend introduces a modern architecture that simplifies plugin integration, reduces complexity, and improves overall performance. This is a significant shift from the monthly updates and incremental improvements you may be used to. By adopting this new architecture, you’ll gain better control over plugin dependencies, reduce overhead, and set the stage for easier future development.

This migration is particularly exciting because it marks the culmination of a long journey towards a more modular and efficient backend. Originally introduced as an experimental feature, the new backend has matured into a fully supported system that greatly enhances how plugins interact with core services. It’s a big leap forward from the previous architecture, and while the transition may require some effort, the long-term benefits—such as improved scalability, clearer separation of concerns, and better dependency management—make it a worthwhile investment for teams looking to future-proof their Backstage setup.

Overview

In this guide, we’ll focus on helping you migrate your existing Backstage backend with minimal disruption. We’ll walk through key steps, such as backing up your current system, creating a bridge for your existing plugins, and gradually integrating them into the new architecture. By the end of this article, you’ll have a clear understanding of how to approach the migration while maintaining your current functionality, and how to unlock the benefits of Backstage’s new backend.

We’ll focus on performing a minimal migration that gets your main backend (index.ts in packages/backend) up and running in the new system. At this stage, we won't migrate all plugins, as that can be a more time-consuming process. Instead, we’ll leverage a temporary bridge function to convert existing plugins into the new system using a transitional environment, allowing you to start benefiting from the new architecture right away.

Preparation for Migration

Before diving into the migration process, it's essential to ensure that your new system will function as expected post-migration. Proper preparation can significantly reduce the risk of issues arising during or after the transition.

Establish a Testing Plan

Manual Testing: Develop a comprehensive plan for manually testing your application with the new backend. This should include detailed scenarios that mimic real-world usage of your application. Identify critical paths and functionality that must work seamlessly in the new environment.
Integration Tests: In addition to manual testing, it's crucial to have a robust set of integration tests. These tests should cover all functionalities, particularly those related to any custom modifications you've implemented in your existing system. Ensure that your tests validate the expected behavior of all integrated components.

Pay special attention to any custom features or modifications you’ve made. These areas are often the most prone to issues during migration, so thorough testing is essential. And of course it goes without saying, but make sure all your existing tests don’t fail!

By laying the groundwork with a well-defined testing plan—comprising both manual and automated integration tests—you can increase your confidence in the migration process. This preparation will help ensure that your new backend system operates smoothly and consistently after the migration is complete.

Migration Guide from Backstage

Before proceeding with this migration guide, it's essential to familiarize yourself with the resources available from Backstage. The official migration guide on the Backstage website provides valuable insights and best practices for transitioning your backend system. While there may be some overlap with this guide, repetition can reinforce key concepts and ensure that you don't miss critical steps.

Take the time to read through the Backstage migration guide before continuing with this document. Doing so will provide you with a solid framework and context that will aid in your migration efforts, ultimately leading to a smoother transition.

What’s the Plan for Rollout?

To ensure a seamless transition to the new backend, it’s crucial to carefully plan and prepare your rollout strategy. Here are some key considerations to guide your approach:

Deployment Strategy:
- New Image Creation: Consider creating a new Docker image for the updated backend. This allows you to maintain clear versioning and ensures that your deployment environment matches your development and testing environments.
- Blue-Green Deployment: If feasible, implement a blue-green deployment strategy. This involves running two identical environments (blue and green) where one is live while the other is idle. You can switch traffic between them to minimize downtime and facilitate rollback if necessary.
Monitoring and Metrics:
- Set Up Monitoring Tools: Implement monitoring solutions to track performance metrics, error rates, and resource usage. Tools like Prometheus, Grafana, or your existing APM solutions can provide valuable insights during and after the rollout.
- Health Checks: Ensure that health checks are in place to quickly identify any issues with the new backend as it goes live.

Migrating Your `index.ts` to the New Backstage Backend System

In this section, I’ll walk you through the process of migrating your Backstage backend to the new backend system. We’ll focus on updating your index.ts file, which serves as the main entry point for your backend. This migration will allow you to start leveraging the streamlined architecture and dependency injection provided by the new system.

Step 1: Backup Your Existing `index.ts`

Before starting any migration, it’s essential to create a backup of your current index.ts file. This way, you can reference it or roll back if needed. Let’s save this backup as index.backup.ts.

Additionally, to avoid issues with type checking while you’re in the middle of the migration, add @ts-nocheck at the top of your backup file.

// index.backup.ts
// @ts-nocheck

Step 2: Set Up the New `index.ts`

Now, create a new index.ts file, which will be the entry point for your new backend system. For this initial step, you can keep it as minimal as possible. The goal is to have a working backend skeleton, which we will build upon later.

Here’s an example of a minimal setup:

import { createBackend } from '@backstage/backend-defaults';

const backend = createBackend();

backend.start();

At this point, your new backend doesn’t do much—it's just an empty shell. But it’s important to verify that the basic setup works before adding any of your legacy plugins.

What’s Next?

In the next part of the migration, we’ll create a temporary legacy environment for your existing plugins and gradually integrate them into the new backend system. This approach allows for a smooth transition, letting you keep the old plugins running while starting to take advantage of the new architecture.

Creating a temporary plugin environment

To ease the migration process, Backstage provides a handy bridge function, makeLegacyPlugin, which helps create a temporary environment compatible with the old backend system. This allows your legacy plugins to continue working while you transition to the new backend architecture. The function ensures all required dependencies are injected into the plugin, simulating the old system’s behavior.

Here's an example of using makeLegacyPlugin:

const legacyPlugin = makeLegacyPlugin(
  {
    logger: coreServices.logger,
    cache: coreServices.cache,
    database: coreServices.database,
    config: coreServices.rootConfig,
    reader: coreServices.urlReader,
    discovery: coreServices.discovery,
    tokenManager: coreServices.tokenManager,
    permissions: coreServices.permissions,
    scheduler: coreServices.scheduler,
    events: eventsServiceRef,
    eventBroker: eventBrokerService,
    auth: coreServices.auth,
    httpAuth: coreServices.httpAuth,
    userInfo: coreServices.userInfo,
    pluginName: coreServices.pluginMetadata,
    identity: coreServices.identity
  },
  {
    logger: log => loggerToWinstonLogger(log),
    cache: cache => cacheToPluginCacheManager(cache),
  },
);

In this setup, your makeLegacyPlugin function acts as a temporary replacement for the older makeCreateEnv function. If makeLegacyPlugin returns the same dependencies as makeCreateEnv, you're set for a smooth transition.

You can also provide type conversion functions within the second parameter to handle any type discrepancies between old and new service structures. For example, converting a logger or cache to the format expected by the new backend system.

Caveats and Considerations

One important note is that all of these dependencies will be instantiated for every plugin that uses the legacyPlugin bridge. This can become problematic, particularly if any dependencies involve heavy operations (e.g., opening database connections). While this bridge is a useful short-term solution, it's highly recommended to fully migrate your backend plugins to the new system as soon as possible.

Using the `legacyPlugin`

You can add your legacy plugins to the backend using legacyPlugin like this:

backend.add(legacyPlugin('todo', import('./plugins/todo')));
...
backend.start();

By focusing on migrating only your index.ts file initially, you minimize disruption while keeping the migration manageable. This strategy allows you to stay on top of the process while taking advantage of the new backend architecture in stages, ensuring a smooth transition.

Adding new backend system plugins

You can directly add your plugins that are already migrated to the new backend system into your index.ts file.

backend.add(legacyPlugin('todo', import('./plugins/todo')));
...
backend.add(import('@roadiehq/foo-bar')); // a backend plugin in the new system
...
backend.start();

Overriding Core Services

Backstage offers a robust architecture that includes a set of core services added to the application by default. When a plugin or service references a core service, it automatically receives the appropriate instance.

Custom Implementations

If you had custom implementations of certain services in your old system, you will need to override them in the new backend. This is crucial for maintaining functionality and ensuring that your custom logic remains intact.

Recommended Approach

To keep your Backstage directory structure organized and maintainable, I recommend creating a separate package for these service overrides. This approach promotes clarity and separation of concerns within your project.

Package Naming Convention: In the upstream OSS version of Backstage, this package is commonly referred to as backend-defaults. Adopting this convention in your project will help standardize your codebase and make it easier for others to understand your structure.

Implementation Steps

Create the backend-defaults Package:
- Set up a new package in your Backstage repository specifically for service overrides.
Implement Overrides:
- Define the necessary overrides for the core services that require customization. Ensure that these overrides integrate seamlessly with the existing Backstage architecture.
Update Your Application:
- Modify your application’s configuration to reference the new backend-defaults package, ensuring that your custom implementations are used where needed.

The current core services that backstage provides are the following:

// backstage/backstage/packages/backend-defaults/src/entrypoints
auth
cache 
database
discovery
httpAuth
httpRouter
lifecycle
logger
permissions
rootConfig
rootHealth
rootHttpRouter
rootLifecycle
rootLogger
scheduler
urlReader
userInfo

You can import the core services from the respective path like this:

import {
  RootHttpRouterConfigureContext,
  rootHttpRouterServiceFactory
} from '@backstage/backend-defaults/rootHttpRouter'

Creating Your `backend-defaults` Package

To effectively manage your service overrides in Backstage, you can create a backend-defaults package using the Backstage CLI. This package will serve as a centralized location for your custom service implementations.

Step-by-Step Instructions

Run the Backstage CLI Command:
Execute the following command to create a new package:
```
npx backstage-cli new
```
Select Package Type:
When prompted, choose the option for creating a Node library:
- Node Library: This will allow you to export shared functionality for backend plugins and modules.
Set the Package ID:
Add the package ID as backend-defaults when prompted. This step ensures that your package is correctly identified within your Backstage setup.
Package Creation:
Upon completion, this command will generate a new package in your packages directory named backend-defaults.
Organizing Your Overrides:
To keep your package structured and maintainable, I recommend creating a folder within the backend-defaults package called services. This folder will house your overrides or any new services you implement.
```
mkdir packages/backend-defaults/services
```

The following is an implementation of an override of the coreServices.database service. This is a potential fix for cases where your createEnv() and pluginID strings did not match so in your old system the database name does not match the API path.

In the new backend system it will always be the case that your plugin’s ID determines the name of its database and the path it will be attached to. This “fix” is needed because the current upstream Backstage doesn't provide a way to override the database name for these rare edge cases.

import {
  coreServices,
  createServiceFactory,
} from '@backstage/backend-plugin-api';
import { ConfigReader } from '@backstage/config';
import { DatabaseManager } from '@backstage/backend-defaults/database';

export const databaseServiceFactory = createServiceFactory({
  service: coreServices.database,
  deps: {
    config: coreServices.rootConfig,
    lifecycle: coreServices.lifecycle,
    pluginMetadata: coreServices.pluginMetadata,
  },
  async createRootContext({ config }) {
    return config.getOptional('backend.database')
      ? DatabaseManager.fromConfig(config)
      : DatabaseManager.fromConfig(
          new ConfigReader({
            backend: {
              database: { client: 'better-sqlite3', connection: ':memory:' },
            },
          }),
        );
  },
  async factory({ pluginMetadata, lifecycle }, databaseManager) {
    const pluginId = pluginMetadata.getId();
    let databaseName;
    switch (pluginId) {
      case 'foo-bar':
        databaseName = 'foo_bar';
        break;
      case 'tech-insights':
        databaseName = 'tech_insights';
        break;
      default:
        databaseName = pluginId;
    }
    return databaseManager.forPlugin(databaseName, {
      pluginMetadata,
      lifecycle,
    });
  },
});

Configuring the httpRouter service

The backend system comes with its default configured httpRouterService. This is good for basic use cases. It contains multiple middleware and is responsible for the default configuration of the express router.

If the default configuration is not enough, or you made customizations on your router in the old system you can provide your configurations like this:

// index.ts
import {
  RootHttpRouterConfigureContext,
  rootHttpRouterServiceFactory,
} from '@backstage/backend-defaults/rootHttpRouter';
...
backend.add(
  rootHttpRouterServiceFactory({
    configure: (context: RootHttpRouterConfigureContext) => {
      const { app, config, logger, routes, applyDefaults } = context;
            // register your custom middlewares

            applyDefaults() // apply the default middlewares from the backstage core service
    },
  }),
);

Backstage comes with a default request logger. You cannot turn it off and can not be overridden. If you want to replace it (or any of the default middleware) and use your own request logger you have only one choice: don’t apply the default middleware, and instead use each individual middleware as needed, and your custom implementations.

// index.ts
...
backend.add(
  rootHttpRouterServiceFactory({
    configure: (context: RootHttpRouterConfigureContext) => {
      const { app, config, logger, routes, applyDefaults } = context;

      const backstageMiddlewares = BackstageMiddlewareFactory.create({
        config,
        logger,
      });

            // we leave out the backstageMiddlewares.logging() and use our own logger.
      app.use(myCustomRequestLoggingHandler());

            // add rest of the default middlewares
      app.use(backstageMiddlewares.helmet());
      app.use(backstageMiddlewares.cors());
      app.use(backstageMiddlewares.compression());
      app.use(routes);
      app.use(backstageMiddlewares.notFound());
      app.use(backstageMiddlewares.error());
    },
  }),
);

Healthcheck

If you want to override the default healthcheck you can easily do it by attaching a new endpoint for it.

You can create a backend plugin for your healthcheck and then register it in your index.ts file.

Use the backstage-cli to create a new plugin.

// run and select   backend-plugin - A new backend plugin 
npx backstage-cli new

This will create a new backend plugin inside your project under the plugins folder. It will be called the ID that you provided in the prompt.

// plugins/healthcheck-backend
export const healthCheck = createBackendPlugin({
  pluginId: 'healthcheck',

  register(env) {
    env.registerInit({
      deps: {
        rootHttpRouter: coreServices.rootHttpRouter
      },
      init: async ({ rootHttpRouter, rootLifecycle }) => {
        rootHttpRouter.use('/healthcheck', (_, res) => {
          res.json({ status: 'ok' });
        });
      },
    });
  },
});

// packages/backend/src/index.ts
backend.add(import('healthcheck-backend'));

Using the lifecycle hooks

The new backed system provides a core service to hook into the different lifecycles of the process. There is a plugin scoped and a root scoped lifecycle and rootLifeCycle service respectively.

In the old system you might have hooked into the service start promise. In the example below we call the function runOnStartup

  // packages/backend/src/index.backup.ts

  const service = createServiceBuilder(module)
  service
    .start().then((_server: Server) => {
      logger.info(`Startup finished`, {
        uptime: process.uptime(),
      });

      runOnStartup({
        config
      });
    })
    .catch(err => {
      logger.error(
        'The http server threw an unexpected error',
        isNativeError(err) ? err.message : `unknown error raised: ${err}`,
      );
      process.exit(1);
    });

The same functionality can be achieved with the lifecycle hooks in the new backend system.

Create a module for your functionality. The example demonstrates how to add a lifecycle hook to the tech-insights plugin:

export const techInsightsModuleCalculateNew = createBackendModule({
  pluginId: 'tech-insights',
  moduleId: 'calculate-new',
  register(reg) {
    reg.registerInit({
      deps: {
        config: coreServices.rootConfig,
        discovery: coreServices.discovery,
        featureFlagStore: featureFlagStoreServiceRef,
        lifecycle: coreServices.rootLifecycle,
      },
      async init({ config, discovery, featureFlagStore, lifecycle }) {
        const onStartup = () => {
          ...
        };
        lifecycle.addStartupHook(onStartup); // Add the function to be run at startup
      },
    });
  },
});

Testing the New Backend

Testing your migrated backend is a critical component of any upgrade process. Ensuring that your application functions as expected after migration can prevent a host of issues down the line.

Setting Up a Development/Test Environment

It is strongly recommend to establishing a dedicated dev/test environment. This allows you to deploy your new version and conduct tests with minimal traffic interference. Since we are re-architecting the entire Express application and modifying how plugins are integrated, validating that your plugins are accessible is absolutely paramount.

Implementing Health Check Tests

A good starting point for testing is to implement basic health check tests using your existing end-to-end (e2e) or integration testing framework. These tests should verify that each plugin is mounted correctly to its expected path. Here’s an example of how you can do this with Playwright:

import { test, expect } from '@playwright/test';

test.describe('/healthcheck', () => {
  test.describe('GET', () => {
    test('The healthcheck endpoint is configured', async ({ request }) => {
      const result = await request.get(`/healthcheck`);
      expect(result.status()).toBe(200);
    });
  });
});

Reviewing End-to-End Test Coverage

Before migrating, take a moment to review your end-to-end test coverage. Understanding your current testing landscape will help identify gaps, especially given the extensive changes involved in this migration. The more coverage you have over critical services, the better equipped you will be to catch issues early.

Pay special attention to:

Custom Implementations: Review any middleware, metrics, or logging mechanisms that may impact the migration.
Company-Specific Plugins: Ensure that any frontend/backend plugins unique to your organization are covered.
Startup Hooks: Verify that these are functioning correctly post-migration.
Legacy Functions: Check the usage of custom legacy functions to avoid conflicts with plugin IDs and database names.

Manual Testing

Finally, don't underestimate the importance of manual testing. Take the time to navigate through your application before completing the merge. The migration touches a broad surface area, making it challenging to cover every scenario with automated tests. If you've managed to create comprehensive automated tests—kudos to you! But a thorough manual check can help catch any edge cases that might otherwise be missed.

Conclusion

By following these testing practices—establishing a robust test environment, implementing health checks, reviewing your test coverage, and conducting manual tests—you can significantly reduce the risks associated with your backend migration. This proactive approach will help ensure a smooth transition and a reliable application post-upgrade.

Quirks Encountered in the New Backend System

As we transitioned to the new backend system, we encountered several quirks that are important to highlight, particularly regarding database naming conventions and plugin path correlations.

Database Name and Plugin Path Correlations

In the updated architecture, database creation is directly tied to the plugin ID. While this design should streamline the process, it can lead to issues if there has been a lack of consistency in the previous backend, specifically within the createEnv function.

const createEnv = makeCreateEnv(config);
const todoEnv = useHotMemoize(module, () => createEnv('todo'));

In the legacy system, the database name was simply derived from the parameter passed to the createEnv function. The new backend would generate the databases using the plugin ID or the string you pass to the legacyBackend bridge function and it would use this same string as the API path. This is an issue if your parameter to the createEnv function and the path you attached your plugin to were not the same.

In the new system, the plugin ID takes precedence—it dictates both the database name and the API path.

Key Considerations

Consistency is Crucial: Ensure that you consistently use kebab-case for plugin identifiers across both the legacy and new systems to avoid unintended database creations.
No Official Override: Currently, there is no configuration option to override this behavior, making it essential to align naming conventions proactively.

Catching Errors in Plugin Configuration

It's crucial to ensure that any misconfigurations or missing settings are promptly identified. One common issue is that if a plugin configuration is not available, the application may still start up without any visible indications of the problem. This can lead to missed error messages and difficult-to-diagnose issues later on.

Effective Monitoring Strategies

To improve your visibility into potential errors during startup I recommend an approach to run your backend service with output filtering that focuses on error and warning messages. Here’s how you can do this:

Start Your Application: Launch your backend application as you normally would.
Use Grep for Real-Time Monitoring: Pipe the output of your application to grep to filter for key terms like "error" and "warn". This can be done in a Unix-like terminal using the following command:
```
yarn start-backend | grep -E "error|warn"
```
This command allows you to see real-time log messages that could indicate issues with your plugin configuration.

By integrating these practices into your development workflow, you'll significantly enhance your ability to catch and respond to configuration errors on time.

Conclusion

Understanding these quirks is vital for a smooth transition to the new backend system. By ensuring consistent naming conventions, you can mitigate potential issues related to database and plugin management. Catching errors and warnings early can save you a lot of time in the upgrade process. Stay vigilant to avoid complications that could arise from discrepancies in your configurations.

Image by GenerativeStockAI from Pixabay

Update the Backstage catalog instantly without touching any YAML

Sam Blausten — Tue, 22 Oct 2024 08:59:18 +0000

Updating an entity in the Backstage catalog generally means manually updating a YAML file in a repository somewhere and getting it merged to the main branch.

For many, this manual process can feel like a series of hurdles—switching contexts, waiting for pull request reviews, and depending on other team members or systems.

This friction increases significantly when you scale across hundreds or thousands of repositories. For example, changing a group name might require chasing after numerous teams to raise and merge PRs, turning what should be a simple update into a months-long process.

Having to edit YAML makes keeping your catalog up-to-date a challenge, let alone enriching it with additional data or plugins like PagerDuty.

How long does it take to add a PagerDuty plugin?

Lets use the PagerDuty plugin as an example. Suppose you want to add PagerDuty monitoring to a Backstage entity representing a backend service. All you need to do is add an annotation to the YAML file with the relevant service ID.

The quickest way to edit the YAML file might seem simple: open GitHub, click the pencil icon on the About card, and start editing.

Manual Process in GitHub

Once inside GitHub’s web editor, you can make the change, commit it to a new branch, and open a pull request.

Sounds straightforward, right? But here’s where it gets tricky. To add the PagerDuty plugin, you need the correct service ID, which requires logging into PagerDuty, searching for the service that matches your entity, and then copying the ID from the browser’s address bar (since it’s not easily accessible in the UI).

In GitHub’s case, we can edit this immediately, commit it to a new branch and open a pull request rather quickly all in the web editor.

But to find the correct Service ID we must first go to PagerDuty. Hopefully we can log in and access the available services, and then search for the correct one using the names in the Component YAML.

Now, after locating the ID (in the address bar) and adding it to the YAML file, you still need to commit the changes, submit a PR, and wait for it to be reviewed and merged. Depending on the team’s review cadence, this could take days or longer.

Multiply this by hundreds of teams and thousands of repositories, and the process becomes incredibly slow and frustrating. Minor updates can take weeks or months to implement at scale.

A Better Way: Entity Updates with Roadie

At Roadie we’ve built UI based tools that allow you to set things like PagerDuty annotations with a few clicks. Instead of seeing a missing annotation card you’ll be able to select from a list of PagerDuty services and update the entity immediately, all from the same page in the UI.

Bulk updates via UI

But what if you need to update multiple entities at once? Roadie also provides a bulk editor that allows you to make updates across many entities using a simple table format. This dramatically reduces the manual effort required to keep your catalog accurate and up-to-date.

Behind the Scenes: How It Works

Our UI-based editors are powered by a backend feature we call Fragments. Fragments are partial entity data stored in a database table and merged into the existing entities in the catalog by a processor. These fragments can be updated through our UI or via API.

Leverage external APIs to make things easier

The custom UI we’ve seen earlier in this post also leverages external API’s to provide a dropdown of available options.

In this example we’ve used PagerDuty’s API to fetch a list of services using the API token already added to Backstage to make your PagerDuty plugin work. This call is made via the Backstage proxy which is set up to point to PagerDuty using your API Token on the backend.

Script bulk changes via API

However, sometimes bulk editing of metadata in the catalog is best scripted, and in this scenario, Roadie users make use of our Fragments API to make changes to entities, such as changing the ownership of a group’s entities when the group gets absorbed into another group, or updating any relationship references to a user entity when that individual leaves and someone else takes their role.

We’re not the only ones who’ve done something like this - companies like Twilio and others have built similar fragment functionality so that they can script updates to there catalog via API and free up platform teams to help keep the catalog data rich and accurate.

Reach out to us on Discord or our website’s chat messenger if you want to hear more about this and other improvements Roadie has made to the Backstage experience.

The Power of Customization: Making Backstage Work for You with Roadie

David Tuite — Thu, 17 Oct 2024 08:48:37 +0000

Backstage is the most flexible way to build an Internal Developer Portal that exists. The downside of this flexibility is that it can take a lot of effort to use. Roadie’s mission is to take the hassle out of Backstage, but we want to deliver on this goal without preventing our users from customizing Backstage to the fullest extent possible.

This article explains many of the customization options available in Roadie today. From look-and-feel customization like theming, to deep data model flexibility, custom scaffolder actions, and completely custom plugins, Roadie has the flexibility you need.

Backstage customizability

Backstage is not a Developer Portal. It’s actually a framework for building developer portals. You can think of it as being more like a Software Development Kit for building your own developer portal. Each company is expected to mix & match and extend the libraries that the Backstage community make available, so that they end up with a completely customized and unique IDP.

This fact is evident from day one of using Backstage. To get started, you don’t download and run a Docker container like you might expect. Instead, you run a command line tool to scaffold your own Backstage instance. Once that’s done, you write your own code to customize it.

-> npx @backstage/create-app@latest
? Enter a name for the app [required]: acme-corp-idp

Creating the app....

Another example of this flexibility is evident in the plugin architecture. The Backstage community has created hundreds of open-source plugins that can be installed into a Backstage-based IDP in order to provide visibility into the many many tools that might be used. Backstage is designed in a way that makes these plugins easy to install and configure.

From authentication hook points to source code management tool integrations to self-service scaffolder templates and UI components…. each and every part of Backstage can be ripped out and replaced and customized to your needs and desires. This is part of the philosophy of Backstage.

Roadie customizability

Despite being built on Backstage, Roadie is a complete, out-of-the-box, developer portal. We take away all of the work of building and maintaining your IDP, so that you can focus on getting value from the tool.

We believe that most organizations shouldn’t need to build a team of 4 or 5 engineers around Backstage. You should want most of your engineering efforts focussed on initiatives that deliver direct customer value, rather than building internal tools from scratch.

This is one of the reasons Roadie is delivered as Software as a Service. We want you to show up on day one and just start using it.

At the same time, we also want to retain the philosophy of Backstage. You need to be able to customize Roadie just as much as you would customize Backstage. You should to be able to connect your own authentication providers, integrate your homegrown tools, and mix and match the plugins that make sense to you. For this reason, we strive to make Roadie as customizable and extensible as we possibly can.

We believe that by combining Roadie’s ease of use with the flexibility and scope of the Backstage ecosystem, Roadie offers the best of both worlds, and is the best IDP on the market.

This post explores many of the major ways you can customize Roadie to meet your needs. Here are your options.

Theming and UI

Your IDP should look and feel familiar to your users, and it should focus their workflows in order to improve efficiency.

Logos and branding

Roadie let’s you replace the logos and branding across the site so that you can expose an IDP that matches your brand and themes.

This image shows the dark-mode version of a theme for a fictional company called BeautyBox.

Sidebar

Roadie’s sidebar configuration lets you cut the navigation down to focus it on the most important use cases in your company. If you don’t need self-service automation capabilities in your IDP then you can easily remove them to streamline things.

We also support reordering of sidebar sections, custom plugins in the sidebar, and deep links to important documentation.

Catalog customization

The Roadie catalog offers control over how the catalog is collected in the first place, and how it is represented to end users who need to consume information in the catalog.

Data model

Many engineering organizations have custom terminologies they use to describe their particular software development life cycle. If your company uses Domain Driven Design, you might want top level concepts like “Value Streams” in your catalog. Other companies might want to support groups of people called “Tribes”. Probably every company will want a list of “Products”.

Roadie supports a flexible and customizable data model that lets you rename existing Kinds and create your own types of entities.

The screenshot below shows:

Backstage’s Group kind renamed to “Teams”,
A custom entity type called “Products” has been created,
Unused kinds like Locations and Domains have been hidden from users.

Table columns

Different IDP users have different jobs, and that means they need different views. Roadie’s tables can be customized on a per user basis, so that everyone can streamline the UI to match the workflows they need.

Users can turn columns on and off, resize columns and change the density of catalog tables. The best part is that these settings are persisted so you can easily dive back into your workflow.

The launch announcement covered the benefits of Roadie’s catalog UI in more depth.

Entity providers

Custom entity providers are a way to provide entities into the catalog from external systems and existing data sources. They’re a critical integration point that helps ensure that your catalog will automatically stay up to date with external systems.

Roadie makes it easy to use custom entity providers via the Roadie Agent. The agent is a library that you can dump entities into in order to have them appear in the catalog.

Here’s a simple example where we use the Roadie Agent to dump an array of hardcoded entity metadata.

const { RoadieAgent, createRoadieAgentEntityProvider } = require('@roadiehq/roadie-agent');

const fakePayload = {
  type: 'full',
  entities: [
    {
      entity: {
        // Standard Backstage entity metadata omitted for brevity
      },
    },
  ],
};

const myEntityProviderHandler = async (emit) => {
  await emit(fakePayload);
}

RoadieAgent.fromConfig()
  .addEntityProvider(
    createRoadieAgentEntityProvider({
      name: 'testprovider',
      handler: myEntityProviderHandler
    }),
  )
  .start();

Auto-discovery settings

Any software metadata in YAML files should be auto-discovered by Roadie so that engineers don’t need to remember to register it via the UI.

Roadie supports custom auto-discovery settings, including glob matching.

Group and User ingestion

Roadie needs an organization chart in order to ensure that ownership of software can correctly be assigned to teams (aka. Groups), and to ensure that scorecard data is correctly rolled up through the org chart.

The best way to make sure the org chart is up-to-date and correct is to automatically sync it from a HR tool. To facilitate this, Roadie supports built in integrations for Microsoft Graph (also known as Microsoft Entry ID, or Azure Active Directory) and Okta. Other solutions can easily push Users and Groups into the catalog via our API.

Plugins

The real power of the Backstage ecosystem comes from its plugins. Roadie supports more than 70 Backstage plugins out of the box at the time of writing (September 2024). It typically takes us about a day to integrate a new open source plugin once it’s created by the community.

Plugin configuration

Most plugins have configuration options that Roadie admins need to be able to configure to suit their particular setup. This is easily done via admin panels we have built into the product.

Take the Argo CD plugin for example. Roadie supports two completely separate ways of integrating with Argo CD:

The app locator method allows you to dynamically search and identify Argo CD registered applications from multiple Argo CD instances.
The proxy method is used to construct links to individual Argo CD applications.

You can choose the best one for your needs, and even configure minute details like Namespace and Resource allowlists and blocklists.

The APIs of your Argo CD instances are probably not exposed on the public internet, but it’s still easy for Roadie to access them securely. We provide an open-source broker that runs in your infrastructure and makes an outbound connection to Roadie.

For other plugins, like the Kubernetes plugin and EKS, we support AWS Role Sharing as the preferred configuration method. Whatever your connectivity needs, we probably have a solution for it.

Of course Roadie’s responsibilities don’t begin and end with plugin configuration. We’re also:

Vetting plugins for quality issues as we integrate them,
Scanning plugins for licensing issues (which your legal team will be happy about),
Documenting plugins so people know how to use them,
Updating plugins as their authors release new features.

Frontend plugins

If there isn’t a community created plugin that suits your needs, or you need a plugin for a homegrown tool that only exists in your company, Roadie can support that too.

Users can write native Backstage plugins that run on Roadie and securely connect back to your private internal APIs via the broker.

The plugin development workflow supports the ability to locally host custom plugins and run them right on Roadie during development. This basically means that you can make a change to your plugin code in your IDE, and simply refresh Roadie to see your changes live.

We also support running production and development versions of the same custom plugin in Roadie at the same time, so you can iterate on your plugins without disrupting your users.

Productionizing a plugin is as simple as running the Roadie CLI against it to package it up and push it to Roadie.

roadie plugin:build --location /my-custom-plugin-folder/ --host https://static-assets.roadie.so/<my-tenant>/myCustomPlugin

The full range of Backstage frontend APIs are available to your custom plugins. You can query the catalog, check Role Based Access Control permissions, and even push analytics to see how people are using your plugin.

Proxies

Plugins frequently make use of proxies to upgrade requests with token authentication before forwarding them on to a backend service to retrieve data. Roadie users can create their own proxies inside the application. These proxies can be used to connect to third-party SaaS APIs like PagerDuty, or private internal APIs.

Here’s a user created proxy for SonarCloud.

Entity Pages

Entity pages are the pages in the catalog which pull together information on a particular piece of software, a team, or an infrastructure resource.

UI layouts

Adding plugins to Roadie interfaces is a simple matter of picking them off a list. This process is the same for both community created plugins (including plugins created by Roadie, plugins created by Spotify, and official plugins from companies like PagerDuty and Snyk) and for custom plugins you create yourself.

Once you’ve chosen your plugin, simply drag-and-drop it on the page, and resize it to suit your needs.

Different types of entities need different layouts. The Backstage Lighthouse plugin makes sense for a website, but not for a Kubernetes cluster. Roadie organizes the plugins in this way so that it’s easy to create custom views without too much setup. Our built in role based access control ensures that admins have the power to set up layouts, without overloading regular users with too many options. Team pages are customizable in the same way, they just typically have different cards and widgets.

Props on plugins

Sometimes individual cards will have their own editable properties. Roadie supports that too. For example, this card displays a list of recent GitHub Actions CI runs and their status. But how many recent CI runs should be displayed, and what layout should be used? Well, you can set that with the editable props.

This screenshot shows the edit mode of the Dynatrace Backstage plugin.

Metadata cards

We’re aware that many platform teams wish to display custom information in Roadie, but don’t necessarily have the TypeScript skills on hand to create custom plugins from scratch. To make this easier, Roadie includes no-code cards that can display information from custom metadata on the Entity.

Imagine we have a Backstage entity with some custom properties in its metadata.

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: sample-service
spec:
  type: service
  owner: group:roadiehq/engineering
  lifecycle: production
  custom:
    engineeringManager: user:dtuite
    productManager: user:samnixon87

How would we display this in the catalog? On Roadie, it’s as simple as adding the EntityMetadataCard and configuring it in a couple of clicks. The card even auto-links the Engineering Manager and Product Manager to their User pages in the catalog.

Home page

Of course it’s not just pages in the software catalog that can be customized. Roadie users can also customize the Home page plugin to display key information that’s relevant to them. This is the place to display your open pull requests, community news, your calendar, and other useful info that’s tailored to the logged in user’s experience.

Scaffolder

Roadie scaffolder supports the full range of customization that you would expect from the Backstage scaffolder.

Actions

Custom scaffolder actions are a way to execute homegrown CLI’s or arbitrary code as part of scaffolder templates. These are executed within your own infrastructure, which provides an added benefit in that it makes it easy to send internal network requests.

Users can use the Roadie Agent (the same library used in the Custom Entity Providers section above) to register custom scaffolder actions with Roadie.

RoadieAgent.fromConfig(config)
  .addScaffolderAction(
    createRoadieAgentScaffolderAction({
      name: 'custom-action', // The name of the action as defined in Roadie
      handler: async (ctx) => {
        try {
          fs.writeFileSync(
            `${ctx.workspacePath}/test.txt`,
            'new file with new contents',
          ); 
          // Writing a new file into the shared workspace
        } catch (err) {
          console.error(err);  // Local logging on the Roadie Agent process
        }

        let count = 0;
        while (count < 5) {  // Additional other actions that is wanted to be taken. This time looping for 5 seconds
          await new Promise((resolve) => setTimeout(resolve, 1000));
          count++;
          await ctx.log(`hello world`); // Sending a log message to be displayed to the end user
        } 
      },
    }),
  )
  // Add a second custom scaffolder action
  // .addScaffolderAction(...) 
  .start();

These can then be used in your templates like this:

steps:

  # This step executes on Roadie
  - id: fetchTemplate
    name: Fetch file
    action: fetch:template
    input:
      url: ./skeleton

  # This sends the workspace context to a self-hosted scaffolder runner
  # and executes the custom action we defined above. The payload passes
  # values to the self-hosted scaffolder runner and logs are written
  # back to Roadie.
  - id: invokeCustomAction
    name: Invoke a custom self-hosted action
    action: roadie:agent
    input:
      action: custom-action
      shareWorkspace: true
      payload:
        name: someValue

Field extensions

Custom field extensions give you the ability to build your own inputs and UI components for the scaffolder. You write them using the normal Backstage API, and push them into Roadie using the custom plugins pipeline.

Once registered, they can be used in Scaffolder templates by matching the name of the ui:field prop.

parameters:
  - title: Fill in some steps
    required:
      - name
    properties:
      name:
        title: Name
        type: string
        description: My custom name for the component
        ui:field: MyCustomInput

Authentication & Authorization

Single-Sign-On (SSO) setups

Roadie provides native support for any SSO provider you can imagine. Okta, Microsoft Entra ID, Google, AWS, Ping Identity…. you name it, we’re probably already using it in production.

Setting this up is a simple matter of sending our docs to your IT team, or whoever controls your SSO setup. We will then work directly with that team to get your authentication provider configured.

Once set up, your IT team can grant and revoke access to Roadie without being blocked by us. They have full control over the process.

Roles

Customers who have purchased our role-based access control add-on can define custom roles which can then be assigned to groups of users.

In this example, you can create a role which can only read the catalog. It wouldn’t be able to execute scaffolder templates or register anything in the catalog.

Roles can also be assigned via your Identity Provider like Okta or Microsoft Entra ID. This is especially useful when your IT team wishes to manage access and roles in a single place.

Permissions/policies (coming soon)

When role-based access control gets really powerful is with the ability to define custom permissions which can then be combined in roles. For example, perhaps you want to hide Component entities which are tagged with sensitive from everyone except the security team.

We’re working on this ability at Roadie and expect it to roll out in 2024.

Tech Insights (Scorecards)

Custom data sources

Data Sources are recurring jobs which gather data that can then be used to write checks against.

For example, Roadie comes with a built-in GitHub Settings Data Source which records facts about each repository in the catalog, like whether or not branch protection is turned on, or whether or not force pushes are allowed.

Users can also create their own Data Sources from scratch. At the time of writing, 7 different types of Data Sources are supported. Here are some examples:

HTTP Data Sources hit third-party APIs and pick values from the JSON response
Component Repository File Data Sources inspect files in the repository associated with each Component and record values from them.
Push Based Data Sources receive webhook events and store them for processing.

Creating Data Sources is a simple matter of filling out a few fields. You don’t need to write any TypeScript or YAML. Data Sources support advanced features like metadata variables, JSONata processing, inline testing, and error handling.

Custom checks

Checks inspect a value created by a Data Source and give it a pass or fail for each entity it applies to.

This check ensures there is a README.md in the repository associated with each piece of software.

Advanced features like boolean logic and live testing of the check are supported. Checks can link to documentation when they fail, or even link to a scaffolder template that can help the owner of a service to fix the failure.

Custom scorecards

Users can create their own scorecards that apply to the software in the catalog in order to communicate best practices and expectations to teams.

Each scorecard is a user defined collection of checks that shows up in reporting, in the catalog, and on team pages. Scorecard data is rolled-up through the catalog so that you can dip in at any level of the org chart and see aggregated data from below that point.

And there’s more coming

We’re constantly adding to the ways you can customize Roadie. If you want to stay up to date with the latest customizations, subscribe to our newsletter to hear what’s coming.

Image by GrumpyBeere from Pixabay

Backstage and Cost Insights: shifting cloud costs left

Jian Reis — Wed, 16 Oct 2024 07:31:48 +0000

Cloud computing - the double-edged sword

Managing cloud costs is fast becoming a strategic priority. While cloud service providers (CSPs) like Google Cloud, Microsoft Azure, and Amazon Web Services offer incredible flexibility and scalability, they can quickly lead to ballooning costs if left unchecked. We’ve all seen the cloud billing horror stories, but even for companies that have a good grip on their cloud costs, it’s easy to get into a situation where cost growth begins to outstrip associated revenue growth.

Such a position can undermine profitability, growth, and operational flexibility, ultimately putting the organization on an unsustainable trajectory. Beyond simple expense, runaway costs can mask inefficiencies in architecture, resource allocation, and service usage, creating a vicious cycle of wasted consumption. Left unmanaged, this can undermine even the most successful products, ultimately affecting their long term viability.

A notable case in point is Spotify, who encountered this scenario early on which forced the company to rethink its approach to managing cloud costs. Spotify realized that to get a handle on this, they needed to empower their engineers - the people closest to the actual cloud usage - to own the costs and be responsible for optimizing them. Rather than a top-down approach mandating a cost reduction, Spotify opted to make the engineers themselves responsible for their own cloud costs, an excellent example of the phenomenon of shifting cloud costs left.

Shifting cost awareness left: a cultural change

Shifting cost awareness left is a concept in effective cloud cost management that is gaining traction. Essentially a form of embedding cost considerations earlier in the engineering process, not as an afterthought for finance teams, that’s exactly what Spotify did, according to James Governor at RedMonk: “Spotify engineering teams are used to having a lot of autonomy, so the company couldn’t just introduce new cost guardrails as a top down concern. Therefore the Cost team tried to foster a culture where optimization would be fun.”

Shifting cost awareness left isn’t just about giving engineers tools, it’s a cultural shift where engineers take ownership of the financial impact of their work and are invested in cost optimization. When engineers are empowered with real-time cost insights, they become agents of change. Instead of waiting for an end-of-month cloud bill to identify costly inefficiencies, engineers can make informed decisions in real-time. This shifts cost management left from a reactive process driven by finance to a proactive, engineering-led initiative.

There’s a philosophical value here that goes beyond dollars and cents. By making cost awareness a core part of the development lifecycle, companies can drive a sense of ownership and even healthy competition among teams. Engineers begin to actively look for ways to optimize their cloud usage, often competing with each other to drive down costs. This culture of cost ownership not only saves money but also leads to better architecture and more efficient systems overall.

According to Janisa Anandamohan, Spotify Senior Product Manager, Cost Engineering:

We know engineers are natural optimizers when it comes to reliability, security, performance, et cetera. And now we’re telling them, hey, add costs into the mix. And they were super excited about that. We had many teams that were just able to tweak their services and data pipelines and to make them more efficient. And we know efficiency doesn’t just help cost, but helps reliability and performance as well. So we were getting double, triple wins along the way.

Spotify’s solution: Cost Insights

Spotify’s approach to shifting managing cloud costs left took the form of a cost optimization and visualization plugin called Cost Insights, on their own internal version of Backstage. The concept is simple: provide engineers with the tools to visualize and manage the costs associated with the services they build, all within the internal platform they are already using - their internal developer portal. By tying cloud costs to specific entities in the Backstage catalog, teams are empowered to take control of their spending and optimize resources more effectively.

This approach worked well for Spotify, partly due to their internal engineering culture, but mostly as a result of the significant engineering effort invested into getting Cost Insights integrated into their developer platform. While they have since open-sourced a pared-down version of the Cost Insights plugin back to the community, most organizations attempting to replicate their success will find it challenging, even when using their plugin. This is largely because of the technical complexity required, and because the current Spotify Cost Insights plugin, like many of the other aspects of the Backstage framework, is far from an out-of-the-box solution.

Just how much work would it take an engineering team, even working with the open-sourced Cost Insights plugin to implement a working solution? It’s a significant lift; here’s what they’d have to do:

Cloud billing integration: Set up a mechanism such as an API to pull cost data. This involves not only access configuration but also understanding the data schema of your cloud provider, which can be complex and time-consuming.
Backend development: Develop a backend to fetch, process, and store the billing data. This step requires designing a database schema that can handle potentially large amounts of data efficiently and setting up a server to run this service.
Cost Insights API implementation: Implement the API endpoints required by the Cost Insights plugin. This includes endpoints for fetching cost data, projecting future costs based on current trends, and breaking down costs by services, projects, or departments.
Frontend integration: Integrate the Cost Insights plugin into your Backstage instance. This may involve customizing the plugin to fit into your organization’s Backstage environment and ensuring it interacts correctly with your newly developed backend.
Testing and validation: Thoroughly test the plugin with real data to ensure accuracy. Validate the cost projections and insights with your finance team to ensure they align with actual expenditures.
Maintenance and updates: Continuously update the backend and frontend as cloud providers change their APIs or pricing models, and as new features or fixes become available in the Cost Insights plugin.

For a small team of three to four engineers, this could take anything from several months to a year to fully implement. This is a big lift for most organizations, which means that for many, this complexity keeps the solution remains out of reach. While the concept of cost transparency and shifting cost awareness left resonates, the time and effort required to implement and maintain a working Cost Insights tool deters widespread adoption.

As if it wasn’t hard enough, all of the work above assumes an organization is using only a single CSP in their stack. In reality, any organization that is using multiple CSPs (say, AWS and GCP) faces a thorny additional hurdle: the lack of data homogenization and normalization from CSPs. Each CSP often presents cost data in a different format, making it extremely challenging for organizations to consolidate and interpret this data in a unified manner if they’re using multiple CSPs.

Fortunately, the recent introduction of the FOCUS (FinOps Open Cost and Usage Specification) standard has changed the landscape for the better. Developed as a collaborative effort by the FinOps Foundation, the FOCUS standard aims to normalize the cost and usage data across different CSPs, providing a common format that makes it easier for organizations to integrate and analyze their cloud spending. This standardization is a crucial enabler, simplifying the data integration process and reducing the overhead associated with translating disparate data formats.

Introducing: Roadie’s Cost Insights Plugin

The adoption of the FOCUS standard significantly simplifies the integration of cost data across various cloud platforms. However, the complexity and effort of setting up Cost Insights is still high - which is where we at Roadie saw an opportunity to help. We recognized that the idea of empowering engineers to manage cloud costs was spot on, but the solution needed to be simpler, more accessible, and ready to use out of the box. As such, we’re in the process of refining and enhancing the existing Cost Insights plugin, making it significantly easier to deploy and use right out of the box.

Our enhanced Cost Insights plugin builds on Spotify’s version, but aims to reduces the setup friction, allowing engineering teams to access powerful cloud cost insights immediately - no custom backend development required. This streamlined approach means engineers can spend more time optimizing their services and less time managing the infrastructure for cost tracking.

With Roadie’s plugin, teams can quickly see which services are driving up their cloud costs, how those costs have trended over time, and what actions can be taken to reduce unnecessary spending. The key here is not just providing data but making it accessible and actionable, so engineers can immediately use it to make decisions.

Roadie’s Cost Insights plugin takes full advantage of the FOCUS standard, ensuring that the data we present is both accurate and comparable across providers. By eliminating discrepancies in how cost data is reported, the plugin enables engineers to gain a clear understanding of their cloud usage without needing to reconcile different data formats.

The ability to assign costs to specific entities within a Backstage catalog is what we’re most excited by. In traditional cloud billing tools, costs are often presented at a very high level, making it hard to drill down and see what’s truly driving the expenses. By associating costs with individual services and the teams responsible for them, the plugin makes cloud costs real and relatable for developers. When a team sees exactly how much their service is costing, they can no longer ignore it or avoid responsibility - it becomes part of their job to optimize those costs.

The Roadie Cost Insights dashboard displaying cloud cost by product

How it works

Our Cost Insights plugin, currently in internal beta, integrates seamlessly with a Roadie-managed Backstage instance to provide teams with an out-of-the-box solution for tracking and managing cloud costs. Key features include:

Easy setup: No complicated setup required - simply connect to your cloud service provider to Roadie Cost Insights via your cloud administration settings or via a secure broker, and allow the Roadie Cost Insights compatibility layer to take care of all the data modeling and translation.
Project and group-based cost tracking: Track cloud costs at the entity, team, or product level, and drill down into specific projects or groups for more detailed insights.
Trend visualization: View cloud cost trends over time, breaking down data by dimensions like products, services, and regions to identify patterns and anomalies.
FOCUS standard integration: The plugin leverages customer-provided FOCUS data, ensuring consistency and like-for-like comparability across cloud providers.
Actionable insights: Dashboards help engineers take ownership of their services, and immediate action by optimizing those services or reducing over-provisioned resources.

For example, a team might notice that the costs of a particular product are rising faster than expected. With Roadie’s plugin, they can drill down into the cost breakdowns, identify high-cost services, and take corrective action - such as optimizing resource allocations or reducing usage.

Roadie Cost Insights also supports multiple regions, meaning a DevOps team managing multiple cloud services in different regions could use Roadie’s Cost Insights to track cloud costs by region. For instance, they could spot that their compute resources in one region are significantly more expensive compared to others. Using the plugin’s trend visualization and breakdown capabilities, the team can identify the services or instances driving up the cost and adjust their architecture to either reduce or move resources to more cost-effective regions.

Roadie Cost Insights Architecture - note that the user configuration overhead is limited to installing the Cost Insights client on the cloud service provider infrastructure (or the use of a broker client) while Roadie takes care of all the backend setup.

The future of Roadie’s Cost Insights Plugin

We believe that by simplifying cost management and putting it directly in the hands of engineers, companies can not only save money but also drive innovation, efficiency, and alignment across their teams.

Cloud cost management is no longer just a financial issue—it’s an engineering one. By providing real-time insights into cloud spending and shifting cost awareness left, Roadie’s Cost Insights plugin helps companies empower their engineering teams to take ownership of their cloud usage. This leads to smarter decisions, lower costs, and more efficient systems.

While Cost Insights is still currently in an internal beta, if you’re interested in learning more or becoming an early design partner, get in touch with us today. We’d love to work together to bring the future of cloud cost management to your team.

Title image by Brian Penny from Pixabay

The Ultimate Guide to Backstage Software Catalog Completeness

David Tuite — Wed, 09 Oct 2024 10:10:00 +0000

Internal developer portals (IDPs) like Backstage and Roadie are, at their core, software catalogs. The software catalog is the backbone upon which much of the other functionality hangs, and building a complete catalog is vital to the success of the IDP project.

Without a complete catalog, an IDP cannot fulfill it’s primary purpose: improving developer productivity. It cannot drive improved discoverability if the software that teams are trying to discover is not in there. It cannot help to measure the standardization or security posture of software it does not know about.

This article is a comprehensive guide to catalog completeness in Backstage. Roadie is built on top of Backstage, and so all of the same lessons apply. We cover why it’s important, how to measure it, and how to achieve it.

First, let’s learn why you want to build a complete catalog in the first place.

Why building a complete catalog is important

The primary reason companies deploy an IDP is because they want to make developers more efficient. A big part of this is helping teams answer basic questions about the software around them. They want to make it easy to answer questions like “do we have a geocoding service?”, “which team owns the checkout service?” and “where is the API spec for the users service?”. These questions can only be answered by the IDP if the geocoding service, checkout service, and users service are registered there in the first place.

A complete catalog is essential when using an IDP to measure the maturity of the software that teams are producing. Applying scorecards to software in the catalog of the IDP can be a great way to determine the security posture of each production service. But scorecards can only apply to software which is actually in the catalog.

The software catalog is also the fundamental unit of navigation in the IDP. The lack of a software catalog is the reason that wikis like Confluence or Notion cannot solve the discoverability problem that IDPs solve. In a wiki, content is organized into pages, which are intentionally unstructured and flexible. In an IDP, content is organized by Component (think “piece of software”) and is structured so that the same information is available for each Component.

How to measure catalog completeness

Having a complete catalog doesn’t necessarily mean that every piece of software is in the catalog. Most organizations have their share of abandoned code. It may have been created during hackathons or for brief experiments that never saw production usage. Having all of this stuff in the catalog can create clutter and noise.

There may also be parts of the organization for whom it doesn’t make sense to be in the catalog. We work with companies which have large hardware divisions who write code for embedded devices. They don’t necessarily follow the DevOps lifecycle and thus are sometimes intentionally omitted.

Production software is the most important stuff to have in the catalog. It’s the software that most engineers work with most of the time. Production software will have the most frequently referenced APIs and docs and it’s much more important to have an understanding of the maturity of the software that runs in production environments, since it has the most attack vectors.

For this reason, we frequently see customers create a list of software which is deployed to production by referencing ArgoCD or some other deployment tool. They then compare this against software in the catalog, frequently by accessing the Roadie APIs or by using our CSV export functions.

The next best bet is to compare number of pieces of software (aka. Components) in the catalog against the number of active, unarchived repositories in your source code solution. This will never give you a perfect answer, because “pieces of software” don’t necessarily map perfectly one-to-one to repositories (monorepos etc), but it’s a good start.

How Roadie Helps

At Roadie we give every customer a chart which shows the percentage of their active (non-archived and received a commit in the past 12 months) repositories against the number of Components in the catalog.

Roadie customers can achieve high catalog completeness

Achieving a high level of catalog completeness is definitely possible.

The majority of established Roadie customers are happy with their level of catalog completeness and we have many customers who have a catalog completeness level (measured as a percentage of active repositories which are registered in the catalog) which is above 80%.

Here’s a chart showing catalog completeness for Uplight who onboarded in September 2023. Four months later they were at 88% catalog completeness, with more than 600 components in their catalog.

Here’s another Roadie customer who increased their catalog completeness from 40% to 90% over a period of four months. They now have more than 750 components registered.

There are countless more examples like this amongst Roadie customers. The goal of this post is to help all companies achieve the same results.

Strategies for building a complete catalog

There are multiple strategies that can be used to build a complete catalog. The strategy that is right for your company can depend on factors like the size of the company and the enthusiasm of the developers to have an IDP.

You may also want to run multiple strategies in parallel, or start with one in order to get the bulk of software into the catalog, and switch to another to get closer to full catalog completeness.

Think about expanding catalog completeness like the layers of an onion. Start with the most enthusiastic early adopters and get them onboard. Then use them as an example as you expand out to the rest of the organization.

We recommend you import users (aka. employees) and teams (called Groups in Backstage nomenclature) into the catalog before creating any software components. You will ideally want to assign ownership of each component to a team as you import it. This task is much easier if all of your teams already exist in the catalog.

If you don’t know which strategy to choose…

Experiment! You don’t need to roll out to the whole organization in one go. Pick some friendly teams, run one strategy on each team, and analyze the results. Did you end up with software in the catalog? What is the feedback from each team? Where did they get stuck? Take this feedback and use it to improve the process before expanding out to more teams.

Strategy 1: Centralized automated

This strategy involves connecting to a chokepoint in the organization in order to push software metadata into the catalog programatically. It does not require anyone to write catalog-info.yaml files.

Frequently, the initial chokepoint will not have all of the information required to build a useful catalog. When this happens, the software metadata must be enriched after the fact.

Depending on your tech stack and permission settings, following options may be viable:

An ArgoCD instance which does deployments to the production environment. It has knowledge of the most critical software in the org (software that goes to production) and can thus can be a good starting point for the catalog.
A Helm chart which is used by a large percentage of the deployable software in the organization.
A centrally owned CI job or build tool which can be written by the centralized team and applied to software which is owned by other teams in the organization. For example, Lunar Bank have talked about how they populate their catalog from their build tool called shuttle.
A legacy software catalog, developer portal or spreadsheet.

Pros

The software catalog can quite quickly be bootstrapped to a highly complete state.

Cons

The ownership link between software and teams is not immediately established.
Companies with no source of truth or a heavily fractured deployment ecosystem may not be able to implement this strategy.
The product teams will be less well educated on the value of the IDP when the process is finished.

Tactics to prioritize in order to succeed with this strategy

Use custom entity providers
Make catalog presence a requirement for deployment

Strategy 2: Centralized manual

This strategy tries to avoid asking the individual product teams to do work. Instead, the centralized team takes it upon themselves to populate the catalog. They may do this in collaboration with the product teams, but they likely won’t ask them to write any YAML.

Pros

This strategy is likely to be faster than the distributed manual strategy, especially for companies which don’t have thousands of microservices.

Cons

The product teams will be less well educated on the value of the IDP when the process is finished.
It may be difficult for the centralized team to gather enough data about each individual software component.
It becomes the centralized teams job to manually keep the catalog up to date.

Tactics to prioritize in order to succeed with this strategy

Store software metadata in a single repository
Open scripted pull requests
Customize the catalog nomenclature

Strategy 3: Distributed manual

This strategy involves asking all of the individual product teams to register the software that they own in the catalog.

Typically, the central team who own the IDP will meet with and educate the product teams on the value of the IDP, either on an individual basis, or in larger groups. The central team will provide tools and materials to the product teams in order to teach them what they need to do to get their software into the catalog (typically create a catalog-info.yaml file), and give them clear steps to take in order to achieve catalog completeness.

Pros

Product teams are more likely to feel a sense of ownership over their software in the catalog because they put it there in the first place.
Product teams have an opportunity to learn about features of the IDP as they are registering their software. They may then choose to use features in the IDP such as technical documentation.

Cons

This strategy requires a lot of work from the central team in order to yield high catalog completeness. It will take time. They will need to educate and continually follow up with product teams throughout the company.

Tactics to prioritize in order to succeed with this strategy

Give teams a scaffolder template to register software
Share catalog completeness numbers publicly
Tie catalog completeness to a wider initiative
Write custom plugins to create value

Tactics for building a complete catalog

The tactics you need to use will depend on the strategy you are implementing. This is a full list of all the tactics we know. Please refer to the strategies above in order to know which tactics to choose.

Some tactics will apply regardless of the strategy that is chosen. They are:

Customize the catalog taxonomy
Move onboarding docs into Roadie
Present on the value of Roadie to people managers
Tie catalog completeness to a wider initiative

Each of the tactics mostly fall into one of the following categories:

Reduce friction for developers who want to get their software into the catalog.
Create incentive for developers to put their software into the catalog.
Educate developers on how to use the catalog.

Give teams a scaffolder template to register their software

Category: Reduce friction

This involves writing a scaffolder template that teams can use inside Backstage in order to create a catalog-info.yaml file in their repositories.

The scaffolder template will ask the user to fill out some information about their software, typically by picking from pre-defined values, and will open a pull request against a repository when finished. Once the user reviews and merges the pull request, auto-discovery will pick up the catalog-info.yaml file and populate the catalog.

Benefits

Individual developers don’t need to understand the (long) Backstage YAML API spec in detail.
The owners of the IDP can use the template to constrain the “type”, “lifecycle” and other properties that is assigned to each software component. This puts guardrails in place that will help create more consistency in the catalog. Catalog consistency is important, and will help you avoid problems in future.
The form can integrate with external APIs to pull in sensible options. For example, in the screenshot above, the “Component Owner” is a selection of all the engineering teams in the company. The user doesn’t need to type an exact string.

How Roadie helps

Roadie provides a starting point for a software registration template in the getting-started repo. Customers can fork it into their own GitHub org, edit it to meet their needs and import it into their own Backstage instance.

Archive unused repositories

Category: Reduce friction

If our measure of catalog completeness is:

then we can increase catalog completeness by archiving old repositories that nobody is using.

It may sound somewhat silly, but every organization has abandoned hackathon projects and old test repos that are unused and simply causing clutter. By archiving them, we clean up our source code management tool while improving catalog completeness.

Believe it or not, Roadie has one customer who increased catalog completeness from 45% to 75% just by archiving repositories.

How Roadie helps

Our catalog implementation ensures that Components are removed from the catalog when the repo they reference is archived.

Customize the catalog nomenclature

Category: Reduce friction

By mapping the Kinds of entity that show up in the catalog to familiar concepts in your company, you can create instant recognition for developers who land in the catalog.

For example, if your company has a concept of “Valuestream” then make this front and center in the catalog so that users instantly understand what they’re looking at.

Benefits

Users can orient themselves quickly and get value rapidly.

How Roadie helps

Roadie customers can use our admin interfaces to customize and rename the core catalog concepts, and create completely new ones.

Write custom plugins to create value

Category: Create incentive

Custom plugins provide value for product teams by giving them faster ways to perform bespoke workflows inside the catalog. By creating custom plugins, a central team can incentivize developers to add their software into the component.

For example, Lunar Bank have custom plugins for dealing with dead letters in RabbitMQ. These plugins are regularly useful for developers. This causes them to visit the catalog to use the plugin.

Benefits

Custom plugins are quite simple to produce and can quickly create value for software engineers.
Custom plugins unlock tailored value for engineers to help them do things more quickly or more easily than they otherwise could. They sometimes even allow them to perform a task that they cannot do at all outside of Roadie or Backstage.

How Roadie Helps

Roadie provides an interface for registering and managing custom plugins. It also facilitates live reloading of custom plugins, the ability to run multiple versions of a plugin side by side, and a scaffolder template to bootstrap a custom plugin monorepo. Custom plugins on Roadie can securely connect back to a private network to load data from internal APIs. Check out our docs to learn more.

Move onboarding docs into Roadie

Category: Educate

Engineer onboarding docs are typically used to help a new engineer to set up their environment and get to productivity quickly. Expedia Group, Spotify and other Backstage adopters have successfully used TechDocs and scaffolder templates to speed up engineer onboarding and help new engineers become familiar with the IDP on day one. The exact same tactic can be deployed on Roadie.

Expedia Group put 850+ engineers through their Backstage based bootcamp in 2022. They discuss it in their case study on the Backstage website.

Benefits

Newly onboarded engineers start using Roadie on day one. They get used to it and understand how to come back.
Engineers learn how to use a scaffolder template to create a new service during onboarding. This service is added to the catalog, and they learn how the catalog works.

How Roadie Helps

Roadie supports standalone TechDocs that are not tied to a particular software component in the catalog. These are perfect for onboarding docs.

Present on the value of Roadie to people managers

Category: Educate

Roadie provides specific value to managers, directors and VPs that is different than the value that developers might care about. By educating managers on the value they will receive, you can encourage them to work with their teams to get their software into the catalog.

For example, did you know that frequent Backstage users at Spotify are 5% more likely to be at the company one year later. Retention is important for managers, so they need to know about this.

How Roadie Helps

Roadie provides Scorecards which can help managers ensure that their teams are producing mature and high quality software. This feature is not available for open-source Backstage.
Roadie gives customers value calculators to help them estimate the dollar value they can expect to receive.

Open scripted pull requests

Category: Reduce friction

Opening an automated pull request containing a catalog-info.yaml file is a good way to ease the burden on developers who want to get their software into the catalog. All they need to do is edit the pull request a little bit, review it and merge it.

Keep in mind that your script will need permissions to open a pull request against a majority of repositories in your source code management tool. Depending on your security model, this may not be possible.

This method can work especially well in companies that operate out of large monorepos. A monorepo setup allows the generation of a single pull request that can add many catalog-info.yaml files in one go. It can also be reviewed and merged by a single person with elevated permissions.

While this is a tempting option to quickly build a catalog with YAML files, we have seen customers experience issues with catalog correctness when they use this method. Some teams may blindly merge the pull request without validating the information that it contains. This tactic should be executed alongside an education program to teach teams what to do. Go slowly and experiment.

How Roadie Helps

Roadie provides a token authenticated API which the centralized team can use to tell which repositories are already in the catalog. A simple script can consume this to open a PR against the repos which are not already accounted for.

Our solutions engineering team can work with you to write a simple script that will open a pull request containing a YAML file into each repository.

Make catalog presence a requirement for deployment

Category: Create incentive

By making catalog presence mandatory for deployment, platform teams can be confident that they have all of the important software in the catalog.

In the early days of Backstage at Spotify, teams could not SSH into their machines unless their services were in the catalog. The catalog owner was referenced to determine who was and was not allowed to access the machine.

This tactic works best when the IDP is orchestrating a new greenfield platform that other teams are migrating onto. Adding the catalog-info.yaml file can be one simple step in what is likely a series of steps that teams have to do to migrate. Outside of this situation, it can be politically problematic to block deployments due to a missing YAML file.

Automate catalog collection with custom entity providers

Category: Reduce friction

If developers find writing YAML files tedious, potentially the best thing to do is to make them optional. Backstage’s custom entity providers allow adopters to programmatically shovel software entries into the catalog. Custom entity providers are a great way to connect Backstage to an existing source of truth for catalog data, such as a legacy IDP, an ArgoCD instance, a kubernetes cluster, or a CICD tool.

How Roadie helps

Roadie gives customers the roadie-agent. This wraps the custom entity provider concept with a secure connection to Roadie so that customers can easily dump software metadata into the agent and have it appear in the catalog.

Roadie has an Entity Provider API. Simply push an array of software metadata to this endpoint and it will appear in the catalog. To update the metadata, simply push again.

Frequently the programatic source of truth will have some but not all of the metadata that the catalog needs. For example, it may be missing the name of the team who owns the software. Roadie allows users to decorate software in the catalog with extra metadata within the UI. This ensures that the catalog can become complete and rich over time.

Share catalog completeness numbers publicly

Category: Create incentive

One great way to get people bought into the idea of building a complete catalog is to make it a group effort. By transparently reporting on the completeness and “health” of the catalog, a shared sense of ownership over the goal can be created.

Twilio explained how they do this in their catalog at the Autodesk Developer Productivity Summit.

How Roadie Helps

Roadie gives all Tech Insights customers an out-of-the-box measurement of catalog completeness.

We also report on important aspects of catalog richness, like the percentage of pieces of software in the catalog which have an owner assigned. Customers can use these building blocks to measure other attributes of the health of their catalog.

Tie catalog completeness to a wider initiative

Category: Create incentive

Companies usually want a complete catalog so that they can accomplish some wider engineering goal. By promoting catalog completeness in service of this wider goal, teams can better understand why it is important to be in the catalog, and why they should help.

For example, we recently worked with a customer who needed a complete and correct list of all software that had access to their users personally identifiable information (PII). By attaching to this company wide goal and leveraging the influence of the company CTO, the company was able to rapidly label hundreds of software components in Roadie with their PII status.

How Roadie Helps

We run regular customer success meetings with customers to help them identify wider engineering initiatives where Roadie can help accomplish the goal more quickly or more easily. We will then work with teams in order to project manage the delivery of a solution.

Do a whiteboarding session

I did some eduction with a team at [company] where we brainstormed the services they wanted to cover and their relationships, this was a whiteboard exercise and I then created the PR's and had them review.

When Paddle started with Backstage, they realized that they didn’t have an existing source of truth they could lean on to populate their catalog, and they would have to collate it manually.

They started with a whiteboarding exercise so they could iterate quickly. Meeting with each team in small groups, Ioannis Georgoulas (Director of SRE), led the process. He first spent time brainstorming the services that the groups wanted to catalog, and defining their boundaries and the relationships between them. This information was all collected in a simple document to start.

Once he had a solid understanding of the service map, Ioannis opened pull requests against each repository with the catalog-info.yaml file that was needed. All the teams had to do was review and merge it. Because they had participated in the process to gather this information, they were already bought in and could understand the value of it.

How Roadie Helps

We provide frequent customer success calls with every customer through the initial stages of implementation. We’ll partner with your implementors to run these whiteboarding sessions and gather the information you need to be successful.

Key takeaways/Conclusion

Building a high level of catalog completeness in Backstage need not be intimidating. By carefully choosing the strategy that will work at your company, expanding outwards from the most eager adopters first, and communicating widely as you go, you can reach a high level of catalog completeness in a short amount of time.

Two topics we have not discussed yet, are catalog richness and completeness. These areas go hand in hand with completeness, and work together to ensure that your IDP has the answers that developers need, when they need them.

We’ll be covering richness and completeness in another article. If you want to be among the first to read it, make sure to subscribe to the newsletter below.

Easier Relationship Mapping in the Backstage Catalog

Sam Blausten — Mon, 07 Oct 2024 07:45:17 +0000

In the Backstage software catalog, relationships provide a glue between the different software, systems, resources and people in your organisation. They can allow you to easily answer questions which otherwise might be much harder to find out in a large or rapidly growing organisation.

Plugins like the Catalog Graph Plugin allow powerful visualisation of these relationship graphs.

Similarly, relationships can become a core piece of information displayed on entity Overview pages.

Relationships can be added in one of two ways - via a YAML file update in an SCM repository, or in a third party source like GitHub Teams or Okta. A YAML update looks something like this:

kind: Component
metadata:
    name: identity-backend
spec:
    ...
+   dependsOn:
+       - component:default/users-backend
+       - resource:aws/identity-backend-ec2

Restrictive Relationships

There are a very limited and restrictive set of allowed relationships between different Kinds in Backstage by default. Knowing what is allowed requires looking up the Backstage documentation each time to check the schemas.

For instance, a Domain cannot “depend on” anything in the default Backstage relationship model. However, maybe you have a domain like Shipping that depends on another like Identity. The Identity domain exposes APIs for customer addresses that the Shipping domain breaks without access to. You might even want to directly model the hard dependency on specific external API’s inside the Identity domain so that its easy to identify issues and notify the right people in charge of the Shipping domain when something breaks in those APIs.

Confusingly, many of the default relationships terms that can be used in the YAML spec definition don not map directly to the actual relationship in the catalog. For instance if you want to add a partOf relationship for a Domain to another Domain, you would need to use subdomainOf . Adding it as an explicit partOf field would not work unless you add that in the extended processor as we’ll see shortly. Each Kind has its own differing semantics for each relationship type, which needs to be referenced and checked to ensure a relationship is correctly added.

Expanding available relationships

Luckily its an easy engineering task to add a processor that can handle more expansive relationships and even new terms for those relationships such as manages/managedBy for Users. This processor would run in addition to the BuiltinKindsEntityProcessor that does the default relationship processing for the catalog and would look something like this:

import { CatalogProcessor, CatalogProcessorEmit, processingResult } from '@backstage/plugin-catalog-node';
import { Entity, getCompoundEntityRef, parseEntityRef, RELATION_DEPENDENCY_OF, RELATION_DEPENDS_ON } from '@backstage/catalog-model';
import { LocationSpec } from '@backstage/plugin-catalog-common';
import { RELATIONSHIP_MANAGED_BY, RELATIONSHIP_MANAGES } from '../constants';

export class ExtendedRelationshipsProcessor implements CatalogProcessor {
  getProcessorName(): string {
    return 'ExtendedRelationshipsProcessor';
  }

  postProcessEntity(
    entity: Entity,
    _location: LocationSpec,
    emit: CatalogProcessorEmit,
  ): Promise<Entity> {
    const selfRef = getCompoundEntityRef(entity);
    function doEmit(
      targets: string | string[] | undefined | null,
      context: { defaultKind?: string; defaultNamespace: string },
      outgoingRelation: string,
      incomingRelation: string,
    ): void {
      if (!targets) {
        return;
      }
      for (const target of [targets].flat()) {
        const targetRef = parseEntityRef(target, context);
        emit(
          processingResult.relation({
            source: selfRef,
            type: outgoingRelation,
            target: {
              kind: targetRef.kind,
              namespace: targetRef.namespace,
              name: targetRef.name,
            },
          }),
        );
        emit(
          processingResult.relation({
            source: {
              kind: targetRef.kind,
              namespace: targetRef.namespace,
              name: targetRef.name,
            },
            type: incomingRelation,
            target: selfRef,
          }),
        );
      }
    }

    if (entity.kind === 'Domain') {
      doEmit(
        entity.spec?.dependsOn as string[],
        { defaultKind: 'System', defaultNamespace: selfRef.namespace },
        RELATION_DEPENDS_ON,
        RELATION_DEPENDENCY_OF,
      );
      doEmit(
        entity.spec?.dependencyOf as string[],
        { defaultNamespace: selfRef.namespace },
        RELATION_DEPENDENCY_OF,
        RELATION_DEPENDS_ON,
      );
    }

    if (entity.kind === 'User') {
      doEmit(
        entity.spec?.managedBy as string,
        { defaultKind: 'User', defaultNamespace: selfRef.namespace },
        RELATIONSHIP_MANAGED_BY,
        RELATIONSHIP_MANAGES,
      );
      doEmit(
        entity.spec?.manages as string[],
        { defaultKind: 'User', defaultNamespace: selfRef.namespace },
        RELATIONSHIP_MANAGES,
        RELATIONSHIP_MANAGED_BY,
      );
    }

    if (entity.kind === 'Group') {
      doEmit(
        entity.spec?.managedBy as string,
        { defaultKind: 'User', defaultNamespace: selfRef.namespace },
        RELATIONSHIP_MANAGED_BY,
        RELATIONSHIP_MANAGES,
      );
    }

    return Promise.resolve(entity);
  }
}

Questions to consider

There is a rational that restricting the available relationships for each kind in the catalog prevents misuse of relationships by users adding things to the catalog. However it is worth bearing in mind that relationships in the catalog are mostly just syntactic sugar over the same link between two entities. There is nothing different about dependsOn and partOf in Backstage except the perceived meaning of these terms.

Educating users

Easily accessible documentation on standards and definitions is the best way to educate contributing users to your Backstage catalog. Backstage has its own schema docs and descriptions on existing relationships but you can also host internal documentation in a more compact format that is easier to reference quickly.

What we did to fix it

At Roadie, we’ve added expanded relationships for all kinds in the catalog to make it easier for people to correctly add relationships using the terms they understand. We’ve documented them in a compact format and are working on a UI based editor for these relationships so that users do not have to manually edit each YAML file and go through a PR process to build up a comprehensive mapping of dependencies in an organization.

We've also added a new card that can show all relationships for an entity, regardless of what kind of relationship it is in list format.

Contact us via Discord on on this website's chat to find out more about what Roadie can help with or to talk to our engineers about this topic or other issues you might be having adopting Backstage.

Improving Backstage performance (by up to 48x)

Brian Fletcher — Fri, 04 Oct 2024 07:57:19 +0000

Backstage is an excellent framework for building an internal developer portal. It provides all of the fundamental building blocks to improve developer experience in an organization.

Core to Backstage is its Catalog of entities. The Catalog provides a database of software components, resources, libraries, and other kinds of software items. It provides client code and an API backend to retrieve items in the catalog, along with the software interfaces required to populate the entity catalog. Its model is flexible, customizable, and powerful.

However, with great power comes great responsibility. Without experience, it's easy to develop anti-patterns in Backstage catalog usage. These anti-patterns can then turn into major performance issues at scale. This in turn leads affects trust and usage of the product as a whole.

At Roadie, we provide an out-of-the-box version of Backstage for our customers. We have come across many of the ways in which non-optimal Catalog client usage can affect performance of the application as a whole. We have seen these performance issues result in lagging page loads and (in extreme cases) causing page loads to fail in Backstage.

By applying the patterns explained in this post, you could see a huge improvement in Catalog response time. In some cases, you may even see Catalog queries perform 48x faster!

Architecture of the Entity catalog

The entity catalog in Backstage is made up of three components. A Catalog client, the Catalog backend, and the Catalog database. When Backstage starts up for the first time, it will have a Catalog database and a catalog backend. When you visit the Backstage application in your browser, it will make use of the Catalog client to retrieve data from the Catalog backend. The Catalog backend in turn retrieves the requested catalog items from the Catalog database.

Using the Backstage Catalog Client

Soon after deploying Backstage in an organization, users will want to customize it.

Frequent customizations we come across include loading entities into the Catalog from an in-house platform or visualizing data from an internal system in the Backstage UI. Customization is normal and is a sign that Backstage is adding value for teams.

When developers write extensions to Backstage, it's likely they will come across the need to interact with the Catalog. There are two ways they can do this:

via a Frontend Backstage extension
via a Backend Backstage extension

To make use of the Catalog client in a frontend Backstage extension, you are likely to be using the useApi hook, along with a useAsync function.

import { catalogApiRef } from '@backstage/plugin-catalog-react';
import { useApi } from '@backstage/core-plugin-api';
import { stringifyEntityRef } from '@backstage/catalog-model';
import useAsync from 'react-use/lib/useAsync';

export const CustomReactComponent = () => {
  const catalogApi = useApi(catalogApiRef);
  const {
    value: entities,
  } = useAsync(async () => {
    const response = await catalogApi.getEntities();
    return response.items || [];
  }, [catalogApi]);

  return (<>{entities.map(stringifyEntityRef).join('\n')}</>)
}

In a backend Backstage extension, you are likely to be constructing the Catalog client using the discovery client. The discovery client is a helper that allows plugins to discover the API location of other clients.

Generally, if you are writing a backend plugin, like a new REST API or a Catalog processor, you will have access to the discovery client. Depending on your particular situation, you may have access to the discovery client in a different way.

import { CatalogClient } from '@backstage/catalog-client';
import { DiscoveryApi } from '@backstage/core-plugin-api';

export const getAllEntities = async (discovery: DiscoveryApi) => { 
  const catalogApi = new CatalogClient({
    discoveryApi: discovery,
  });

  const response = await catalogApi.getEntities();
  return response.items;
}

You will notice that once you have an instance of the CatalogApi, it is used in the same way in either the frontend or a backend extension.

(await catalogClient.getEntities()).items;

Check the Backstage docs for a more comprehensive explanation of the full Catalog interface.

How big can a Backstage Catalog get?

When thinking about Catalog size, it’s useful to think about two things:

How big is each individual entity?
How many entities do you have?

A typical Entity

A typical Backstage Entity looks like this:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: artist-web
  description: The place to be, for great artists
  labels:
    example.com/custom: custom_label_value
  annotations:
    example.com/service-discovery: artistweb
    circleci.com/project-slug: github/example-org/artist-website
  tags:
    - java
  links:
    - url: https://admin.example-org.com
      title: Admin Dashboard
      icon: dashboard
      type: admin-dashboard
spec:
  type: website
  lifecycle: production
  owner: artist-relations-team
  system: public-websites

It describes a website called the artist-web. It has a few basic Backstage entity properties, and some annotations, tags, and links. It's encoded in YAML here, but in Backstage's database, it is stored as plain text in JSON format.

Uncompressed, this entity definition is about half a kilobyte. Therefore, a Catalog containing about 20,000 similarly sized entities would add up to about 10 megabytes of data uncompressed. That's a pretty big chunk of data to be sending over the wire.

However, we haven't seen anything yet…

The Backstage Catalog model defines an API Kind. These are used to document the endpoints that services make available in Backstage. API entities often contain an embedded OpenAPI doc.

For example:

apiVersion: backstage.io/v1alpha1
kind: API
metadata:
  name: artist-api
  description: Retrieve artist details
spec:
  type: openapi
  lifecycle: production
  owner: artist-relations-team
  system: artist-engagement-portal
  # The embedded OpenAPI spec is in the definition
  definition: |
    openapi: "3.0.0"
    info:
      version: 1.0.0
      title: Artist API
      license:
        name: MIT
    servers:
      - url: http://artist.spotify.net/v1
    paths:
      /artists:
        get:
          summary: List all artists
    ...

At Roadie, we have seen multiple customers with API kind entities in their Catalog with embedded OpenAPI docs as large as 1 megabyte in size. It's easy for even a small-sized engineering organization to have 50 such APIs documented in Backstage. Unoptimized, that's 50MB+ of data that's being transferred every time we query the full Catalog.

This Catalog size is important because poor use of the Catalog APIs can cause huge database queries and API response sizes to result, which will cause both unwarranted traffic across the network and unwanted wasted time transferring, encoding, and decoding that data.

How to Make Good Use of the Entity Catalog

With all this said, we wanted to run through some good practices that are going to help with improving the Backstage experience. The examples we use in the tables below are measured in the browser on a production Catalog that has 14k entities.

Unoptimized, we're looking at 2.16 seconds and 59.5 MB of data. That's our starting point. Now each experiment we do below is going to improve that data.

Only query the entities fields that you need

By default, when retrieving entities from the Backstage Catalog, Backstage will return the whole entity for each item listed. As mentioned above, an entity might be as large as 1 megabyte. As such, limiting fields requested to the ones that are strictly required can help a lot. For example, you might have code like the following that is requesting every entity in the Catalog and then converting the result into an array of entity references.

(await catalogClient.getEntities()).items.map(stringifyEntityRef);

If you look under the hood, you'll find that the stringifyEntityRef function only makes use of the kind, name, and namespace. As such, we can cut down on the amount of data transferred across the network by limiting the fields that are requested.

(await catalogClient.getEntities({
  fields: ['kind', 'metadata.name', 'metadata.namespace']
})).items.map(stringifyEntityRef);

Operation	Time	Size
`catalogClient.getEntities()`	2.16 seconds	59.5 Mb
`catalogClient.getEntities({ fields: ['kind', 'metadata.name', 'metadata.namespace'] })`	0.767 ms	1.2 Mb

Make use of the filter option to retrieve only entities that you need

We have seen a pattern develop whereby the Catalog client is used to retrieve all of the entities in the Catalog, and then the list is filtered client side.

(await catalogClient.getEntities()).items.filter((entity) => entity.kind === 'Group');

It is more efficient to send a filter to the catalog client so that filtering is done either in the Backstage backend or the Backstage database.

(await catalogClient.getEntities({ filter: { kind: ['Group'] } })).items

Operation	Time	Size
`catalogClient.getEntities()`	2.16 seconds	59.5 MB
`await catalogClient.getEntities({ filter: { kind: ['Component'] } })`	69 milliseconds	2.5 MB

Avoid retrieving all entities in order to count entities

A common pattern we see in Backstage is for developers to download the whole contents of the Catalog in order to count the entities in that Catalog. The following code will cause Backstage to query the database for every entity, then the client will need to decode the JSON it retrieves in order to count the number of entities.

(await catalogApi.getEntities({})).items.length

There is a far more performant way to do this, using the query API. The following requests a limit of 1 entity to be returned, and also requests that the uid field from the entity is the only item that is returned for that entity. The query API always returns the total count of entities for that query. As such it gives us what we need with out downloading the whole Catalog to the client.

(await catalogClient.queryEntities({
  fields: ['metadata.uid'],
  limit: 1,
}).totalItems;

The change suggested here is going to save work for the Backstage database, the Backstage backend, and the Backstage frontend.

Operation	Time	Size
`catalogClient.getEntities()`	2.16 seconds	59.5 MB
`catalogClient.queryEntities({ fields: ['metadata.uid'], limit: 1 })`	45 milliseconds	0.5 Kb

Enable Gzip Compression

When not using the Catalog Client, we recommend using gzip encoding to reduce the amount of data transferred. This is crucial because requests for large amounts of data directly from the Backstage APIs can be massive. Enabling compression significantly decreases the data volume sent to the client. You can achieve this by including the Accept-Encoding header with your client requests.

Operation	Size
`curl https://backstage-server/api/catalog/entities`	59.5 MB
`curl https://backstage-server/api/catalog/entities -H 'Accept-Encoding: gzip'`	6.7 MB

Keep Backstage up to date

The first, and perhaps the most important thing to consider is to keep Backstage up to date. If you are using Roadie, you are already using a very recent version of Backstage. However, if you are managing Backstage yourself, you may have fallen behind. Backstage releases new versions at least once a month, and these versions often contain very valuable performance improvements to the Catalog.

For example, in version 1.6.7 of the Catalog client library, there was an optimization. Previously, the Catalog client would sort all entities before returning them to the caller. This is a nice, helpful utility until there are thousands of entities to sort. Often, it is not necessary or optimal to receive a sorted list of entities.

Collaborate on the OSS Backstage core project

As part of research for this document, we spoke with the core maintainers of Backstage, and there are some great ideas about how to continue to improve the performance. For example, it has been discussed that by default, the getEntities function should be replaced by an iterator object. That iterator would be used to page over the list of entities rather than retrieving the whole list.

As such, keeping up to date with Backstage releases will allow you to benefit from these performance improvements.

Conclusion

This article is illustrative of some of the performance gains that can be achieved, and your mileage may vary. We have not delved into the performance implications of these changes on backend memory and database query performance. However, we can say that these changes can greatly improve these items too. It's difficult to quantify; however, at Roadie, we were seeing huge memory spikes and large garbage collections occurring in Backstage when the whole Catalog is queried. This is possibly due to the physical sizes of the entity Catalogs and the serialization and deserialization that occurs between the client, backend, and database.

We have shown that making use of some good patterns can result in a much improved load times for users. We have shown some examples where timings are reduced from multiple seconds to sub-second. We have also shown that the sizes sent across the wire can be greatly reduced from multiple megabytes to tens of kilobytes.

A well-managed and optimized internal developer portal can make your software engineers more efficient and empower them with the information they need. When load times are reduced from multiple seconds to sub 1 second, developers enjoy a fast, responsive experience that means they’re more likely to use Backstage and find what they need.

Adopting Backstage - Documentation and Support

Jian Reis — Thu, 03 Oct 2024 11:02:51 +0000

This is the first in a series of posts aimed at helping organizations to adopt Backstage.

Making Backstage Easier for New Users

Imagine opening Backstage for the first time, searching for a service, and finding... nothing. How do you even search properly? You’d need to understand concepts like entity kinds, types, and ownership.

And if you can't find the entity, how do you add it? This requires knowing your organization’s ingestion patterns: Do you need a YAML file in the code repo? Or do you manually input the URL into the import flow? Without internal support or clear, beginner-friendly getting started documentation, this process can feel like a maze.

Backstage isn't always intuitive, especially for new users without internal support or clear documentation. Most available open source documentation is aimed at implementers, not end users. It’s often generic, overwhelming, and assumes you’re using GitHub as your SCM system. So, what can you do to make Backstage easier for your team?

How to Simplify Backstage for Your End Users

Invest in User-Friendly Documentation

For Backstage to succeed in your organization, internal getting started documentation to help users use Backstage is a must. This documentation should be front and center for new users, which might mean putting it in an existing documentation platform even if your goal is to eventually move all docs into Backstage’s TechDocs. You can use the Homepage plugin to highlight certain top level info for new users including a preview card for your getting started docs in TechDocs.

Your getting started docs should open with a clear intro to what Backstage is and how it helps. Include simple examples of its core features, remembering that many new users won’t even know what an internal developer portal is meant to do.

As well as have a intro page in your internal documentation explaining it you could also publish a blog post introducing IDPs to your engineers.
i.e. Sample content for an intro to IDPs and Backstage

### Internal Developer Portals
An Internal Developer Portal (IDP) is an application which is designed to give our developers easy access to information and commonly used workflows. Its goal is to reduce context switching and toil for developers by providing a “single pane of glass” that helps to improve productivity, reduce duplication of effort, and foster a more cohesive and efficient development environment.

An IDP is a place to find information about the software we build and use, the teams who build that software, and the API specs, code repositories and documentation associated with that software. It also typically provides self-service automation scripts for common tasks like creating applications or making changes to infrastructure, and scorecards to help ensure that software is adhering to best practices.

### What is Backstage
Backstage is an open-source platform for building developer portals. It was originally created by Spotify to manage and streamline their complex microservices architecture and was released for public use in 2020. 

**Key features of Backstage include**:
- Software Catalog: A centralized listing of all services, providing an overview and allowing easy management and discovery.
- Software Templates: Streamlined processes for setting up new projects and services with pre-defined templates.
- Plugins: Backstage is extensible with plugins to integrate with various tools and services used in AstraZeneca.
- TechDocs: A documentation site generator that converts Markdown files into a browsable documentation site.

Backstage aims to improve developer productivity and collaboration by providing a single, cohesive interface for all development-related activities.

Identify key user journeys and craft your getting started content around them:

Why would a software engineer or product manager initially visit Backstage?
What are they trying to accomplish?
What problems could Backstage help with?

Talk to teams who are just onboarding or align these journeys with company-wide initiatives. If, for example, you see the Scaffolder as a high-value feature, start with docs explaining how to use it and how to modify templates.

You could create these docs in TechDocs and link them from wherever engineers currently go for org-level documentation or just create them in existing documentation systems like Confluence.

Ensure they are searchable by including the right keywords in titles and descriptions, whether hosted in Backstage or elsewhere.

Lastly, publicise these docs as much as you can - get a few blog posts out on any internal news distribution channels, ping organization wide channels with the link and a short teaser description (we’ll be writing a more detailed post about internal marketing very soon with a bunch of resources for you to use).

Create a Dedicated Support Channel

Establish a clear support channel—like a Slack or Teams group—where users can ask Backstage-related questions. This not only helps with adoption but also builds a community where knowledge is shared and best practices emerge.

Pin relevant internal and external docs in these channels to avoid repeated questions. Support channels are also a great place to identify gaps in your documentation, allowing you to improve onboarding and make it as self-service as possible.

You could even nominate Backstage champions—advocates within your organization who can help answer questions and lighten the load on your Platform Engineering/DevOps teams.

At Roadie, support channels with our customers have been essential to successful Backstage rollouts. While documentation is the first line of defense, having a place for follow-up questions is key to encouraging usage and reducing friction for busy engineers.

By streamlining these two areas—user-friendly getting started documentation and a dedicated support channel—you’ll make engaging with Backstage a smoother experience for your team and boost adoption.

Forem: Roadie

Backstage Adoption: The Day 2 Problem

Build it and they will might come

Strategy: treat Backstage like a product

Tactics for sustaining adoption

Automate catalog completeness

Leverage templates for early wins

Measure and communicate value

Make adoption a cultural effort

Invest in custom plugins

Plan for Day 2 from Day 1

What to think about when you’re thinking about an IDP

Focus on the Three Big Challenges

1. Discoverability

2. Self-Service

3. Developer Experience (DX)

Tying It All Together

Understanding the Backstage System Model

Why do we need a system model?

The Basics

Entities

Kinds

Types

Relationships

4. Creating Entirely New Custom Kinds

Conclusion

Useful links:

Why Hybrid is Best for IDPs

Open-source

Benefits of the open-source model for IDPs

Benefits of the open-source model for IDPs

Drawbacks of the open-source model of IDPs

Proprietary

Benefits of the proprietary model for IDPs

Drawbacks of the proprietary model for IDPs

Hybrid

Conclusion

Migrating to Backstage’s New Backend: A Step-By-Step Guide

Introduction

Overview

Preparation for Migration

Establish a Testing Plan

Migration Guide from Backstage

What’s the Plan for Rollout?

Migrating Your index.ts to the New Backstage Backend System

Step 1: Backup Your Existing index.ts

Step 2: Set Up the New index.ts

What’s Next?

Creating a temporary plugin environment

Caveats and Considerations

Using the legacyPlugin

Adding new backend system plugins

Overriding Core Services

Custom Implementations

Recommended Approach

Implementation Steps

Creating Your backend-defaults Package

Step-by-Step Instructions

Configuring the httpRouter service

Healthcheck

Using the lifecycle hooks

Testing the New Backend

Setting Up a Development/Test Environment

Implementing Health Check Tests

Reviewing End-to-End Test Coverage

Manual Testing

Conclusion

Quirks Encountered in the New Backend System

Database Name and Plugin Path Correlations

Key Considerations

Catching Errors in Plugin Configuration

Effective Monitoring Strategies

Conclusion

Update the Backstage catalog instantly without touching any YAML

How long does it take to add a PagerDuty plugin?

Manual Process in GitHub

A Better Way: Entity Updates with Roadie

Bulk updates via UI

Behind the Scenes: How It Works

Leverage external APIs to make things easier

Migrating Your `index.ts` to the New Backstage Backend System

Step 1: Backup Your Existing `index.ts`

Step 2: Set Up the New `index.ts`

Using the `legacyPlugin`

Creating Your `backend-defaults` Package