Forem: Drew Piland

Getting started with CloudBees DORA metrics

Drew Piland — Wed, 27 Mar 2024 17:10:55 +0000

DORA metrics help measure and improve the performance of software delivery teams. They help companies understand how well their engineering teams are performing in driving business value. Through these metrics, teams can identify bottlenecks in the software delivery process and help improve its effectiveness. DORA metrics are foundational for companies seeking to implement value stream management (VSM) initiatives.

This tutorial will explain the four DORA metrics, address the questions they answer, and examine how DORA metrics are instrumented within the CloudBees platform. This blog targets engineering managers seeking more visibility into how their software delivery efforts impact business performance.

UI overview

Before moving into each widget, let’s discuss some common UI elements:

Filtering: Use the filters to choose the component and the duration for which I want to see flow metrics down to the component level. When filtering, please note that weeks run from Monday to Sunday.
Drill downs: You can click any data point in the bold blue font for a deeper dive.
Hovering: Each report has a tooltip explaining its coverage. You can hover over each graph type to get a breakdown.
Viewing: All CloudBees platform pages can be viewed in either light or dark mode. We use dark mode for this post.

DORA metrics have been studied for several years and, thus, act as a benchmark to help organizations determine where they fit, broken down from elite to low performers. All teams are different in terms of what level they strive to achieve. When setting your software delivery goals, you must know your current status and incorporate measures to help you progress in your performance metrics.

DORA metrics in the CloudBees platform

DORA metrics within the CloudBees platform are available at the organization, sub-organization, or component level.

Let’s start by zooming in on the top row where the four DORA metrics are displayed.

Deployment frequency

Deployment frequency helps answer questions about how often software deployments are made to production. Overall, deployment frequency provides valuable insights into the team's agility, speed, and ability to release software to production, helping teams identify opportunities to optimize their release process and deliver new features and fixes more quickly and reliably.

In our example, we average 19.55 deployments per day. As a software vendor developing a new product, it makes sense to have such a high frequency when adding core functionality.

While increasing deployment frequency indicates an agile team, ensuring you are deploying the right features is vital. Thus, ensuring deployment frequency isn’t measured in isolation is necessary for optimal outcomes.

When benchmarking performance:

Elite Performers: Multiple times a day
High Performers: Once a week to once a month
Medium Performers: Once a month to once every six months
Low Performers: Less than once every six months

Deployment lead time for changes

Lead time for changes measures the time it takes for a code change to be implemented and deployed to production. It provides valuable insights into the team's efficiency and speed in implementing and deploying changes, helping teams identify areas for improvement and optimize their development and release process to reduce lead time and improve customer satisfaction.

Our example shows a deployment lead time for changes of four minutes and two seconds. This number will likely increase as the product matures and the team grows. Engineering managers should be aware of several factors that could lead to slower lead time for changes, such as:

Poor communication: Projects often involve large, dispersed teams, which can lead to misunderstandings of requirements. As a manager, you can reduce this likelihood by creating appropriate-sized and centralized teams.
Tech debt: impacts lead time as old issues must be addressed.
Lack of automation: manual building, testing, and deploying processes can significantly slow lead time. Ensure automation is incorporated as necessary to help.
Lack of resources: if a team needs more resources (people or infrastructure), this can slow down the deployment process.
Regulatory compliance: Highly regulated industries often require additional rigorous checks before deployment, impacting the lead time.

When benchmarking performance:

Elite Performers: Less than one hour
High Performers: One day to one week
Medium Performers: One month to six months
Low Performers: More than six months

Change failure rate (CFR)

Change failure rate provides valuable insights into the reliability of the team's release process and the quality of their releases. Low failure rates are desirable. For example, a 10% change failure rate indicates that 10% of all changes made to the system failed.

These insights help engineering managers identify areas for improvement and optimize their release process to reduce the risk of failed changes and improve customer satisfaction. For example, a high CFR may indicate that too many changes are being introduced simultaneously. High CFR also raises the question of investment in tooling and infrastructure. If the change failure rate continues to be high, there’s also a cultural impact on morale, leading to lower developer productivity.

When benchmarking performance:

Elite Performers: 0-15%
High, Medium, and Low Performers: 16-30%

Mean time to recovery (MTTR)

MTTR provides valuable insights into the effectiveness of the team's incident response process, helping identify areas for improvement and optimize their incident management process to reduce downtime and improve customer satisfaction. Teams should strive to make this number as low as possible. MTTR helps address questions such as:

How quickly can development teams respond to and recover from incidents?
How effective is the team’s incident management process?
How much downtime does the team experience?

When benchmarking performance:

Elite Performers: Less than one hour
High Performer: Less than one day
Medium Performers: One day to one week
Low Performers: Over six months

Additional insights

The DORA reports section of the CloudBees platform goes beyond the four metrics. We also offer two trend reports to provide teams with additional insights.

Deployment frequency and lead time trend

This widget tracks the number of deployments and lead time for the selected date. These two metrics work together for the following purposes:

Assessing Efficiency: A high deployment frequency coupled with a short lead time for changes indicates an efficient and effective software delivery process—teams can frequently deliver small, manageable changes with minimal delay.
Identifying Bottlenecks: If deployment frequency is high but lead time for changes is also high, it may indicate that while the team is deploying often, it's taking a long time for those changes to go from idea to deployment. This could point to development, testing, or deployment bottlenecks.
Balancing Speed and Stability: If deployment frequency is low but the lead time for changes is also low, it suggests that the team is focusing on delivering large batches of changes quickly. This could lead to a risk of instability or issues in production. Balancing the two metrics can help achieve speed and stability in the delivery process.

By monitoring both deployment frequency and lead time for changes, software delivery teams can better understand their delivery process, identify areas for improvement, and make more informed decisions about optimizing their workflow.

Failure rate and mean time to recovery trend

This widget tracks the number of failed deployments and the mean time to recovery (MTTR) for the select dates. These two metrics work together for the following purposes:

Identifying Problem Areas: A high failure rate and a high MTTR could indicate serious problems with the system's reliability and resilience. This might suggest the need for a thorough review and overhaul of the system.
Improving System Design: Understanding the relationship between failure rate and MTTR can help teams design better systems. For example, if the failure rate is high but MTTR is low, it might be due to frequent minor issues that are quickly resolved. This could lead to a focus on improving overall software quality to reduce the number of minor issues.
Balancing Resources: If the failure rate is low but MTTR is high, it might indicate that while the system is generally stable, significant issues take a long time to resolve. This could suggest the need for more resources to be put into faster problem detection and diagnosis or building more robust recovery mechanisms.

By monitoring and understanding both the failure rate and MTTR, software delivery teams can gain valuable insights into their system's performance, identify areas for improvement, and make informed decisions about where to invest resources for the most significant impact.

Configuring DORA metrics in the CloudBees platform

As of March 2024, DORA metrics rely on users tagging steps as Deploy within the Kind operator. Once applied, all data will funnel into the tabs. To learn more about this, visit our documentation.

Next steps

The CloudBees platform offers organizations a quick way to access DORA metrics, with granularity down to the organization, sub-organization, and component level. Equipped with this information, engineering managers can track the progress of delivering business impact with the ability to communicate this upstream efficiently.

With the information provided throughout this blog, you should better understand DORA metrics, how to set them up, and how to interpret the results. Now, it’s time to get started. Try the CloudBees platform for free to put these steps into practice. For complete documentation on DORA metrics, click here.

To learn more about additional CloudBees analytics reports, visit the below documentation.

CloudBees Security Insights Overview

Drew Piland — Mon, 18 Mar 2024 13:30:13 +0000

The CloudBees security insights report provides detailed insights into the results of security scans to users. It helps software development teams, security officers, and CISOs gain insights into security vulnerabilities and bugs so that users can quickly resolve such issues and improve the overall quality of their software. The report provides a single bird's eye view of how vulnerable an organization, sub-organization, or component is.

This blog aims to provide an overview of each widget, explain its value, and demonstrate how to act on the vulnerability insights presented. Users would visit this report to answer several questions:

How vulnerable are our workflows to breach? What is the severity level?
When vulnerabilities appear, do we know where they occur for remediation purposes? Are we improving our response time based on this information?
Which scanner types are we using? What percentage of workflows are these in?
What percentage of workflows are breaching SLA?

How does all this work?

Each widget uses a scalable and flexible Cassandra database to ingest data from workflows, scans, and integrations like JIRA. The data is indexed by an open search cluster, where an analytics service provides some pre-computation services so that the data is already pre-computed for all the widgets. When a user opens one of the reports, it connects to the report service via a secure API gateway for quick access.

UI overview

Before moving into each widget, let’s discuss some common UI elements:

Filtering: Use the filters to choose the component, duration, org, and sub-org level. Please note that weeks run from Monday to Sunday. Our overview will focus on data for December 2023, cloudbees-staging org level, and all its components.
Drill downs: Click any data point in the bold blue font for a deeper dive. Most widgets in this report allow you to view the vulnerability ID, components affected, the scanner user, and insight into the impacted line of code with a direct link to the GitHub repository.
Hovering: Each report has a tooltip explaining its coverage. You can hover over the data for each graph type to get a breakdown.
Viewing: All CloudBees platform pages can be viewed in either light or dark mode. We use dark mode for this blog.

Components, workflows, and successful workflow runs

Our first set of widgets aggregates information and tells you how many components and other workflow runs exist, along with their scanner status. You can see there are 489 components with 468 repos, broken out by how many components have a scanner or are without a scanner. So, out of 489 components, 51 have some kind of scanner, and 438 have no attached scanners.

Additionally, I can click on the 51 components, which tells you which scanners exist. For example, in the analytics component, snykcast and sonarqube scanners exist.

The Workflows widget provides a breakdown of workflows, highlighting those equipped with scanners versus those without security scanners. Here, it tells you that 489 components have 3,713 workflow files, composed of 2,221 branches, of which 141 workflows have some kind of scanners, and 3,572 workflows have no scanners.

You can drill down into which workflows have no scanners attached. Similarly, if you click on 246 workflows, you can see which scanner is attached to run as part of this workflow.

These 3,713 workflows ran 12,570 times, of which 504 times included a scan step and 12,066 times there was no scan step.

By better understanding your security scanner coverage across workflows and components, engineering managers, security officers, and CISOs unlock visibility into their threat level and can take action. The summary record shows how often workflows are run with or without a scanner. Typically, running workflows without scanners in the early stages of development is okay. Still, as an enterprise customer, you want to avoid having production deployment of code that was never scanned.

Vulnerabilities overview

The vulnerabilities overview widget tells you how many vulnerabilities are found, reopened, resolved, or open. Let’s define these categories:

Found: new vulnerabilities encountered for the first time (i.e., not specific to the current duration) for a component. Reopened: vulnerabilities previously encountered and resolved are now showing up again.
Resolved: vulnerabilities discovered earlier but not appearing in the latest scan of the specified period are considered resolved.
Open: vulnerabilities appearing in the latest scan of the specified period.

For December, we found 415 vulnerabilities, reopened one, resolved nine, and 406 remain open. Let’s drill down into found vulnerabilities.

When I click on 415, I see a Vulnerabilities Overview screen, which includes ID, discovery date, name, stats, severity, and impacted components. Next, I decided to drill down into the vulnerability ID of CWE-259 since it has a very high severity. I now want to dig deeper into the occurrence from the snykcast scanner and see the exact repo location, if available. In this instance, it exists on line 50. This highlights how granular these reports are and how they allow security teams to troubleshoot for root cause issues.

Open and reopened vulnerabilities
The reopened vulnerabilities widget provides the mean age of open vulnerability occurrences by severity, showing 37 vulnerabilities are very high severity, 105 are high severity, 164 are medium severity, and 100 are low severity. I can further click on these numbers and get a drill down of these 37 very high vulnerabilities, which is where I should prioritize.

For example, I can drill down into CW-295 and notice the vulnerability name is some kind of improper certificate validation that needs fixing. I can click on this number of occurrences, click on number four, and see the repo, file, and line number where this is happening.

This widget also tells you the vulnerability criticality level (very high, high, medium, and low) and how long they've been open (mean, median, max) using a box plot on a candlestick chart—ideally, the higher the severity, the quicker the resolution time.

Scanner types

These widgets focus on better understanding the utilization of your security scanners.

Scan types in workflows
The scan types in workflows widget track the distribution and frequency of four scan types:

Static application security testing (SAST): SAST takes an “inside-out” approach to finding security vulnerabilities by analyzing the app’s source code in a nonrunning state.
Dynamic application security testing (DAST): Like SAST, DAST also focuses on finding security vulnerabilities by analyzing the source code. However, DAST differs in that it does this while the application is running. DAST solutions analyze the application from the "outside-in" by simulating cyber attack behaviors and techniques.
Software composition analysis (SCA): This method achieves open-source software compliance and is used to identify open source components with known vulnerabilities.
Container scanners: These tools help identify issues with your container images, such as outdated libraries, dependencies, or potential vulnerabilities. They can be used as part of a CI/CD pipeline to catch potential security issues before they make it into production.

The scan types across workflows help ensure consistent security evaluations. It also tells you how many workflow runs are part of which scanner type. For example, 156 workflows with a SAST scanner type execute 470 workflow runs.

Vulnerabilities by security scan type
Whereas the scan types in the workflows widget display the number of workflows and workflow runs by scanner type, the security scan type widget tells you how many of these vulnerabilities were found. This widget further breaks the number of vulnerabilities found by very high or high for different security scan types, so you can see how many of them are high, very high, or low per category of a security scan type.

For example, we found 44 vulnerabilities of SAST, 17 of DAST, 253 from container scans, and 112 vulnerabilities from SCA. You can click further into each of these to determine the affected component and drill down to the GitHub repo and line level.

SLA status overview by occurrences

For the SLA status overview by occurrences widget, we have defined some SLAs in the current version that should fix vulnerabilities within three days.

On track: less than two days
At risk: two days
Breach: after three days, we mark it as breached.
Please note these SLA statuses are hard-coded for all users but can be modified on an account or user basis. Please reach out to your CloudBees representative to assist here.

For all open vulnerabilities, seven are on track, zero are at risk, and 3,098 open vulnerabilities have breached the three-day marker to break our SLA. This view provides a benchmark for tracking vulnerability resolution time moving forward.

Mean Time to Resolve (MTTR) for vulnerabilities occurrences

The mean time to resolve (MTTR) for vulnerabilities occurrence tracks how long it takes users to fix vulnerabilities based on severity type. If MTTR is improving monthly, that shows the team is solving vulnerabilities more efficiently.

Further, we break these numbers down weekly to see how fast the engineering teams have resolved our vulnerabilities.

CWE top 25 vulnerabilities

The CWE Top 25 is a list compiled by MITRE that lists common security vulnerabilities with the most severe impact. This widget lets you instantly identify impacted components, how many occurrences, and the SLA status to help troubleshoot to understand better how exploitable the system is.

Within the cloudbees-staging organization, we detected five of these vulnerabilities. Further, I can click on any of these components, and this will give me a drill down of what those components are.

Next Steps

The security insights report provides an aggregated view of your vulnerability status across workflows. The report aims to provide software development teams, security officers, and CISOs with insights into security vulnerabilities to help quickly resolve issues and improve software quality. Using this report, you can quickly answer the following questions:

Which scanners are currently in our workflows?
How many vulnerabilities are we seeing weekly?
What is the severity breakdown of our vulnerabilities?
How quickly are we addressing vulnerabilities?
What scan types are we using, and how does this relate to severity?
How often are we breaching SLAs?

youtube.com

Now that you understand this report better, it’s time to start. Try the CloudBees platform for free. For complete documentation on security insights, click here.

To learn more about additional CloudBees analytics reports, visit the documentation below.

Interpreting Software Delivery Activity Within The CloudBees Platform

Drew Piland — Thu, 14 Mar 2024 15:31:52 +0000

Software delivery activity is a cornerstone analytics report in the CloudBees platform. It provides users with detailed insights into the performance and health of engineering processes. The report helps software development teams and engineering managers optimize their workflows, identify and troubleshoot issues quickly, and improve the quality of their software.

The software delivery activity report's data is pulled from a connected GitHub repository or an equivalent code repository. The data is then aggregated at the component, organization, and sub-organization levels to present a bird' s-eye view of engineering work.

Before moving into each widget, let’s discuss some common UI elements:

Filtering: Use the filters to choose the component, duration, and org level. Please note that weeks run from Monday to Sunday.
Hovering: Each report has a tooltip explaining its coverage. You can hover over the data for each graph type to get a breakdown.
Drill downs: You can click any data point in the bold blue font for a deeper dive.
Environments: Any spot where environments are shown will populate with your data (staging, production, QA, etc.). Throughout this tutorial, our data only focuses on staging.
Viewing: All CloudBees platform pages can be viewed in either light or dark mode. We use dark mode for this blog.

Components and workflows

Our first row of widgets tells how many applicable components, workflows, and workflow runs for the selected duration. Further, this information distinguishes between active and inactive components. We define active components as those with at least one workflow executed in any branch for the selected time frame.

We can interpret the prior chart as follows. There are 423 components spanning 411 repos, with only 188 components running an active workflow. This results in 2,654 workflows, of which 1,406 are active. These workflows span 10,210 runs, of which 6,292 are successful. For further analysis, users can click to drill down into the numbers. For example, if users are curious about the 3,918 build failures, they can click on it to see a list of the failed workflows.

Commits, pull requests, and code churn trends

In the next set of widgets, we dive into commits, pull request trends, and code churn. As an engineering manager, these help provide overall throughput to help with resourcing.

Commits trend
The commits trend shows the total number of commits for the selected components within the chosen time frame. As you can see, there are 9,383 commits by 109 active developers, yielding 86.1 weekly commits/active devs (commits/active devs). Active developers refer to developers who have contributed and made at least one commit in the selected duration for any of these components.

Pull request trends
Pull request trends break down these commits into pull requests. In our example, there are 1,266 pull requests. You can see the status of pull requests, whether they were approved, whether changes were requested, and whether they are still open or rejected. Open will indicate all the open pull requests that are still actively being worked on or in review.

Code churn
Code churn tells you how many lines of code were added or deleted over time. Code churn can be helpful for users because it should match the user’s internal mental model of what normal looks like. In our case, the number of additions is slightly higher than deletions, which is typical for our product because we are actively working on this product and adding new features. Thus, it resonates with our mental model. If there is a deviation from this, further investigation is warranted.

Code progression, successful build duration, and deployment overview

The next series of widgets explores how code progresses from commits to deployment, both from a timing and success rate perspective. Using these widgets requires users to define the step: kind in their workflows .yaml so that the system can accurately distinguish between builds jobs, scans, and deployment jobs.

Code progression snapshot
The code progression snapshot highlights the coding journey from run-initiating commits through builds to deployments, providing a view into the overall efficiency.

In this example, you can see 10,210 run-initiating commits, which means this commit triggered some workflow run. If users have defined multiple environments, they will receive a breakdown of where each run-initiating commit sits.

To progress these charts, we show that the 10,210 run-initiating commits generated 753 builds, of which 705 succeeded (94% success rate), resulting in 606 successful deployments, all from staging. If you have multiple environments (pre-prod, production, QA), they will also appear here.

You will notice there were 48 failed builds. For triage purposes, you could drill down to see the run ID, status, start time duration, and the component name of the run ID build that failed. From there, you could click on the run ID, which takes the user to the details about why this run failed.

Successful build duration
In the next row, users can see information about builds and deployments. In the successful build duration widget, we will provide a box plot for each component and how long the build takes. Users can see the minimum, median, and maximum durations when you hover your mouse over this. Users can use these arrows to scroll through many of their components and know how these build durations vary over time.

In this example, the reports-service component is interesting because it shows a minimum build time of 26 seconds but a max time of three minutes. These results should make engineering managers curious why some builds take three minutes instead of a few seconds and warrant further investigation to improve efficiency.

Deployment overview
The deployment overview tracks successful and unsuccessful deployments in each environment over time. In our example, we have a reasonable 98% success rate of deployments to staging. Furthermore, we provide weekly breakdowns of successful and failed deployments.

Measuring your development cycle times

Our final set of widgets explores the development cycle time and average deployment time.

Development cycle time
The development cycle time widget is a personal favorite. It breaks down your full development cycle time into various components to show your efficiency. Engineering managers can view the data from multiple levels (org, suborg, or component) to optimize processes. This granularity allows root cause analysis to identify the most impactful bottlenecks and offers a blueprint for where fixes must occur. We recommend starting small and measuring the impact on the broader system.

Our example shows us that, on average, the development cycle time is composed of two days, six hours, 44 minutes, and 49 seconds; this meets our expectations because we have a globally distributed engineering team.

The average development cycle time comprises three stages: coding, pick-up, and review.

Coding time spans from when an individual contributor proposes changes to the codebase until the pull request (PR) is created.
Pickup time is calculated from when the PR is created to when the review begins. It represents the first handoff that requires another person to push it forward, and it is often the biggest bottleneck.
Review time is calculated from the start of the review to when the code is merged; it addresses everything necessary to promote code into production. Depending on the review, this stage can restart the process through complete code rewrites.

Average deployment time
Our final widget tracks the average deployment time for the selected duration across environments and builds upon the development cycle time widget.

We only have a defined staging environment, which takes three minutes and 57 seconds to deploy. Those would also appear here if we had additional environments, such as production or QA. Similar to the deployment overview widget, average deployment time requires you to select the deploy Kind option when assigning steps in your workflow to pull data over.

Putting it all together

As a platform engineering team or individual engineering team manager, you can’t improve processes without visibility into what’s occurring. CloudBees provides visibility into your software delivery activity levels in multiple ways (component, organization, or sub-organization). When bottlenecks or issues are identified, the ability to drill down to the source helps quickly resolve issues.

With this foundation in place, you can create the necessary benchmarks to measure and improve efficiency. Understanding where your teams are strongest also helps allocate resources to ensure organizations can meet growing customer demand. Ultimately, the goal is to create a great developer experience with as little friction as possible. By taking a proactive approach to efficiency, you are equipped to improve the amount of time your development teams are writing code.

Use Case #1 - Engineering manager sees high cycle time
John, an engineering manager at a big tech firm, sees a very high cycle time in the development cycle time widget (one day, 16 hours, and 12 minutes), which does not match his intuition. The average cycle time further breaks down into cycle, pickup, and review times.

While the coding time of 16 hours and review time of 5 hours and 31 minutes meet expectations, the code pickup time of 18 hours and 38 minutes seems very high. It implies that after coding is finished (PR created), on average, PRs are waiting nearly a day (18 hours) to be picked up for review.

John reviews the agile process within his team and addresses the high code pickup time number with the team. The team points out various reasons why PRs are sitting in a pickup state, such as missed PR notifications and the frequency of scrum calls. Based on the discussion, the team optimizes the process, reducing the pickup time to a few hours and, thereby, reducing their development cycle time to a day.

Next steps

The software delivery activity report offers a great source of information for engineering managers looking to optimize their team’s software delivery performance. Most of the data in these reports will automatically appear once your GitHub or other repo is connected. Users can then aggregate data at the org, sub-org, or component level for further analysis.

Now that you understand this report better, it’s time to implement that knowledge. Try the CloudBees platform for free. For complete documentation on software delivery activity, click here.

To learn more about additional CloudBees analytics reports, visit the documentation below.

Getting Started with CloudBees Flow Metrics

Drew Piland — Mon, 04 Mar 2024 18:03:35 +0000

Flow metrics measure how value moves through a software product's end-to-end value stream. This tutorial explains how CloudBees uses flow metrics in our platform to provide a high-level view of the software development lifecycle (SDLC) through a business lens.

Flow metrics can help businesses make more decisive decisions by showing the rate of value delivery concerning desired business outcomes. They can also provide a historical analysis of performance over time. CloudBees can aggregate data from your ticket management or work item management software like JIRA and connect it to your actual work in a code repository such as GitHub, and backtrack and put this information together and tell you how value is moving and progressing through your development lifecycle.

CloudBees categorizes work tasks for a given product value stream into four flow item types:

Feature (a business value)
Risk (possible lack of security or compliance)
Bugs (a coding defect)
Tech debt (a potential blocker to future delivery)

Configuring your flow metric dashboards

Configuring flow metrics requires connecting your Jira using the Analytics Configuration side navigation. If you've never set up flow metrics, you must set this up the first time.

From the side navigation, select Analytics → Analytics configuration
Click the Create Project button in the upper right corner.
Select a project management software, such as Jira.
If no options appear, navigate to Configurations →
Integrations and create a JIRA integration. Once completed, your JIRA integration will appear.
Once you choose your project management, you can select a project from JIRA.
Click Next
In the next section, you must map feature, defect, risk, and tech depth to other issues.
JIRA issue types include tasks, bugs, or epics. For example, features could be epics, and you can match them to an issue type. You can also map it to a full-blown JQL (Jira Query Language) instead of a ticket type. Similarly, you can do the same for defects, risks, and tech debt.

Here's my current configuration for the software delivery platform.

Having set this up correctly, I can return to my flow metrics report.

Before moving into each widget, let’s discuss some common UI elements:

Filtering: Use the filters to choose the component and the duration for which I want to see flow metrics down to the component level. When filtering, please note that weeks run from Monday to Sunday.
Drill downs: You can click any data point in the bold blue font for a deeper dive. These drill-downs will present you with all relevant Jira links for further investigation.
Hovering: Each report has a tooltip explaining what it covers. For each graph type, you can hover over to get a breakdown.
Viewing: all pages within the CloudBees platform can be viewed in light or dark mode. For the purposes of this blog, we use light mode.

Work load

This widget displays the work load, which is the amount of work that the team is managing, providing insights into the team's workload and capacity.

The work load widget shows the average number of work items in progress within a specific value stream, including bugs, features, risk, and technical debt; these are indicators of productivity.

When analyzing the graphic, we notice a few trends for October.

There are about 80 features week over week that are always in progress. Towards the end of the month, this number dropped.
The number of bugs in progress increases throughout the month
Tech debt and risk remained zero.

An increasing work load suggests excess work in progress, which can lead to future delays in flow item completion time. Frequent software delivery reduces workload and improves cycle time and velocity.

Work item distribution

This widget displays the distribution of work items, providing insights into the types and volume of work that the team is handling.

Work items distribution tells you the distribution of completed work items. When analyzing the prior example:

You can see what percentage of time the team has been working weekly on bugs versus features, but not risk and tech debt, due to no data for the defined filter.
You can hover over the graph to get week-by-week views. The trends here show that the team spent much more time on features than bugs at the beginning of October. As the weeks progressed, the team focused more on bugs and fewer features.

Monitoring work items distribution helps prioritize specific types of work during different product lifecycle stages. For new business features, this helps to balance the mitigation of risk, bugs, and technical debt

Velocity

This widget displays the work velocity, which is the rate at which the team is completing work items, providing insights into the team's productivity and efficiency.

The Velocity widget displays the number of completed work items by type. While the work item distributions widget tells you the distribution and the percentage of work items by type, velocity tells you the number of items completed on time.

In this example, you can see that the team completed 241 work items. Most of the feature work was completed in early-mid month. Towards the end of the month, we worked less on features and will spend more time on bugs, which is the red line here.

An increase in velocity over time signals that productivity is improving. Tracking velocity helps forecast software delivery rates and uncover any problems at an early stage.

Cycle time

This widget displays the cycle time, which is the time taken to complete a work item from start to finish, providing insights into the team's efficiency in completing work items.

Cycle time conveys how long the team takes to achieve a specific item type. You would assume that features are larger to fix than bugs. But that is only sometimes the case. This example displays the average time to create and close an issue in Jira.

As you increase the speed of your software delivery, you reduce the cycle time. With detailed data on flow item completion times, you can confidently predict future cycle times and better understand the efficiency of your value stream.

Work efficiency

This widget displays the trend of flow efficiency over time, providing insights into how the team's efficiency and productivity are evolving.

Work efficiency tells you how much time the team spent actively working on it and how much time these bugs and features typically were in the waiting period. The work efficiency and cycle time widgets can be used together.

Average active work: provides a breakdown by issue type of active work. In this example, bugs were 95%, and features were 65%.
Work wait time: The chart explains what makes up the waiting time. In this example, 22% is due to code review, while 15% was in a Blocked state.

Next steps

You should better understand flow metrics, how to set them up, and how to interpret the results. Now, it’s time to get started. Try the CloudBees platform for free to put these steps into practice. For complete documentation on flow metrics, click here.

To learn more about additional CloudBees analytics reports, visit the below documentation.

How to set preconfigured actions in the CloudBees platform

Drew Piland — Mon, 26 Feb 2024 16:42:09 +0000

Previously, we described concepts associated with our Composer UI for creating workflows within the CloudBees platform. This blog dives deeper into one of those elements: preconfigured actions.

This blog is written for platform engineering /administrative teams as the primary audience, with concepts that will also appeal to application developers. Our goal is to provide a clear understanding of what we mean by preconfigured actions, the tangible benefits of this approach, and how to create your own preconfigured action.

Setting the stage: Terminology

Let’s first align on some core terminology used within the platform to help understand where preconfigured actions fit:

Workflow
A CloudBees workflow is a collection of jobs developers use to create a “component,” or a piece of software to deliver to customers.

A user can create a workflow in two ways:

Edit the UI using the actions available within the CloudBees software delivery platform and do so under each step.
Edit the code itself.

Jobs
Jobs are comprised of steps, which are detailed sets of activities. You can have one or multiple jobs; under each, there can be a single or multiple steps. Each step performs an action that could integrate with external tools.

CloudBees actions

CloudBees actions are a custom application for the CloudBees platform that performs complex but frequently repeated tasks. They serve as the starting point for common tasks. Visit our CloudBees actions catalog for a complete listing.

What are preconfigured actions and their advantages?

Preconfigured actions are actions with already defined values, such as access tokens, URLs, and settings. This means they come ready with the necessary configurations for use.

Any action can be preconfigured to protect sensitive information, simplify ease of use, and allow for straightforward integration into workflows.

Preconfigured actions offer several advantages, including:

Simpler and faster action setup for developers
Improved security, as sensitive information remains hidden.

For example, developers want to use a Snyk SAST Scan as part of their workflow. They can use an action for their tool (snyk-sast-scan-code) that has been preconfigured with the Snyk organization name, client secret, and language inputs already defined.

By creating preconfigured actions with necessary credentials and settings, developers can seamlessly utilize these actions without directly handling sensitive information like secrets or tokens. This setup ensures that only the administrator needs to manage these details, significantly enhancing security by limiting access to sensitive data within the software delivery process.

How to create a preconfigured action on the CloudBees platform

Here are the detailed steps an administrator needs to follow to configure and save a preconfigured action with specific inputs using a CloudBees-created action as a starting point:

Select the drop-down arrow next to Configurations on the left pane, then select Actions.
Select CREATE ACTION.
Search for a CloudBees action by entering all or part of an action name into Search by Name, then select an action.
Enter a Name.
(Optional) Enter a Description.
Enter Action inputs for Jenkins. Inputs marked with an asterisk are required to run a workflow but are not required to create a preconfigured action.

You can submit a preconfigured action without entering values for all required inputs. However, a user of the preconfigured action must enter values for all remaining inputs marked as required before saving and running the workflow.

Select SUBMIT.

Success! You have created a preconfigured action listed in Actions.

Adding Preconfigured Actions to a Workflow

Once your preconfigured action has been created, the next step is to add it as a workflow; this can be done through either the Composer UI or the code editor. The following gif shows how to add this through the graphical composer. In this example:

User would zoom in on the graphical composer and click + Add a Step
Provide a name for the step.
Click SELECT FROM CATALOG
Select the desired Preconfigured Action and then click APPLY SELECTED. From here, the form will be prepopulated with your information.
The developer will complete the remaining blanks as needed.
Select a Kind option if you want data populated into the Analytics reports. Click Save.

Success! You will now see your preconfigured action added as a step in the graphical composer UI and the code editor.

Getting started

Preconfigured actions are another way the CloudBees platform enhances security, speed, and efficiency for developers. Start using CloudBees for free.

Use the following resources to learn how preconfigured actions with the CloudBees platform can benefit your development team and software delivery process.

Review our preconfigured actions documentation for more detailed steps on using, adding, editing, and deleting preconfigured actions in CloudBees.
Access our complete catalog of CloudBees actions on GitHub.