Forem: Middleware

15 Top Software Development KPIs You should track in 2025

Rajni Rethesh — Thu, 03 Apr 2025 14:44:23 +0000

Managing a software development team is no mean feat. Till they take the project to the finish line, an engineering project manager can't take a breather. That is the reason software engineering managers look for ways to improve the performance of their projects as well as their teams. And, that's exactly where things like Kpi come in God's disguise.

So, what is a KPI?

KPIs are like your team's fitness tracker -- they help you see where things are working smoothly and where you might need to tighten the screws. But with a zillion KPIs out there, which ones should you actually care about? Let's break down the top 15 that will make you look like a rockstar software team manager and a few you might want to ditch.

Also read: Key Metrics for Measuring Engineering Team Success

Why bother with KPIs?

KPIs are more than just numbers on a screen -- they're your roadmap to better decision-making. By tracking the right metrics, you can identify where your team is excelling and where there's room for improvement. It's like having a crystal ball that helps you predict project timelines, resource needs, and potential roadblocks. Learn more on 22 Key Performance Indicators Engineering Metrics. How to Measure Engineering Productivity?

Top software development KPIs you should track in 2025

Cycle Time: Your Team's Speedometer!

Imagine you're in a race, but instead of cars zooming around a track, your team is racing to complete tasks in a sprint.

The question is: how fast can they get from the starting line ("to-do") to the finish line ("done")?

That's where Cycle Time comes in -- it's the stopwatch that tells you just how quick your team is at getting stuff done.

Cycle Time is all about speed, but it's not just about going fast for the sake of it.

It's about efficiency and knowing where the slowdowns happen. On average, high-performing teams have a Cycle Time of about 1.8 to 3.4 days per task.

If it's taking longer, it might be time to look under the hood and see what's causing the delay -- maybe it's a process bottleneck, too much multitasking, or just plain old technical debt.

Let's break it down with an example:

Say your team is working on a new feature for a mobile app. The task moves from the backlog to "in progress" on Monday morning. Your dev team starts coding, testing, and pushing commits, and by Wednesday afternoon, the task is complete and marked "done." That's a Cycle Time of 3 days.

Now, let's say another task hits a snag -- maybe the code review takes forever, or there's a dependency that's holding things up. If that task drags on for 7 or 10 days, it's a sign that something's not quite right.

Here's where the magic happens: By tracking Cycle Time, you can spot patterns

Maybe your team is super speedy on some tasks but bogged down on others. With this insight, you can dive into the specifics and figure out how to streamline the process. Maybe it's as simple as tweaking the code review process or prioritizing tasks differently.

The goal? To reduce Cycle Time, so your team is consistently knocking out tasks like pros.

And when that happens, you're not just moving fast -- you're moving smart.

Code Coverage: Quality Control for Your Code

When it comes to code, it's not about writing a ton of it -- it's about making sure what you do write actually works. That's where Code Coverage comes into play.

Think of Code Coverage as your code's health checkup.

It tells you how much of your codebase is being tested, so you know you're catching those sneaky bugs before they become a problem.

In the world of software development, a good benchmark for Code Coverage is around 70-80%. If you're hitting that, you're doing pretty well.

But remember perfection isn't the goal here -- 100% coverage is like trying to catch every grain of sand on a beach.

Instead, focus on making sure the critical parts of your code are covered.

Let's put it into perspective with a simple example:

Imagine you're building a new feature for an e-commerce site -- let's say it's a shopping cart.

You've written code that adds items to the cart, calculates totals, and processes payments. Now, you want to make sure all this works before customers start using it.

You write tests for each part:

Adding items to the cart -- You test to see if items are added correctly.
Calculating totals -- You check that the math is right when someone adds multiple items.
Processing payments -- You test the payment gateway to make sure transactions go through smoothly.

If your tests cover all these scenarios, and they run without errors, you've got solid Code Coverage. But if you skip testing the payment process (maybe because it's complex or takes extra time), you're leaving a critical part of your code untested -- which is like leaving your door unlocked at night.

By keeping an eye on Code Coverage, you ensure that most of your code is being tested, which reduces the chance of bugs sneaking into production. It's all about catching issues early, so they don't turn into customer complaints later on.

3. Code Rework: The Hamster Wheel of Development 🐹

Picture this: your dev team keeps rewriting the same chunks of code over and over. Instead of sprinting toward progress, they're stuck on a hamster wheel, going in circles without actually moving forward. That's Code Rework in action, and it's a sign that something's off.

Ideally, your team should spend more time building new features and less time redoing what's already been done. Too much Code Rework can be a productivity killer.

In fact, studies show that frequent rework can consume up to 40% of a developer's time -- time that could be better spent on innovation.

4. Change Failure Rate (CFR): The Bug-O-Meter 🐞

Think of Change Failure Rate (CFR) as your dev team's "bug-o-meter." It measures how often your code changes end up breaking stuff. A high CFR is like having a leaky boat---you're constantly bailing water (fixing bugs) instead of sailing smoothly (building cool new features).

In an ideal world, every change you make to the codebase would work flawlessly. But in reality, things break. According to the Accelerate State of DevOps Report, the industry average for CFR is around 16-30%, meaning that out of every 10 changes, 1 to 3 might cause a hiccup. If your CFR is creeping above that, it's a sign that your code needs more TLC before hitting production.

Quick Example:

Let's say your team rolls out a new feature, and immediately, users start reporting crashes. You dig into the data and realize that 40% of your recent deployments led to issues. Ouch! That high CFR means your team will be spending more time firefighting bugs and less time innovating.

The goal? Lower your CFR by improving testing and code reviews, so you can spend more time building the next big thing and less time fixing what's already been shipped.

5. Defect Detection Ratio (DDR): The Bug-Catching Scorecard 🎯

Defect Detection Ratio (DDR) is like your bug-catching scorecard---it tells you how many bugs you catch before the code hits the wild versus how many slip through after launch. The higher your DDR, the better your testing game is. But if more bugs are sneaking past you and showing up in production, it's time to sharpen your testing tools.

A good DDR shows that your testing process is solid, typically aiming for 85% or more of bugs caught before release. If your DDR is low, it's like missing a bunch of red flags, only to find out later when users start complaining.

Quick Example:

Imagine you release a new app update. During testing, you catch 8 bugs, but after the launch, users report another 5. That gives you a DDR of 8/13, or about 62%. Not great. It means your testing missed nearly 40% of the bugs, which is a clear sign it's time to beef up your pre-release checks.

To bump up your DDR, consider improving automated tests, getting more thorough code reviews, or even running more user acceptance tests before the big launch. The better your DDR, the happier your users---and fewer "uh-oh" moments post-launch!

6. Bug Rate: The Red Flags in Your Code 🐛

Bug Rate measures how frequently those pesky bugs show up in your code. A high bug rate can be a big red flag, signaling that code is either being rushed out the door or written by someone still learning the ropes. Industry data suggests that experienced teams typically aim for fewer than 10 bugs per 1,000 lines of code.

Quick Example:

Your team launches a new feature, and within hours, 15 bugs are reported. If you're regularly seeing this kind of thing, it's a sign that code reviews or testing need more attention---or that your devs might need more time to do it right.

7. Mean Time to Recovery (MTTR): The Comeback Kid 🛠️

MTTR is all about how quickly your team can get back on its feet after a system crash.

It's your disaster recovery stopwatch, showing how fast you can bounce back from a mess. Ideally, you want a low MTTR---think minutes, not hours.

Quick Example:

Your website crashes at 2 PM, and your team has it back online by 2:15 PM. That's an MTTR of 15 minutes. If it usually takes your team an hour to recover, it might be time to refine your incident response plan.

8. Velocity: The Sprint Speedometer 🏃‍♂️

Velocity measures how much work your team gets done during a sprint. It's your productivity gauge, but don't forget---it's not always apples to apples across different teams. What's important is tracking how your velocity changes over time, not just comparing numbers.

Quick Example:

Last sprint, your team completed 50 story points. This sprint, they finished 55. A higher velocity could mean your team is getting into a groove---or it could mean they took on easier tasks. Keep an eye on consistency here.

9. Cumulative Flow: The Traffic Report for Tasks 🚦

Cumulative Flowshows you where tasks are piling up in your workflow.

Think of it as a traffic report for your project---if tasks are stuck in one stage too long, you've got a bottleneck.

Quick Example:

You notice a bunch of tasks lingering in "code review" while others move smoothly. This might mean you need more reviewers or better-defined criteria to keep things moving.

10. Deployment Frequency: Code Hits the Road 🛣️

Deployment Frequency tracks how often your team pushes code into production. More frequent deployments generally mean your team is agile and adaptable---just make sure you're not sacrificing quality for speed. Learn more on What is Deployment Frequency

Quick Example:

Your team deploys updates twice a week. That's good if those updates are solid, but if each deployment leads to bugs, it might be time to dial it back and focus on quality.

Also read: Deployment Frequency 101: Leverage DORA Metrics to Improve Software Delivery

11. Queue Time: The Waiting Room 🕰️

Queue Time measures how long tasks sit in a waiting state, like when they're stuck in the "to-do" pile. Long queue times could signal inefficiencies in your process, like too few team members handling too many tasks.

Quick Example:

If tasks are sitting for days waiting for QA approval, it's a sign that either the QA team needs help, or the criteria for moving tasks forward need streamlining.

12. Scope Completion Rate: Can You Finish What You Start? 📋

Scope Completion Rate tells you how much of the work your team planned to do actually gets done. If your team's regularly leaving tasks unfinished, it might mean they're biting off more than they can chew.

Quick Example:

Your team planned to complete 20 tasks this sprint but only finished 15. A low scope completion rate like this might indicate that your team needs to set more realistic goals or manage their time better.

13. Scope Added: The Sneaky Creep 🌱

Scope Added tracks how often new tasks get added after a sprint starts. A high rate here can be a sign of poor planning or, worse, scope creep---when your project's goals keep expanding without adjusting timelines or resources.

Quick Example:

You start a sprint with 10 tasks, but by the end, you've added 5 more. That's a 50% increase in scope, which might mean your team isn't scoping out the work thoroughly enough during planning.

14. Lead Time: The Clock's Ticking ⏰

Lead Time measures the total time from when a task is created to when it's completed. It's like the full journey from idea to execution. A shorter lead time usually means your team is efficient, while a longer one might signal delays or bottlenecks in your process.

Quick Example:

A feature request comes in, and it takes two weeks to go from concept to deployment. If similar tasks used to take one week, it's time to investigate what's slowing things down---maybe there are approval delays or too many handoffs between teams.

Also read: Lead Time for Changes: A Deep Dive Into DORA Metrics & Their Impact on Software Delivery

15. Churn Rate: Spinning Your Wheels 🌀

Churn Rate tracks how often your code gets rewritten or significantly changed shortly after it's been written. High churn can be a sign that your initial approach wasn't quite right or that requirements are shifting too much.

Quick Example:

Your team writes a feature, and within a week, they have to rewrite half of it because the initial implementation didn't meet the needs. If this keeps happening, it's a sign that more time should be spent on planning or that the requirements need to be clearer from the start.

What KPIs Should You Keep an Eye On? The Must-Have Metrics for Your Success Checklist 📊

Wondering which KPIs are worth your attention? Focus on the ones that give you the full picture of your team's performance and progress. Look out for:

Coding Efficiency: How fast and smoothly your code flows from "Hey, I wrote this!" to "Wow, it works!"
Collaboration Metrics: How well your team's playing in sync---like a well-rehearsed band or a synchronized swimming team. Learn more about Cross-Functional Collaboration: Aligning Engineering, Design, and Product Teams for Success
Predictability Metrics: How accurately you can forecast project outcomes, making your predictions as reliable as a weather app (but more accurate!). Learn more about Predictive Analytics in Engineering: Forecasting Productivity and Project Success
Reliability Metrics: How solid your code is and how well your testing catches those sneaky bugs before they become show-stoppers.

These KPIs help you avoid surprises and keep your projects on track. Think of them as the essentials for your success toolkit---no fluff, just the good stuff!

Wrapping it up: Middleware's DORA Metrics---Your KPI Tracking BFF! 🌟

So, here's the lowdown: KPIs aren't just numbers---they're your secret weapon for smart decision-making. It helps you navigate the twists and turns of your engineering productivity like a pro. And when you add Middleware's DORA metrics into the mix, you've got an unbeatable team. Middleware takes the guesswork out by effortlessly tracking DORA metrics like deployment frequency, lead time, change failure rate, and mean time to recovery. It's like having a personal sidekick that keeps an eye on your KPIs and ensures you're always on the right track. With Middleware, you're not just reacting to problems---you're anticipating them and steering your software development toward success. Check out our

Also read: What Are DORA Metrics & How They Can Help Your Software Delivery Process

FAQs

What's a software development KPI?

A software development KPI (Key Performance Indicator) is a measurable value used to assess the effectiveness and efficiency of development processes, including metrics such as code quality, deployment frequency, and lead times. KPIs help in evaluating progress towards specific goals and improving overall performance.

What tools should I use to track KPIs?

To track KPIs, including DORA metrics, use Middleware for comprehensive performance tracking, along with Jira for project management and GitHub for code insights.

Cycle Time vs. Lead Time: Understanding the Metrics That Drive Engineering Efficiency

Rajni Rethesh — Fri, 28 Mar 2025 13:54:58 +0000

If you've ever found yourself lost in the endless abyss of project timelines, wondering why things take longer than expected, you're not alone. Understanding Cycle Time and Lead Time can be the key to unlocking efficiency and keeping your projects on track. These two time-based metrics are the unsung heroes of project management, software development, and manufacturing---but they're often misunderstood.

So, let's break it down in the simplest, no-nonsense way.

Want to take your development efficiency to the next level? Try Middleware today and see how precise tracking of Lead Time and Cycle Time can supercharge your software delivery process! Book a demo now!

What is Lead Time?

Lead Time is the granddaddy of all time-tracking metrics. It measures the total time taken from the moment a request is made (think of placing an order) to when the final product is delivered. In software development, it's the time from when a feature is requested to when it's deployed.

Note: The above image is from a Dora Metrics case study we did on Flutter open source repository. Learn more on Flutter's Engineering Pipeline: A Great Workflow with Minor Hiccups

Example:

Imagine ordering a pizza. The Lead Time starts the moment you place the order and includes the entire process: prepping, baking, boxing, and finally, delivery to your doorstep. Learn more on Lead Time for Changes: A Deep Dive Into DORA Metrics & Their Impact on Software Delivery

What is Cycle Time?

Cycle Time is a more focused metric that measures the time it takes to complete a task once work begins. Unlike Lead Time, which includes waiting periods, Cycle Time starts ticking only when actual work begins.

Cycle time typically includes time spent on rework, reviews, and other activities that are part of the overall process from start to finish, not just the core work itself.

Example:

Going back to our pizza analogy, Cycle Time starts when the chef begins making your pizza and stops when it's ready to leave the oven. The delivery time? Not part of Cycle Time---that's still within Lead Time.

Why Do These Metrics Matter in Efficient Software Delivery?

If you're running a business, managing a team, or simply trying to meet deadlines, tracking both metrics can help you:

Identify bottlenecks: Long Lead Times might indicate delays in starting work, while long Cycle Times suggest inefficiencies in execution.
Improve predictability: Knowing how long things take from start to finish allows better planning and resource allocation.
Optimize workflow: Shortening Cycle Time can improve efficiency, while reducing Lead Time enhances customer satisfaction.

How Middleware Tracks Lead Time and Cycle Time for Software Development Efficiency?

Middleware provides an in-depth analysis of both Lead Time and Cycle Time, helping engineering teams optimize their development pipelines. With automated tracking and visualization tools, Middleware allows teams to:

Monitor real-time data: Get instant insights into how long it takes to move tasks from ideation to deployment.
Identify inefficiencies: Spot delays in development, code review, or deployment processes and take corrective action.
Optimize team workflows: Middleware helps teams fine-tune their development process by pinpointing the exact stages causing slowdowns.
Improve sprint planning: By analyzing historical data on Cycle and Lead Times, teams can better predict delivery timelines and allocate resources effectively.

By leveraging Middleware, software teams can enhance their efficiency, speed up deployments, and ensure a smoother workflow from request to release. Whether you're managing a startup or a large-scale enterprise, tracking these metrics with Middleware provides the clarity needed to make data-driven decisions and keep your development process running like a well-oiled machine.

Also read: How Qoala Optimized their Development Pipeline with Middleware

How to Reduce Lead Time and Cycle Time?

Streamline processes: Automate repetitive tasks and reduce unnecessary approvals.
Improve prioritization: Ensure high-priority work starts as soon as possible.
Remove roadblocks: Identify dependencies and clear them in advance.
Adopt agile methodologies: Regular iterations and feedback loops help speed up both Lead and Cycle Times.

Final Thoughts

While Lead Time and Cycle Time may seem like minor details, they play a major role in overall project success. Think of them as your project's fitness tracker---giving you valuable insights into where time is spent and where improvements can be made. Whether you're developing software, manufacturing products, or delivering pizzas, mastering these metrics will help you move faster, smarter, and with fewer surprises.

So, next time someone asks why a project is taking so long, you'll know exactly where to look!

Want to take your development efficiency to the next level? Try Middleware today and see how precise tracking of Lead Time and Cycle Time can supercharge your software delivery process! 🚀 Book a demo now!

FAQs

What is the difference between Lead Time and Cycle Time?

Lead Time is the total time from when a request is made to when the final product is delivered. Cycle Time, on the other hand, only measures the time from when work begins to when it is completed, excluding waiting periods.

What is the difference between WIP and Lead Time?

Work In Progress (WIP) refers to tasks that are currently being worked on but are not yet completed. Lead Time includes WIP as well as any waiting time before work begins and after it is completed until delivery.

What are the metrics for manufacturing Cycle Time?

In manufacturing, Cycle Time metrics typically include:

Process Cycle Time: Time taken to complete one unit of production.
Machine Cycle Time: Time a machine takes to complete its function.
Lead Time to Cycle Time Ratio: Measures efficiency by comparing total Lead Time to actual work time.
Takt Time: The rate at which products must be produced to meet demand.

What is the difference between Cycle Time and Lead Time in DevOps?

In DevOps, Lead Time refers to the time from code commit to production release, including all review and testing phases. Cycle Time focuses only on active development time, from when work begins on a feature to when it is ready for deployment.

Best Project Management Tools in 2025: Features, Pricing & Pros/Cons

Rajni Rethesh — Mon, 24 Mar 2025 16:33:44 +0000

Project management tools have become the backbone of modern businesses, streamlining workflows, improving collaboration, and ensuring deadlines are met. In 2025, the competition among project management software is fiercer than ever, with AI-driven automation, real-time analytics, and enhanced integrations taking center stage.

In this guide, we'll explore the best project management tools in 2025, covering features, pricing, and pros/cons, along with a comprehensive comparison table to help you pick the right tool for your needs.

Top Project Management Tools in 2025

1. ClickUp

Best for: Versatile task management and team collaboration.

ClickUp is an all-in-one project management tool that caters to various industries, offering a highly customizable workspace with powerful automation. It integrates seamlessly with popular apps and provides robust task tracking, making it ideal for growing teams.

Features:

Fully customizable task management.
AI-powered automation.
Native document collaboration.
Time tracking and goal setting.
Integrations with over 1,000 apps.

Pricing:

Free: Limited features
Unlimited: $5/user/month
Business: $12/user/month
Enterprise: Custom pricing

Pros:

✅ Highly customizable workspace.
✅ AI-powered automation reduces manual effort.
✅ Free plan with decent features.

Cons:

❌ Steep learning curve.
❌ Can feel overwhelming with too many features.

2. Monday.com

Best for: Intuitive visual project tracking.

Monday.com simplifies project tracking with its intuitive interface and highly visual task management system. It is designed for teams that need a collaborative and easy-to-use platform for managing workflows efficiently.

Features:

Drag-and-drop UI for task management.
Custom workflows and automation.
Resource and workload management.
Powerful reporting and dashboards.

Pricing:

Free: Up to 2 users
Basic: $8/user/month
Standard: $10/user/month
Pro: $16/user/month
Enterprise: Custom pricing

Pros:

✅ User-friendly interface with color-coded tasks.
✅ Strong reporting and automation features.
✅ Scalable for teams of all sizes.

Cons:

❌ Limited features on the free plan.
❌ Can become expensive for larger teams.

3. Trello

Best for: Simple and visual task management.

Trello is a straightforward project management tool that uses Kanban boards to help teams stay organized. Its drag-and-drop functionality makes it perfect for small teams and freelancers who need an easy way to manage tasks.

Features:

Kanban-style board organization.
Power-Ups for extended functionality.
Drag-and-drop task management.
Automation with Butler.

Pricing:

Free: Limited features
Standard: $5/user/month
Premium: $10/user/month
Enterprise: Custom pricing

Pros:

✅ Simple and easy to use.
✅ Great for small teams and personal projects.
✅ Free version is useful for basic needs.

Cons:

❌ Limited reporting and analytics
❌ Not ideal for complex project management.

4. Asana

Best for: Advanced task tracking and workflow automation.

Asana is a feature-rich tool that allows teams to track tasks, set priorities, and automate workflows with ease. It is a great choice for businesses looking to improve collaboration and efficiency.

Features:

Task and project timeline views.
Workflow builder for automation.
Custom fields and templates.
Portfolio management.

Pricing:

Free: Basic task management
Premium: $10.99/user/month
Business: $24.99/user/month
Enterprise: Custom pricing

Pros:

✅ Powerful automation features.
✅ Excellent collaboration tools
✅ Integrates with most major tools.

Cons:

❌ Can feel complex for new users
❌ Premium features are locked behind higher pricing.

5. Jira

Best for: Software development and agile teams.

Jira is a robust tool designed for software teams that follow Agile methodologies. It offers excellent issue tracking and backlog prioritization, making it ideal for DevOps workflows. When it comes to tracking the productivity of your engineering team, Middleware Jira Reports Plugin is a game-changer. Designed for teams that need real-time insights, this plugin allows you to generate custom reports and dashboards without leaving Jira. It provides a detailed view of your team's performance metrics, including sprint progress, issue resolution times, and more.

Features:

Agile boards for sprint planning.
Bug tracking and backlog prioritization.
Advanced reporting and analytics.
Integration with GitHub and CI/CD tools.

Pricing:

Free: Up to 10 users
Standard: $7.75/user/month
Premium: $15.25/user/month
Enterprise: Custom pricing

Pros:

✅ Ideal for agile and DevOps teams.
✅ Strong integrations with development tools.
✅ Excellent issue-tracking capabilities.

Cons:

❌ Not beginner-friendly.
❌ Can be overwhelming for non-tech teams.

The Best Jira Plugins for Technical Project Managers

6. Notion

Best for: Documentation and project collaboration.

Notion blends project management with note-taking, making it perfect for teams that need a flexible workspace to organize information, tasks, and collaboration in one place.

Features:

Customizable workspace.
Rich-text documents with embedded databases.
Task and project tracking.
Real-time collaboration.

Pricing:

Free: Basic features
Plus: $8/user/month
Business: $15/user/month
Enterprise: Custom pricing

Pros:

✅ Highly flexible and customizable.
✅ Combines documentation and task management.
✅ Great for knowledge management.

Cons:

❌ Not ideal for large-scale project tracking.
❌ Can feel overwhelming with too many customization options.

7. Airtable

Best for: Teams that need database-powered project management.

Airtable combines the best of spreadsheets and databases to provide an ultra-flexible project management experience.

Features:

Customizable relational database.
Kanban, grid, calendar, and gallery views.
Automation and integrations.
Pre-built templates for various use cases.

Pricing:

Free: Basic features
Plus: $10/user/month
Pro: $20/user/month
Enterprise: Custom pricing

Pros:

✅ Excellent for data-heavy workflows.
✅ Powerful automation and integrations.
✅ Customizable views for different project needs.

Cons:

❌ Can be overkill for simple task management.
❌ Requires some learning for advanced features.

8. Middleware - Best Tool to Track Engineering Productivity

Best for: DevOps-driven project management with real-time performance tracking.

Middleware is a project management tool designed for software development and DevOps teams. It helps businesses track performance metrics, manage deployments, and optimize software delivery pipelines with actionable insights.

Features:

Real-time tracking of deployment frequency and lead time.
AI-powered insights for performance optimization.
Seamless integration with DevOps tools like GitHub and Jenkins.
Customizable dashboards for data visualization.
Automated alerts for performance deviations.

Pricing:

Free: Limited access to monitoring features.
Standard: $39/user/month for mid to large teams
Enterprise: Custom pricing with additional analytics capabilities.

Pros:

✅ Tailored for DevOps teams
✅ Advanced real-time insights improve efficiency
✅ Easy integration with existing development tool

Top 20 DevOps Tools for 2025

Conclusion: Choose the Right Project Management Tool

The best project management tool for you depends on your team's needs, project complexity, and budget. Whether you're a startup looking for a free and intuitive tool or a growing business in need of advanced automation and reporting, there's a perfect fit for you.

If you're in software development, the Middleware Jira Plugin is a must-have, helping you optimize workflows, track Dora Metrics, and streamline software delivery like a pro.

Want to improve your project management game? Try Middleware today!

FAQs

1. What is the best tool for project management?

The best tool depends on your needs! Trello is great for simple tasks, Jira for software teams, and Asana for collaboration. For all-in-one power, ClickUp and Monday.com are solid choices.

2. What are the pros and cons of project management?

Pros: Better organization, improved team collaboration, and efficient task tracking.

Cons: Can be time-consuming to set up, requires team buy-in, and some tools have a steep learning curve or high cost.

3. Which platform is best for project management?

It varies! Middleware is the best tool to track engineering productivity, Monday.com and ClickUp are user-friendly, Jira is great for agile teams, and Notion is flexible for both personal and team projects. Pick what fits your workflow best!

4. Which tool is most likely to be used in project management?

Middleware for engineering productivity, Jira for software teams, Trello for visual task management, and Microsoft Project for enterprise-level planning. Most teams use a mix of tools based on their needs.

The 2024 DORA Report: State of DevOps Breakdown Summary

Shivam Chhuneja — Wed, 06 Nov 2024 12:49:05 +0000

For the past 10 years, we have seen Accelerate State of DevOps: Report released annually, built upon the insights and data from thousands of industry respondents. The 2024 DORA Report was published recently.

As you can probably guess, the report is packed with data and insights that not only give a deep dive into software delivery and operations but also pack a lot of value for Engineering Managers, Engineering Leaders and Software Engineers.

So, we thought why not take our notes from this year's DORA Report, expand on them a little bit and share them with you in an article. This blog provides a summarized view of the report's findings, highlighting what matters most for team productivity, AI integration, and platform engineering.

Of course since this is a summary it cannot cover all the nuances that the DORA team have covered in their complete report, so while we do think this article is extremely valuable we still believe that just going through this blog will not be enough especially when it comes to context and methodology.

"Granny sighed. "You have learned something," she said, and thought it safe to insert a touch of sternness into her voice. "They say a little knowledge is a dangerous thing, but it is not one half so bad as a lot of ignorance."\
-- Terry Pratchett, Equal Rites

If you want exceptional visibility & actionable insights for your org's software delivery process, Middleware is your go to tool to get started within minutes & start a process transformation!

1. State of Software Delivery Performance

Performance benchmarks this year look as crazy as they do every year if we compare elite teams to low performing teams.

Chart from The 2024 DORA Report, Pg 14

Elite teams show unparalleled efficiency and recovery rates, which definitely say something about the value of mature DevOps practices.

127x faster lead time for changes
182x more frequent deployments
8x lower change failure rate
2293x faster failure recovery time

Elite teams set a high bar, showing how consistent, high-velocity performance can dramatically improve software delivery outcomes.

However, we loved what the report says about elite performance: "The best teams are those that achieve elite improvement, not necessarily elite performance"

This is something we have mentioned in our articles around DORA Metrics as well. 2 distinct teams, of distinct size, delivering distinct software, to distinct users should not be compared in absolute terms without context.

2. AI's Role in Software Delivery

AI adoption is picking up rapidly, yet concerns around trust in AI-generated code remain.

Chart from The 2024 DORA Report, Pg 20

This year's report dove into how teams are adapting to AI integration and its impact on productivity.

39% of developers report little or no trust in AI-generated code quality
81% say their organizations have shifted priorities to increase AI incorporation into applications
At an individual level 75.9% are relying on AI in things like writing code, summarizing info, documentation, writing tests etc.

Building trust and transparency in AI tools is essential as organizations increasingly adopt AI in development workflows. Even though AI is helping people do meaningful work with increased productivity the overall sentiment still remains somewhat of concern.

One participant even likened the need to evaluate and modify the outputs of AI-generated code to "the early days of StackOverflow, [when] you always thought people on StackOverflow are really experienced, you know, that they will know exactly what to do. And then, you just copy and paste the stuff, and things explode " (P2). --- 2024 DORA Report, Pg 24

3. Platform Engineering's Impact on Developer Experience

Platform engineering transforms developer workflows, enabling self-service and reducing friction.

Platforms behave in a similar way as transformation efforts, the early effects tend to be positive with a dip in the mid-term and recovery as the internal platform matures.

Chart from The 2024 DORA Report, Pg 50

8% increase in individual productivity with internal platforms
6% gain in productivity at the team level
And here we go: an 8% decrease in throughput & 14% decrease in change stability!

While platforms smoothen out the processes, they introduce new layers of complexity that can impact throughput and stability. Increased handoffs and dependencies may hinder speed.

Platform teams should balance automation with flexibility to prevent these operational slowdowns.

Overall, internal platforms show great promise in boosting productivity and team efficiency across development organizations however they are not a cure-all!

4. Developer Independence and Self-Service Workflows

Developer autonomy, a core principle of platform engineering, correlates strongly with productivity gains.

Self-service capabilities reduce dependencies on enabling teams and accelerate project timelines(owing to less handoffs and touchpoints within the process).

5% productivity gain at individual and team levels for developers without an "enabling team"

Interestingly, the impact on productivity of having a dedicated platform team was negligible for individuals. However, it resulted in a 6% gain in productivity at the team level. This finding is surprising because of its uneven impact, suggesting that having a dedicated platform team is useful to individuals, but the dedicated platform team is more impactful for teams overall.

Since teams have multiple developers with different responsibilities and skills, they naturally have a more diverse set of tasks when compared to an individual engineer. It is possible that having a dedicated platform engineering team allows the platform to be more supportive of the diversity in tasks represented by a team. --- 2024 DORA Report, Pg 52

5. The Role of Documentation in Developer Productivity

While Agile emphasizes "working software over documentation," DORA's findings highlight that quality documentation is essential for effective development.

Strong documentation isn't just about quantity but ensuring content is findable and reliable. A user-centered approach to documentation supports developer independence and enables smoother workflows.

Chart from The 2024 DORA Report, Pg 63

Focus on findability and reliability to keep documentation useful
Promote a sustainable documentation culture that maintains relevance
User-focused documentation amplifies technical capabilities and organizational impact

In short, well-maintained, user-centered documentation is foundational to productivity.

6. User-Centric Focus and Transformational Leadership

Focusing on the user in software development yields notable gains, with transformational leadership playing a significant role.

Leaders who empower teams and align projects with user needs increase productivity, satisfaction, and overall organizational performance.

User-centered development correlates with a 40% boost in organizational performance(2023 metric)
Strong transformational leadership(increasing transformational leadership by 25%) leads to a 9% productivity increase

Organizations combining user-centric practices with transformational leadership push team towards success and impactful, user-aligned products.

Let's Wrap This

The 2024 DORA Report highlights that robust DevOps practices, responsible AI integration, and thoughtful platform engineering are key to high performance.

For teams looking to achieve these benchmarks, Middleware offers a straightforward, quick way to measure and refine your DORA metrics, Project Flow Metrics, Bottleneck analysis out of the box to refine your software delivery practices.

Electron Repo—Unpredictable Metrics, Efficient Deployments

Rajni Rethesh — Mon, 30 Sep 2024 11:18:29 +0000

There's a high probability that a bunch of desktop apps you are using are built with Electron. Electron is a framework that combines Node.js, parts of Chromium, and a layer of native code. Apps like Visual Studio Code, Slack, Atom, WhatsApp, or even the installer for Microsoft's Visual Studio use it to build for Windows, macOS, and Linux. Though Electron has its own fanbase, many flock to its competitor, Tauri, for its significantly better startup times and lower memory consumption.

Having said that, with over 114k stars and 15.3k forks, and a thriving community of 1300 contributors, Electron is not going anywhere anytime soon.

As part of our 100 days 100 case studies of top Github repositories, we also toyed around the Electron repo, analyzing their dora metrics using Middleware Open-source.\
Check out how Middleware helps track a project's engineering pipeline in this live demo.

If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community and subscribe to the newsletter for exclusive case studies and more!

Understanding Electron Dora Metrics: The Domino Effect

Team Electron displayed a good performance in the months of June and August with a great deployment frequency of 92 and 170 releases. This shows that the team follows an aggressive continuous deployment strategy. However, July showed a sluggish movement with only 6 deployments.

Digging a little deeper, it was found that the biggest bottlenecks in their development cycle in July were the first response time, rework time, and merge time. All three were way above the benchmark set by the 2023 State of DevOps Report. None of these even came close to meeting the standard, making these three areas the major blockers holding things up.

Let's see the numbers:

First Response Time: Back in June 2024, it took about 25.12 hours to respond. Then, it shot up to 68.17 hours in July, only to drop back down to 20.02 hours in August.

Rework Time: June's rework time was 20.23 hours, but things went wild in July, skyrocketing to a whopping 90.79 hours, before finally cooling off to 14.38 hours in August 2024.

Merge Time: In June, merge time was 35.26 hours, then it blew up to a crazy 282.73 hours, only to fall to a much better 16.53 hours in August.

And just like a domino effect, the wild fluctuations in these three metrics really threw off their cycle and lead time. Cycle Time vs. Lead Time: Understanding the Metrics That Drive Engineering Efficiency

Also read: Hugging Face and DORA Metrics: Fast Code, Slow Response

Electron Engineering Pipeline: Busy Putting out fires

Delays in code reviews, along with the back-and-forth nature of iterations and corrections within pull requests (PRs), can significantly contribute to the extended first response and rework times. When code reviews lag, it creates bottlenecks, forcing developers to spend more time on revisions rather than moving forward with new tasks.

Another significant factor is the disproportionate amount of time allocated to bug fixes---around 45% of the total development cycle. This high percentage suggests that developers are spending nearly half their time putting out fires instead of focusing on innovation. As a result, critical resources that could be directed toward launching new features and enhancing performance are instead diverted to addressing bugs.

In fact, only 25% of their time is spent on performance improvements, and a mere 20% goes towards developing new features. This imbalance highlights a reactive rather than proactive approach to development, ultimately stalling progress and preventing teams from achieving their strategic goals.

Also read: Node.js Dora Metrics: Faster Deployments, Slower Merge Times

The Key Strengths They Can Cash In On

Robust CI/CD Pipelines: The repo's loaded with workflow files like build.yml, pipeline-electron-build-and-test.yml, and pipeline-segment-electron-gn-check.yml. These files basically handle testing, building, and deploying automatically, cutting down on the time developers would normally spend doing things manually. With less manual intervention, teams can push out changes faster, which helps reduce those long first response and rework times. By speeding up these processes, merge times will naturally shrink too.

Frequent Smaller Changes: Instead of dropping massive, monolithic updates that can lead to a ton of back-and-forth, the team is rolling out smaller, bite-sized changes more often. This means PRs are more focused and easier to review, like PRs #43207 and #43332 .

The result?

Fewer issues to fix, faster first responses, and quicker rework cycles. Smaller, frequent changes lead to smoother merges since you're not dealing with a mountain of code all at once.

Comprehensive Automated Testing: The workflows also have a heavy focus on automated testing, with files like pipeline-segment-electron-test.yml and test.yml keeping code quality in check.

Automated testing ensures that new code doesn't break anything major, which means fewer bugs, less rework, and shorter review times. With solid testing in place, they can catch errors early on, which means smoother sailing when it comes to merging PRs later down the line.

Also read: What Developer Productivity Metrics Should An Engineering Manager Really Track?

Leaving Thoughts: Electron Repo---Unpredictable Metrics, Efficient Deployments

Electron should really lean into their active, thriving community and strong CI/CD pipeline to push their product through that final stretch. They also need to step up their documentation game---it'll make life a lot easier for developers, helping them tackle projects with more confidence and efficiency.

If you find these learnings interesting, we'd really encourage you to give a shot at Dora metrics using Middleware Open Source. You could follow this guide to analyze your team or write to our team at productivity@middlewarehq.com with your questions and we'll be happy to generate a suggestion study for your repo --- free!

Also, If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community and subscribe to the newsletter for exclusive case studies and more!

Did you know?

Electron was originally created by GitHub for their desktop app, Atom. Since then, it's grown into a framework used by huge names like Microsoft, Discord, and even Facebook!

Further Resources

TypeScript Dora Metrics Analysis: They Just Need a Smart Game Plan!

Rajni Rethesh — Fri, 27 Sep 2024 04:33:16 +0000

Ask any developer, and they'll tell you: catching errors during the development phase is a game-changer in the development cycle.

As an answer to their prayers, Typescript came into existence. Developed by Microsoft, TypeScript is a high-level, open-source programming language that allows developers to identify and fix errors early, rather than waiting for runtime to strike.

This proactive approach significantly cuts down on bugs and enhances overall code quality. With its emphasis on safety and maintainability, TypeScript has become a core option for projects of all sizes.

Plus, with a thriving community of nearly 800 contributors continuously improving the language, TypeScript is almost a movement! To gain deeper insights into this movement, we analyzed their engineering Dora Metrics using Middleware Open Source. Keen to know what we found?

If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community and subscribe to the newsletter for exclusive case studies and more!

1. Deployment Frequency, Merge Time, First Response Time & Rework Time Analytics

TypeScript powers through rapid iterations and delivery, but the surge in merge time---from 1.76 days in June 2024 to 3.39 days in August 2024---indicates a major slowdown in review times, likely caused by complex pull requests or a backlog in reviews.

Let's delve deep and see what story their number narrates:

June 2024: They had a good 92 deployments

July and August 2024: They hit a deployment slump in July with 26 deployments but managed to claw their way back up in August with 66 deployments.

It's like they pressed pause, then fast-forwarded to resume their usual deployment groove.

Upon closer examination, we observe a steady decrease in the cycle time, dropping from 12.56 days in June to 11.09 days in August 2024, and a reduction in lead time from 13.0 days in June to 11.42 days in August.

However, the fluctuations in first response time, merge time, and rework time raise some concerns that need to be addressed.

Merge Time: In June, it was 1.76 days, which reduced to 0.99 days in July and then increased to 3.39 days in August.
First Response Time: It started at 4.11 days, climbed to 8.05 days, and ultimately decreased to 3.01 days.
Rework Time: Initially, it was 6.59 days, then it dropped to 0.45 days, before rising again to 4.68 days.

The instability in these metrics signals potential roadblocks in their workflow. The rising merge time suggests that while deployments may have increased, the review process is becoming a bottleneck. Meanwhile, the erratic first response and rework times indicate that the team may be struggling to keep pace with review demands and address issues promptly.

Also Read: How Bootstrap maintained a robust merge time

2. Their Strengths: A Powerful Ecosystem and A Solid Backing

Microsoft Backing: TypeScript is maintained by Microsoft, ensuring robust support and regular updates, which adds to its credibility and reliability.

Active Contributions: With over 749 contributors, the repository benefits from a diverse and active group of developers who continuously improve and maintain the project.

Extensive Ecosystem: A vast ecosystem of libraries and tools can help automate and streamline various aspects of the development and deployment process, leading to more frequent deployments. Also, the availability of a wide range of tools can help developers implement changes more efficiently, reducing the lead time for changes.

Robust CI/CD Pipelines: Automated testing and deployment ensure that changes can be reliably deployed frequently without manual interventions.

Clear Contribution Guidelines: The documentation and guidelines, present in files like CONTRIBUTING.md, set clear expectations and reduce friction for new contributions.

Also, check out where Hugging Face shines the most

3. Leveraging Strengths to Build a Robust Engineering Roadmap

Capitalize on Active Contributions

With over 749 active contributors, the wealth of ideas and improvements should not be underestimated. Actively engage the community and foster collaboration, enabling the team to rapidly iterate and enhance features, creating a continuous cycle of improvement.

Bug Fixes: Taming the Beast with Quality Code and Smart Strategies

The TypeScript repo currently spends about 35% of its time fixing bugs. To bring this number down, they should leverage their automated testing expertise to catch issues before they escalate. By implementing unit tests, integration tests, and end-to-end tests, they can ensure that new code doesn't introduce bugs and that all aspects of the application are thoroughly covered.

Strategies to Resolve Delays in Feature Pushes

They spend 40% of their time and efforts in feature pushes and to ensure these features are incorporated and finally deployed into production without delay and hiccups, they can follow the below-given steps:

1. Prioritize Communication

Implement short daily stand-ups to keep everyone updated on tasks and blockers.

2. Streamline the Review Process

Establish and iterate clear code review guidelines to speed up the process.
Assign reviewers to pull requests (PRs) right after they are created to prevent bottlenecks.

3. Automate Testing and Integration

Set up a strong CI/CD pipeline with automated tests for each PR to catch issues early.
Ensure high test coverage for new features to validate functionality before merging.

4. Set Realistic Deadlines

Break down complex features into smaller tasks for easier tracking and identification of delays.

5. Conduct Retrospectives

Hold post-mortem discussions after each feature rollout to identify successes and areas for improvement.

Our Take: TypeScript Has the Tools to Build a Winning Roadmap!

Team TypeScript has all the necessary ammo it requires to build a powerful engineering pipeline. It just needs to leverage it strategically to foster a positive cycle of productivity.

With continued focus on refining their processes, the TypeScript team is well-equipped to enhance their performance even further and set a benchmark for others in the community.

If you find these learnings interesting, we'd encourage you to give a shot at Dora Metrics using Middleware Open Source. You could follow this guide to analyze your team or write to our team at productivity@middlewarehq.com with your questions and we'll be happy to generate a suggestion study for your repo --- free!

We at Middleware really enjoy sharing our insights about productivity from the best teams and would feel really encouraged if you could share with your community/friends as well.

Before you go, here's TypeScript Trivia!

Microsoft provides an online tool called the TypeScript Playground, where developers can write and test TypeScript code directly in their browser. This interactive environment allows users to see how TypeScript compiles to JavaScript in real-time.

Flutter’s Engineering Pipeline: A Great Workflow with Minor Hiccups

Rajni Rethesh — Thu, 26 Sep 2024 04:39:15 +0000

Flutter gained a stronghold in the market with its ability to build beautiful, natively compiled applications for mobile, web, desktop, and embedded devices from a single codebase. Many top companies such as BMW, Alibaba, and Google use Flutter for their flagship apps. And it's not just the big players; developers love it too, with over 1,416 contributors and nearly 165k stars on GitHub.

If you're excited to explore these insights further and connect with fellow engineering leaders, join us in The Middle Out Community and subscribe to the newsletter for exclusive case studies and more!

How did this framework, launched by Google in 2017, climb the ranks so fast? Check out the Annual StackOverflow Survey Report 2023

What best practices have fueled its rise?

Where might this open-source repo fall short?

With Middleware OSS, we set out to analyze Flutter's engineering Dora Metrics.

Flutter's Engineering Pipeline: Robust Yet Vulnerable

Flutter's engineering pipeline seems robust with its high deployment frequency. Comparing it to the benchmarks established in the 2023 Accelerate State of DevOps Report, Team Flutter's performance stands out as exceptional.

However, July brought a significant anomaly, marked by a steep decline in Deployment Frequency, plummeting from 253 deployments in June to just 85 in July, before rebounding to 159 in August. Check out some effective strategies to improve deployment frequency

Several potential reasons for this anomaly could be:

1. High Rework Time:\
The significant jump in rework time, from 18.99 hours in June to 50.34 hours in July, suggests that more code had to be sent back for revisions or fixes. This could be due to:

A sudden influx of lower-quality code, requiring more fixes.
Inconsistent or delayed code reviews, causing defects to slip through and then be caught later in the cycle.

2. Longer Lead and Merge Times:\
Lead time surged to 133.57 hours, and merge time extended to 44.79 hours in July.

This points to a bottleneck in the approval and integration process, either due to:

Resource constraints (e.g., fewer developers or reviewers available during this period).
Review complexity, where the code changes were more substantial or complicated, requiring extra time for analysis and approval.

3. Community Engagement Fluctuations:

Open-source projects often see variations in contributor activity due to external factors like contributor availability, interest levels, or major events.

In July, the Flutter repo may have experienced a drop in active contributors or maintainers, leading to slower review cycles and fewer deployments.

Flutter's Strengths and Best Practices

Flutter has quickly established itself as a powerhouse in the app development ecosystem. A few key factors contribute to its strong performance and rapid growth:

Code Reusability: Write Once, Use Everywhere

Flutter's capability to share a single codebase across multiple platforms (iOS, Android, web, etc.) significantly reduces redundancy in development. By writing code once and deploying it across various environments, developers save valuable time and effort, leading to quicker feedback cycles and a consistent user experience.

CI/CD Made Easy: Smooth Sailing for Releases

The seamless integration of Flutter with continuous integration and continuous deployment (CI/CD) pipelines automates the testing and deployment processes. This automation ensures that every code change is tested and deployed efficiently, minimizing manual errors and speeding up release cycles. As a result, teams can respond swiftly to user feedback and release improvements or new features more frequently.

Also read: How to Leverage DORA Metrics to Optimize Your CI/CD Pipeline

Frequent, Low-Risk Changes

Instead of relying on massive feature drops, Flutter's community focuses on small, incremental updates that reduce the risk of bugs and ensure a steady deployment rate. This practice of delivering smaller, manageable changes enables faster feedback and more responsive iteration, which is crucial in open-source environments.

Active Community Engagement

Flutter's vibrant open-source community is one of its greatest assets, fueling rapid evolution through continuous contributions from over 1,400 developers. This active participation ensures quick issue resolution, regular feature proposals, and a fast development pace.

The efficiency of Flutter's review process is evident in pull requests (PRs) like #151126, which improved documentation with a cycle time of just 22 hours and 56 minutes, and PR #151073, addressing system-related fixes in 1 day, 20 hours, and 44 minutes. This swift handling of both minor and technical updates showcases a well-organized review system that supports fast, incremental changes, keeping Flutter both cutting-edge and stable.

Steady Deployment Cadence

With an active community making frequent contributions, Flutter maintained a steady cadence of deployments. Despite a dip in deployment frequency in July 2024, Flutter managed to push changes at a high rate, keeping the project moving forward smoothly. The combination of active contributors, automated CI/CD pipelines, and small, manageable updates enables this pace.

Improving Performance: How Flutter Can Prevent Future Anomalies Like July's Dip?

Flutter's strong engineering pipeline is evident in its regular high deployment frequency, efficient CI/CD integration, and vibrant community. However, the dip in deployment frequency during July points to areas for improvement that can enhance its overall performance.

1. Backup Reviewers During Community Downtimes

The spike in rework time could be attributed to a temporary slowdown in contributions from the Flutter community. While Flutter's community is typically active, the dip in July suggests a need for dedicated expert code reviewers, with backup reviewers ready to step in during slower periods like this to maintain consistency.

Automate Workflows, Distribute Complex Reviews

July saw a noticeable increase in lead and merge times, pointing to bottlenecks in the approval and integration processes. A solution to this problem is introducing automated approval workflows and distributing complex reviews among senior developers. Automating simpler checks will allow human reviewers to focus on more complex code, reducing lead and merge times while keeping code quality intact.

3. Host Hackathons; Recognize Contributors to Boost Community Engagement

Community engagement is one of Flutter's key assets, driving rapid development. However, fluctuations in contributor activity, such as in July, can lead to slower review cycles and fewer deployments. To counter this, Flutter can maintain contributor enthusiasm with regular hackathons, contributor recognition programs, or clear release goals. Keeping contributors motivated and engaged will prevent slow periods in development, leading to steadier deployment rates.

4. Make CI/CD Pipeline Efficient

Flutter's CI/CD pipeline is well-integrated, but the dip in deployment frequency suggests the system may not handle large volumes or complex updates as efficiently as needed. Improving the CI/CD pipeline's capacity by optimizing test suite performance and implementing parallel testing will allow Flutter to manage heavy workloads without slowing down deployments. By scaling up the pipeline, even during busy months, Flutter can maintain its impressive deployment cadence.

Also, check out some other interesting case studies of God-level repos. What are they doing right? What are the hiccups they are facing and learn from the best!

Dora Metrics Rating

On the Dora Metrics scale, Flutter scores a solid 8 out of 10. Their deployment frequency was impressive, showing strong performance despite a dip in July. They shifted gears in August 2024, maintaining lead times that, while fluctuating, stayed within the benchmarks set by the 2023 State of DevOps Report.

However, their primary weakness lies in rework time, which spiked in July. Although it decreased in August, it still remained above the benchmark. By streamlining their rework process, they could significantly improve their overall performance.

Flutter's Engineering Pipeline: A Great Workflow with Minor Hiccups

Flutter's engineering pipeline performs exceptionally well due to its strategic emphasis on collaboration, code reusability, and streamlined processes. The framework's ability to share a single codebase across multiple platforms minimizes redundancy and accelerates development cycles, allowing for quicker feedback and enhanced user experiences.

Moreover, the integration of CI/CD practices automates testing and deployment, ensuring efficient updates and high-quality releases. Coupled with an active and engaged community, these strengths create a robust ecosystem that not only supports rapid iteration but also fosters continuous improvement. This dynamic approach positions Flutter to adapt and thrive in the app development world.

We'd love to hear your thoughts on this case study and its actionable insights! If you're interested in analyzing and monitoring your project's engineering pipeline, give DORA metrics a try with Middleware Open Source.

Feel free to reach out to us at productivity@middlewarehq.com with any questions. We're more than happy to create a custom suggestion study for your repo---absolutely free!

Moreover, If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community. and subscribe to the newsletter for exclusive case studies and more!

Trivia

The Flutter community loves to share! There are thousands of packages available on pub.dev, allowing developers to add features and functionalities to their apps with just a few lines of code.

Bootstrap: Strong Merge and Cycle Times, but First Response Time Needs a Revamp

Rajni Rethesh — Tue, 24 Sep 2024 09:54:28 +0000

While Bootstrap is still fondly regarded by many, it's no longer the trendiest tech in town. A few years back, if you asked developers about their go-to front-end framework, chances were they'd rave about Bootstrap. Once the gold standard for styling web pages, it powered a staggering 24% of websites. Its straightforward, powerful design was crafted to speed up development with ready-to-use components and responsive layouts. However, in recent times, it has been somewhat overshadowed by Tailwind, which offers a more utility-first approach that many developers find appealing for its flexibility and customization.

Now that we've established Bootstrap's credentials, let's delve into how its open-source repository manages engineering productivity. Buckle up as we explore a deep dive into the Bootstrap case study---one of the most widely-used front-end frameworks---and uncover the secrets behind its success!

If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community and subscribe to the newsletter for exclusive case studies and more!

When it comes to optimizing engineering productivity, Middleware's open-source solution is the benchmark. With detailed metrics on deployment frequency, lead time, MTTR, and change failure rate, it provides unparalleled visibility into workflows.

We took our OSS for a spin by tracking the Dora Metrics of the Bootstrap repository, unearthing valuable insights along the way.

Get ready to explore the key areas of focus in this case study!

Also Read: What are DORA Metrics?

Bootstrap's Key Strengths and Paths for Improvement

Areas Where Bootstrap Shines

Bootstrap has demonstrated remarkable improvements in both Merge Time and Cycle Time.

In June 2024, the merge time stood at a lengthy 159.75 hours (or 6 days and 15 hours). However, by July 2024, this impressive figure plummeted to just 4.77 hours, further decreasing to 3.28 hours in August 2024.

Similarly, while Cycle Time in June was 24.85 days---high but still within the standard Dora metrics benchmark---it saw significant reductions to 5.88 days in July and then 2.68 days in August. Also, if interested, know more about React Native's Cycle Time triumphs .

So, how did they achieve such dramatic improvements?

A combination of quick PR reviews, automated testing, and effective CI/CD integration played crucial roles in accelerating merge times.

For instance, PR #110 showcased swift peer reviews and efficient CI/CD execution, directly contributing to the reduced merge time. Moreover, PR #123 leveraged automated testing to identify issues early, enabling faster merges.

Check out the exceptional Cycle Time of OpenVino

Area for Improvement: Enhancing First Response Time

When it comes to metrics that demand attention, First Response Time clearly stands out as a critical area for improvement. Analyzing the trends reveals a significant drop from 281.44 hours in June 2024 to 88.22 hours in July, and further down to 61.15 hours in August. Despite these positive strides, the First Response Time remains high, adversely affecting the overall Lead Time.

Key issues include prolonged waits for initial reviews and acknowledgment, as well as fluctuating response times that can create bottlenecks and disrupt workflow consistency.

Our Verdict

After analyzing Bootstrap's performance metrics, we assess its engineering performance at a solid 7. Over the past few months, merge time has seen impressive improvement, plummeting from 159.75 hours in June to just 3.28 hours in August---an excellent sign of efficient code integration.

Lead Time has also made significant strides, dropping from 18.37 days to 2.68 days, reflecting a remarkable enhancement in their workflow. Likewise, Cycle Time has decreased from 24.85 days to 2.68 days, showing that the overall development process has been effectively streamlined. Cycle Time vs. Lead Time: Understanding the Metrics That Drive Engineering Efficiency

However, there's still room for growth in First Response Time, which has reduced from 281.44 hours to 61.15 hours. While this is an improvement, it still falls short compared to the benchmark set by top performers highlighted in the Google's Annual Dora Report. Addressing this area could lead to even greater efficiencies and smoother project timelines.
Read: Key Metrics for Measuring Engineering Team Success

Bootstrap's Strength

Its engineering process includes automated testing, continuous integration, and modular design, ensuring high-quality, efficient releases. You can
A highly skilled core team of seasoned web developers oversees the project, manages releases, and reviews pull requests to maintain top-notch code quality.
Bootstrap fosters open collaboration, boasting over 160,000 GitHub stars and tens of thousands of forks, with contributors regularly submitting updates, bug fixes, and enhancements.
Clear contribution guidelines and code review processes ensure that all contributions meet high standards.
Lastly, well-structured project management, transparent roadmaps, and milestones keep development organized. With cutting-edge web technologies, comprehensive documentation, and regular security audits, Bootstrap continues to evolve and remain a highly adopted, reliable front-end framework.

Leveraging Strengths to Overcome Weaknesses

To address the gap in its engineering pipeline, Bootstrap can leverage its strengths in the following ways:

1. Streamlining the PR Review Process

To enhance Bootstrap's efficiency, the team should focus on streamlining the PR review process. Establishing a structured framework with designated reviewers can significantly reduce wait times for initial responses. Here are a few actionable steps:

Define Roles and Responsibilities: Assign specific reviewers to different areas of the codebase. This not only distributes the workload but also ensures that reviewers are familiar with the components they're reviewing. A clear assignment can be based on expertise, allowing for quicker evaluations and decisions.
Implement Regular Review Sessions: Schedule fixed time slots for reviewers to focus solely on PRs. This dedicated time can help maintain momentum and ensure that reviews aren't sidelined by other tasks. For more on optimizing code reviews, see Atlassian's Best Practices for Code Review.
Use a PR Checklist: Incorporate a checklist that contributors can use before submitting their PRs. This can help ensure that submissions are complete and adhere to coding standards, which can reduce the need for extensive back-and-forth during the review process. You can start with a simple checklist like the one found in GitHub's Guide to Pull Requests.

2. Implementing Comprehensive Automated Testing

Automated testing plays a crucial role in catching issues early in the development cycle. This practice not only accelerates the review process but also minimizes back-and-forth communication, leading to quicker merges. Here's how Bootstrap can leverage automated testing:

Integrate Continuous Integration/Continuous Deployment (CI/CD) Tools: Setting up CI/CD pipelines can automate the testing process whenever new code is submitted. Tools like Jenkins, CircleCI, or GitHub Actions can run tests on every pull request, providing immediate feedback to contributors. For more information on CI/CD best practices, check out The Twelve-Factor App and CI/CD for Beginners. Also, learn how to leverage Dora Metrics to optimize your CI/CD pipeline
Create a Comprehensive Test Suite: Develop a suite of automated tests that cover various aspects of the code, including unit tests, integration tests, and end-to-end tests. This ensures that different functionalities are tested and issues are identified early. For guidance on writing effective tests, visit The Ultimate Guide to Automated Testing.
Encourage Test-Driven Development (TDD): Foster a culture of TDD among contributors, where tests are written before the actual code. This approach not only leads to more robust code but also aligns with continuous testing practices. For insights into TDD, see Martin Fowler's Introduction to TDD.

3. Addressing Time Zone Differences

With a global contributor base, addressing time zone differences is essential for maintaining effective communication and collaboration. Here are some strategies Bootstrap can implement:

Coordinate Overlapping Hours: Identify time slots where team members' working hours overlap and schedule important meetings or reviews during these times. This can enhance participation and ensure that crucial discussions happen in real-time. For tips on managing distributed teams across time zones, refer to Harvard Business Review on Managing Global Teams.
Establish "Review Windows": Create specific time frames when contributors are available to review pull requests. Communicating these "review windows" can help manage expectations and improve the flow of feedback.
Utilize Collaboration Tools: Leverage collaboration tools like Slack or Discord to facilitate quick communication among team members across different time zones. These platforms allow for real-time updates and discussions, which can keep everyone aligned. For more on effective communication in remote teams, check out Slack's Remote Work Best Practices.

Conclusion: Bootstrap Excels in Merge and Cycle Time, But First Response Time Remains a Major Weak Spot

Bootstrap has successfully transformed its development efficiency, significantly reducing Merge Time from 159.75 hours to 3.28 hours and Cycle Time from 24.85 days to 2.68 days. These improvements reflect effective collaboration and streamlined processes. However, the First Response Time, currently at 61.15 hours, still poses a challenge, hindering momentum and frustrating contributors. To sustain their progress, Bootstrap should focus on enhancing First Response Time by actively engaging the contributor community, implementing automated acknowledgment systems, and improving communication practices, especially considering the diverse time zones of its contributors.

If you're facing challenges similar to Bootstrap's, consider diving into Dora metrics with Middleware Open Source. Our guide can help you evaluate your team's performance, or you can reach out to us at productivity@middlewarehq.com for a free tailored analysis of your repository.

Also, If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community and subscribe to the newsletter for exclusive case studies and more!

Also don't miss out on reading: Top 10 Tips to Get Started with Open Source and GSoC

Trivia

Did you know that Bootstrap now has its own icon library? Bootstrap Icons includes over 1,000 customizable icons, making it easy to add visual elements to your projects without relying on third-party libraries.\

Further Resources

Optimizing OpenStreetMap’s DevOps Efficiency: A Data-Driven DORA Metrics Analysis

Shivam Chhuneja — Mon, 23 Sep 2024 10:47:28 +0000

With the core goal to build a free, editable map of the world, OpenStreetMap's website repo is where all the magic happens.

This open-source project, powered by Ruby, runs on a great community of developers and cartographers across the planet.

We've been looking at some interesting repositories in our 100 days of Dora case studies series and have uncovered some interesting stuff already.

In this case study, we break down OpenStreetMap's DevOps game using DORA metrics, diving into real-world data to uncover how often code ships, how fast changes go live, and what's driving those numbers.

Understanding DORA Metrics

DORA metrics are the go-to for measuring how well software delivery and operations are performing within DevOps teams.

The four key DORA metrics are:

- Deployment Frequency: How often code makes it to production.

- Lead Time for Changes: How long it takes for a commit to land in production.

- Change Failure Rate: The percentage of deployments that break something and need an immediate fix.

- Mean Time to Restore (MTTR): How fast the team recovers from a production failure.

In this case study, we're zeroing in on Deployment Frequency and Lead Time for Changes---two metrics that directly reflect how fast and efficiently an organization delivers.

These numbers give a clear view of engineering speed and process slowdowns, which are crucial to keep improving in today's fast-paced development world.

If you want to dive a bit deeper into what Dora Metrics are and how you can leverage them for your team, you can check out one of our articles.

Key Findings

1. Deployment Frequency

The OpenStreetMap website repository pushed an average of 58 deployments per month over the last three months, signaling a robust culture of continuous delivery and rapid iteration.

Albeit these DF numbers aren't the best, especially considering the other God level repos that we saw, almost 60 average DFs are nothing to be shy about either.

Key Drivers of Deployment Frequency

Robust Automation Pipelines: The repo leans heavily on automation tools like docker.yml, lint.yml, and tests.yml to keep the build, test, and deployment processes running smoothly. With less manual effort and fewer human errors, they've cut down release times significantly.

Efficient Pull Request Handling: June 2024 saw an almost-instantaneous average merge time of just 10.08 seconds! Even with a slight increase in the following months, merge times remained under 6 hours---proof of an agile review process that keeps things moving fast.

Engaged Reviewer Community: Contributors like gravitystorm and kcne are on the ball when it comes to code reviews, keeping the process fast and thorough. Prompt code reviews facilitate quick integration of changes, maintain code quality, and maintain a collaborative development environment.

Pull requests such as #5056 and #5053 are great examples of this active engagement.

2. Lead Time for Changes

While the repository excels in deployment frequency, the Lead Time for Changes---averaging around 13.26 hours over the past three months---shows room for improvement. The first response time is really good as well compared to the averages.

Although a half-day lead time is impressive, especially for an open-source project with global contributors, shortening this could further boost development speed and efficiency.

Key Influencing Factors

First Response Time Variability: The time between a pull request's submission and the first reviewer's response fluctuated significantly.

July saw an average response time of 6.77 hours, compared to just 1.47 hours in June. That's a 4.6x increase. This multiple looks great on an investment portfolio but not much when it comes to the lead time of a repo ;)

These delays in initial feedback can slow down progress and it does compound, adding unnecessary time to the overall process.

Rework Time Fluctuations: Rework time---the period spent revising code after the initial review---dropped from 11.28 hours in June to 2.94 hours in August. It's bittersweet

While the improved rework time shows an improvement in code quality or review efficiency, rework still adds to the overall lead time and offers an opportunity for further optimization.

For example, PR #5016 ("Allow to edit revoked blocks") required significant rework due to its complexity, in turn extending its lead time.

While the repository maintains great lead times, placing more focus on reducing first response times and streamlining the rework process could drive even faster delivery cycles, enhancing both efficiency and development speed.

Diverse Contributions Pushing Growth

The OpenStreetMap website repository thrives on a diverse range of contributions, showcasing a vibrant and healthy open-source ecosystem.

Feature Development (50%): Innovation is at the forefront, with new features driving user engagement and functionality. For instance, PR #5056 added an "og:description" meta tag to diary entries, improving social media sharing and enhancing the user experience.

Bug Fixes (30%): Stability is key, with bug fixes ensuring the platform remains reliable. Notably, PR #5016 resolved a critical issue with user permissions by enabling editing of revoked blocks, improving system integrity.

Documentation (10%): Clear and accessible documentation is vital for the community. PR #5031 updated the "GPX Import email in text format," making it easier for new contributors to onboard and stay informed.

Testing and Quality Assurance (10%): Testing contributions are crucial for maintaining code quality. By focusing on tests, the project ensures that new changes don't introduce regressions, keeping the codebase robust and dependable

Positive Impact on the Project

The deployment frequency and streamlined workflows in the OpenStreetMap repository deliver substantial benefits to both the project and its community of contributors and users.

Accelerated Innovation: With rapid deployment cycles, new features and improvements are rolled out quickly, enhancing platform functionality and user satisfaction. This speed allows the project to stay responsive to evolving user needs and technological shifts.

Enhanced Contributor Experience: Swift integration of contributions motivates open-source developers by validating their work. The efficient review and merge processes foster a positive, collaborative environment that encourages ongoing participation from both new and experienced contributors.

Quality Assurance: Automated testing and continuous integration maintain stability, ensuring that the fast deployment pace doesn't compromise the platform's reliability. Issues are caught early in the process, reinforcing high-quality standards across releases.

Community Trust and Engagement: Regular updates build trust among users, reassuring them of the project's commitment to reliability and progress. This trust strengthens both the user base and contributor engagement, helping the project flourish.

These practices demonstrate how strong DevOps strategies can fuel innovation, improve community involvement, and elevate the overall success of an open-source project.

Strategic Recommendations for Enhanced Performance

To elevate the OpenStreetMap repository's DevOps practices from great to exceptional, here are some targeted actions:

Standardize and Expedite First Response Times

Implement SLA Policies: Set service level agreements (SLAs) for code reviews, such as committing to an initial response within 4 hours.

Automated Alert Systems: Leverage automation to notify reviewers when new pull requests (PRs) are submitted or pending beyond a certain period

Expand Reviewer Pool: Encourage more contributors to join code reviews by offering clear guidelines and training to reduce pressure on a small group.

Reduce Rework Through Enhanced Code Quality

Adopt Pre-Commit Hooks and Checks: Enforce coding standards with tools like linters or static code analyzers before PRs are submitted.

Code Review Guidelines: Create robust guidelines to set clear expectations for contributors, minimizing the need for back-and-forth revisions.

Peer Programming and Mentorship: Promote collaborative development by pairing experienced developers with newcomers, ensuring better initial code submissions.

Foster and Sustain Active Code Review Culture

Recognition Programs: Acknowledge top reviewers with leaderboards, badges, or shout-outs during community meetings to incentivize participation.

Contributor Onboarding: Streamline the process for new contributors to become reviewers, providing resources and tools to ease their transition.

Feedback Loops: Enable contributors to provide feedback on the review process, creating a culture of continuous improvement.

Leverage Analytics for Continuous Improvement

Monitor DORA Metrics Regularly: Track key metrics continuously to detect trends and pinpoint areas needing optimization.

Set Performance Targets: Establish clear goals for deployment frequency, lead time, and other metrics to align the team toward common objectives.

Share Insights with the Community: Promote transparency by sharing performance data and achievements, fostering collective ownership of the project.

By adopting these strategies, OpenStreetMap can fine-tune its DevOps performance, further reduce lead times, and streamline workflows, strengthening its status as a leading open-source initiative.

Let's Wrap This Up

The OpenStreetMap website repository is a shining example of how effective DevOps practices can drive success in an open-source environment. With its impressive deployment frequency and smooth workflows, the project consistently delivers value to a global user base while maintaining high standards of reliability.

The strong engagement from contributors and maintainers creates a thriving community that continuously pushes innovation forward.

That said, there's always room to optimize. Focusing on cutting down lead times by implementing standardized response procedures and improving code quality could make the repository's delivery pipeline even faster. By incorporating these strategies, the project can enhance performance, boost contributor satisfaction, and ensure long-term sustainability.

Is Next.js the Next Evolution or Just a Passing Trend? A Dora Metrics Case Study

Shivam Chhuneja — Sat, 21 Sep 2024 05:46:10 +0000

Next.js is a repo that is a relatively recent yet powerful JavaScript framework that's taking the modern web development scene by storm. Next.js strengthens React with its server-side rendering, static site generation, and SEO optimization.

Developers love its versatility, and the numbers speak for themselves. In a recent survey, 60% of developers said they preferred Next.js for building production-ready applications due to its ease of deployment and scalability. When comparing deployment times, Next.js consistently outperforms React in speed and efficiency, making it a go-to choice for developers aiming for a seamless build process.

This case study focuses on the OpenSource Next.js repository, particularly highlighting its exceptional deployment frequency.

As developers who've tinkered with Next.js for building dynamic e-commerce sites and quick-launching blogs, we couldn't help but wonder---how did they create and manage such a legendary repo with such efficiency? So, we dug deeper using Middleware Open-Source.

If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community or subscribe to our newsletter for exclusive case studies and more!

But, first things first. Let's understand what Dora Metrics are.

What is Dora Metrics?

Dora Metrics are key indicators that show how efficiently a project progresses from start to final production in software delivery. Feel free to read through our detailed blog on Dora Metrics and how they can help your engineering processes!

Deployment Frequency: How often code hits production.
Lead Time for Changes: How long it takes for a commit to go live.
Mean Time to Restore (MTTR): Time taken to recover from failure.
Change Failure Rate: How many of those deployments break something.

Now that we're all on the same page, let's see how Next.js measures up.

Our Key Findings

Next.js: Setting the Bar High with Exceptional Deployment Frequency

The OpenSource Next.js repository stands out for its exceptional deployment frequency, far surpassing industry standards. Over the past three months, the repo has consistently pushed a high volume of deployments, reflecting the team's efficiency and automation prowess.

In June 2024, there were 247 deployments, followed by 261 in July, and an impressive 279 in August. These numbers highlight the repository's commitment to continuous integration and rapid feature releases, making it a benchmark for deployment frequency in the open-source community.

That's nearly one deployment every 3 hours! This high-frequency deployment isn't just for show; it keeps the project evolving at an incredible speed. How do they do it? A couple of key factors stand out:

Automated CI/CD Pipeline: With automation in place, deployments are quick and smooth---almost like magic.
Frequent, Small PRs: By breaking down changes into bite-sized pull requests, they can merge and deploy faster with less risk.
Fast Issue Resolution: Bugs get squashed quickly, and failed deployments are rare.

Cycle Time leaves room for Improvement

Despite a high deployment frequency, the repository shows notable fluctuations in Cycle Time, which impacts the overall Lead Time for Changes. Cycle time includes all PRs, while Lead Time only includes PRs with available deployment data. In June 2024, the average Cycle Time was around 3.5 days, slightly increasing to 3.6 days in July, and then spiking to 5.3 days by August.

Cycle time has been on the rise, increasing from 3.5 to 5.3 days, indicating some delays in the development process. Identifying bottlenecks, whether in code reviews, testing, or manual tasks, and implementing automated testing or improved sprint planning could help reduce this.

Other Key Metrics that may use some work

First Response Time

In June 2024, the average first response time was 1.6 days (38.4 hours), indicating quick and efficient handling of issues and pull requests in the Next.js repository. By July 2024, this time increased slightly to 1.8 days (43.2 hours), hinting at a minor delay, likely due to a growing number of issues or reduced team availability.

However, in August 2024, there was a significant jump to 2.6 days (62.4 hours), suggesting noticeable delays, possibly caused by higher workloads, bottlenecks in reviewer assignments, or fewer team members due to vacations.

To improve First Response Time, several strategies can be implemented to address the recent increase in delays. Automating notifications for issues and pull requests can ensure faster assignment to reviewers, reducing wait times.

Introducing SLAs (Service Level Agreements) for response times could create accountability and encourage quicker engagement. Moreover, reviewing team allocation and workload distribution could help balance responsibilities, especially during periods of high demand or reduced team availability, such as vacations.

Encouraging cross-functional collaboration could also enable quicker responses by spreading the load across the team more evenly. These changes could effectively bring response times back down and maintain efficiency.

Lead Time for Changes

From June to August 2024, the lead time for changes in the Next.js repository gradually increased from 3.3 days to 4.1 days. In June, the relatively quick 3.3-day turnaround reflected an efficient workflow with smooth coordination. By July, this increased slightly to 3.6 days, hinting at minor inefficiencies, possibly due to longer code reviews or more complex tasks.

However, by August, the lead time reached 4.1 days, indicating more significant delays likely caused by higher workloads, bottlenecks in reviews, or reduced team capacity due to vacations. This trend suggests a need to optimize processes and prevent further slowdowns.

Reasons?

Extended review times. For instance, PR #67498
Complex tasks that involve deep scrutiny and testing. PR #67391
Fluctuations in First Response Times (from 1.6 days to 2.6 days) cause uneven start times for reviews, delaying progress.

Nature of Work

The OpenSource Next.js repo includes a variety of activities, ranging from feature upgrades and bug squashing to improving documentation and refining tests. Here's a breakdown of some key insights:

Features and Improvements: Overhauls including performance optimizations (PR #67397), new functionalities (PR #67215).
Documentation: Significant contributions to documentation ensure clarity and easier adoption (PR #67056).
Bug Fixes: Addressing critical bug fixes, highlighted by 41.5 hours (~1.7 days) of rework time (PR #67022).
Performance Optimizations: Enhancing the speed and efficiency of the framework (PR #67065).

These updates position Next.js as a leader in web development. However, many of the areas needing improvement---cycle time, first response, and lead time---can benefit from increased automation in testing, code reviews, and notifications.

To further optimize, they can track bottlenecks in their pipeline, whether these bottlenecks occur during code review, testing, or deployment, and target those areas to boost overall performance. Also, fostering continuous feedback through more frequent stand-ups or retrospectives can help identify and resolve friction points. By focusing on these strategies, you can create a more efficient and streamlined development and deployment pipeline.

How Does This Affect the Next.js Community?

For internal contributors, the rapid deployment cycle is a dream. Features and fixes roll out quickly, meaning the team gets feedback in near real-time. However, the high lead time could make some contributors feel like they're stuck in a long queue, waiting for their work to go live.

For external contributors, understanding these bottlenecks can set clearer expectations. If you're contributing to Next.js, don't be surprised if your PR takes a bit of time to merge, even though deployments happen frequently.

Key Takeaways

Invest in Automation: A solid CI/CD pipeline can keep deployments flowing like water.
Encourage Small, Frequent PRs: Less complexity means quicker reviews and faster deployments.
Address Bottlenecks in Cycle Time: If lead time is lagging behind, dig into what's causing the delays---whether it's rework or review times.

DORA Score: 8/10

Next.js has an impressive deployment frequency that's soaring, but their lead time, cycle time, and first response times could use some attention. With a few strategic tweaks, they have the potential to be strong contenders against the top performers in Google's Annual Dora report,

Is Next.js Making Other Open-Source Projects Obsolete?

Next.js has set a new benchmark in deployment efficiency, making it the one to watch in the open-source world. With its unbeatable blend of speed, quality, and constant innovation, it leaves little room for competitors to catch up. Whether you're an active contributor or a curious onlooker, there's plenty to admire---and learn---from this trailblazing repo.

As Next.js continues to evolve, it's not just keeping up with the web's demands; it's shaping them. Get ready for more groundbreaking updates as it redefines open-source excellence!

If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community. and subscribe to the newsletter for exclusive case studies and more!

Trivia

Next.js was created by Vercel (formerly Zeit) in 2016 and quickly gained popularity due to its powerful features for building server-side rendered React applications. It supports both static site generation (SSG) and server-side rendering (SSR), making it a flexible choice for developers. Major companies like Netflix, Twitch, and GitHub use Next.js to power their web apps, showcasing its reliability and scalability.

Further Resources

Is FreeCAD Nailing PRs or Just Blazing Through Merges?

Shivam Chhuneja — Fri, 20 Sep 2024 05:04:59 +0000

FreeCAD, the open-source 3D CAD modeler, has garnered quite a community with over 12,000 commits, 1,100 contributors, and more than 5,000 pull requests (PRs) to date. Known for its versatility in mechanical engineering and product design, FreeCAD is a go-to for makers and professionals alike.

But are they actually nailing their PR process, or just racing through merges? With an average PR merge time of 7.5 days, FreeCAD's workflow efficiency is up for debate. Let's break down the numbers and see what's really happening behind the scenes.

We are using Middleware Open Source to get the data and insights for these open source repos, so if you want to recreate or get data of your favorite God level repo you know where to go!

If you're curious to start a conversation with fellow engineering leaders, join The Middle Out Community! Don't forget to subscribe to our newsletter below for exclusive case studies!

Channeling Our Inner Sam Spade: Cracking the Case of FreeCAD's PR Efficiency

As we at Middleware dug into FreeCAD's Dora metrics, we couldn't resist going full "Sam Spade," curious about what made this repository tick. So, we dove in to explore the patterns, looking for what makes them tick!. What we found sheds light on some best practices that other repos can take notes from.

Background on Dora Metrics

Dora Metrics are a set of key performance indicators used to measure software development velocity and efficiency. They include:

Deployment Frequency
Lead Time for Changes
Mean Time to Restore (MTTR)
Change Failure Rate

These metrics help organizations understand their development processes and identify areas for improvement.

Key Findings

Exemplary Metric: Merge Time

One standout metric for FreeCAD is its Merge Time for PRs. Efficient merge times are a testament to streamlined code reviews and strong team coordination.

June 2024 -- Average Merge Time: 15.66 Hours

In June, FreeCAD's PR management was running relatively smoothly, with an average merge time of 15.66 hours.

July 2024 -- Average Merge Time: 20.4 Hours

July showed a bit of a slowdown, with an average merge time stretching to 20.4 hours. This change could be attributed to more complex PRs

August 2024 -- Average Merge Time: 8.04 Hours

In August, things really picked up pace, with the average merge time dropping significantly to just 8.04 hours. This lightning-fast rate suggests that the team and community were hyper-efficient in handling pull requests.

Average Merge Time: Approximately 11.02 hours.

Rapid Merges: PRs like #14693, #14691, and #14688 were merged within less than a day.

Factors Contributing to Efficiency

Ownership: Consistent authors like PaddleStroke, Roy-043, and wwmayer contribute to swift merges.
Structured PR Review: Automated checks via GH Actions, including workflows like CI_master.yml and sub_lint.yml, catch issues early.

Also read: Is freeCodeCamp Sacrificing Quality for Speed with Their Rapid Deployments?

Metric that needs Improvement: Cycle Time

While freeCAD has made commendable strides in reducing Cycle Time, with figures decreasing from 171.12 hours to 130.8 hours and then to 121.68 hours, there's still room for improvement.

Continuing this trend could further enhance their efficiency. Addressing bottlenecks and optimizing workflows could help freeCAD push these numbers even lower, achieving even faster turnaround times and reinforcing their reputation for swift, effective development.

Average Cycle Time: Approximately 21.42 hours.
Fluctuation: Inconsistencies arise, as seen in longer cycle times for complex PRs like #14674.

Factors Contributing to Delays

Complex PRs: More intricate PRs understandably take longer.
Reviewer Availability: Delays due to the availability and assignment of reviewers.

Other Key Metrics

Lead Time

The reduction in Lead Time from 2.51 days in June to 1.81 days in August is a clear indicator of enhanced efficiency in moving code through the development pipeline. This is a commendable achievement, suggesting that freeCAD has improved its processes for deploying code, leading to faster delivery of new features and fixes. This efficiency boost likely contributes positively to the overall project velocity and responsiveness.

First Commit to Open

The increase in the time taken from First Commit to Opening a Pull Request, from 3.77 hours in June to 12.01 hours in August, is a bit concerning. This rise might indicate bottlenecks or delays in the initial stages of the contribution process. Addressing this could involve streamlining the process for handling code submissions, possibly by improving workflows or increasing the availability of reviewers to handle initial requests more promptly.

First Response Time

Although there's a slight improvement from 11.42 hours in June to 10.15 hours in August, there's still potential for faster response times. This metric is crucial for maintaining an active and engaged contributor base, as prompt responses can enhance collaboration and prevent contributors from losing interest or moving on to other projects. Focusing on reducing this time further could improve contributor satisfaction and keep the development process moving smoothly.

Nature of Work: Feature-Focused Development

FreeCAD is rich in diverse contributions, spanning feature development, bug fixes, and optimization work.

Feature Development

Major focus, e.g., Assembly: TNP by PaddleStroke.

Bug Fixes

Substantial effort in maintaining and fixing issues, e.g., Fem: Fix height of reference list widget in spring constraint task panel by marioalexis84.

Optimization and Refactor

Significant, seen in PRs like Guard all uses of basic_random_generator for thread safety by bgbsww.

Each of these contributions highlights the diverse nature of work being done on FreeCAD---from feature development that solves long-standing issues to bug fixes that refine the user experience, to optimizations that ensure the software can handle future demands.

Impact on Project and Community

Efficient PR management leads to better code quality, more frequent releases, and higher contributor satisfaction. FreeCAD's emphasis on swift PR reviews encourages further contributions.

Takeaways

Structured Workflows: Implement CI/CD pipelines to automate checks, as FreeCAD's CI_master.yml ensures consistency.
Active PR Review Culture: Prioritize swift reviews to avoid PR pile-ups.
Clear Ownership Assignments: Assign code owners to specific modules for better PR management.
Balanced Work Distribution: Encourage a mix of new features, bug fixes, and refactoring for a well-maintained codebase.

DORA Score: 8/10

After taking a closer look at FreeCAD with our Dora Metrics toolkit, here's the rundown: FreeCAD scores a respectable 8/10. Its PR merge times are impressively quick, thanks to a committed community that's always in motion. But---and there's always a "but"---Cycle Time is where FreeCAD hits a few speed bumps, with complex PRs dragging down the average.

The takeaway? FreeCAD's team is doing a solid job, but a little change here and there could take them from great to exceptional. Our analysis, stacked against Google's Dora benchmarks, highlights FreeCAD's wins and its opportunities for improvement. Want to see how your project compares or optimize for efficiency? Analyze it using Middleware OSS.

Conclusion - FreeCAD: A PR Powerhouse or Just Polished?

FreeCAD's got some slick moves when it comes to handling PRs---those rapid merge times are nothing short of impressive. But like a dance partner with two left feet, its Cycle Time could use a little more rhythm. With just a bit of fine-tuning, FreeCAD could be on its way to even greater efficiency.

If you're curious to pick the brains of fellow engineering leaders, join The Middle Out Community!

Trivia

FreeCAD was started in 2002 by Jürgen Riegel, Werner Mayer, and Yorik van Havre, making it one of the oldest free and open-source 3D CAD modeling tools

Further Resources

OpenVINO's AI Success: Brilliance or Cracks Beneath the Surface?

Shivam Chhuneja — Thu, 19 Sep 2024 04:37:38 +0000

Ever wonder how some AI projects manage to deliver breakthroughs in record time? OpenVINO, an open-source toolkit designed for optimizing and deploying AI models, is one such standout. With C++ at its core, this repository has consistently delivered on its promises. But what really sets it apart?

It's their ability to fine-tune and optimize Cycle Time---a critical metric that tracks how fast development work goes from inception to deployment. This case study takes a closer look at how OpenVINO manages to maintain an exceptional Cycle Time---showing us why this toolkit is making waves in the world of AI.

If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community. and subscribe to the newsletter for exclusive case studies and more!

What We Discovered When We Peeked Into OpenVINO's Code Playground

Of course we love looking under the hood of how great teams and repositories function so we had to dive into OpenVINO as well leveraging Middleware OSS.

What made this repository tick?

Why was it performing so well in some areas, and what could be improved?

Using Dora metrics as our analytical framework, we dug into the data, analyzing everything from deployment frequencies to response times. What we found might just surprise you---and hopefully, inspire a few new ideas for your own processes.

Background on Dora Metrics

Before we get into the juicy details, let's take a moment to talk about Dora metrics. These are the gold standard for evaluating the performance and efficiency of software development and delivery pipelines. In simple terms, DORA metrics cover four key areas that provide a comprehensive picture of how well your dev processes are running:

Lead Time for Changes: How long does it take for a code change to go from development to production?
Deployment Frequency: How often are you pushing changes to production?
Mean Time to Restore (MTTR): How quickly can you recover from a failure?
Change Failure Rate: What percentage of your changes fail in production?

These metrics are crucial because they help assess the efficiency, speed, and reliability of your development pipeline.

For OpenVINO, we placed a special focus on Cycle Time, a close cousin to Lead Time, to see how efficiently they're moving code from start to finish.

Key Findings

Exceptional Cycle Time and Its Contributors

Cycle Time represents the time taken from when work begins until it's completed and ready for deployment. For OpenVINO, the Cycle Time over the last three months has been impressive. In June 2024, the First Response Time was 50 hours, dropping to 46 hours in July before rising to 68 hours in August.

These numbers highlight a consistent effort to keep development moving smoothly. But what exactly contributes to this exceptional Cycle Time?

Efficient Review Process: PRs like PR #25069 and PR #24953 were merged swiftly, with detailed and comprehensive reviews ensuring that quality wasn't sacrificed for speed.
Active Contributor Base: Regular contributions from a core group of contributors, like Ilya-lavreno and Tsavina have helped keep things on track. Active participation from dedicated contributors can significantly reduce bottlenecks in the pipeline.
Automation and Continuous Integration (CI): OpenVINO's robust CI setup plays a huge role here. They employ workflows like check_pr_commits.yml and build_doc.yml that streamline validation, testing, and integration. Automating repetitive tasks is a game-changer for any repository aiming to optimize its Cycle Time.

Also Read: JuliaLang: Performance Prowess or Just Smoke and Mirrors? Unveiling the Real Story

First Response Time: A Scope for Improvement

While Cycle Time has been a strong point, First Response Time---the time taken for maintainers to initially respond to new PRs---has room for improvement. Here's what the numbers say:

Although July showed a slight improvement, the overall response times remain high, with notable fluctuation. So, what's causing these delays?

Dependency on Key Contributors: Delays occur when critical reviewers are unavailable, which is common in any open-source project. The reliance on a small group of key contributors can slow things down.
Complexity of PRs: Some PRs---particularly those involving significant architectural changes or feature introductions---naturally require more time for initial assessment.

Nature of Work and Contributions

OpenVINO's recent work shows a diverse mix of feature development, documentation updates, and bug fixes. Here's a breakdown:

Some of the top features worked on recently include:

Implement Support for LLMs (PR #24930): Expanding the toolkit's capabilities with large language models.
Enable runtime buffer fusing for crop on GPU (PR #25060): Performance optimization for GPU-based workflows.
Activate SDPA Fusion on ARM (PR #24919): A critical feature for ARM architectures.

Impact on Project and Community

Internal Development Practices: The focus on efficient Cycle Times boosts overall productivity and ensures that development timelines are consistently met. This translates into more predictable releases and happier contributors.

External Contributors: Prospective contributors are more likely to engage with a repository that is both responsive and efficient. A streamlined development process not only enhances collaboration but also drives innovation by attracting new contributors who appreciate fast feedback cycles.

Takeaways

Effective Cycle Time Management

Strategy: Breaking down larger PRs into smaller, manageable pieces helps reduce overall Cycle Time and makes the review process easier.

Example: PR #25023 demonstrated effective segmentation, allowing for quicker reviews and fewer delays in merging.

Enhancing First Response Times

Strategy: Implementing a rotating schedule for first-level reviews can evenly distribute the workload among maintainers, reducing wait times.

Example: Introducing more automated triage processes could maintain response efficiency during high-activity periods, preventing backlogs.

Importance of Documentation and CI Pipelines

Strategy: Maintaining thorough documentation and robust CI pipelines helps streamline the review process and minimizes rework.

Example: Workflows like build_doc.yml have been key to keeping documentation up to date and comprehensive, which speeds up the development process.

Dora Score - 8/10

After diving into the Dora metrics for OpenVINO, we're giving it a solid 8/10. The project excels in Cycle Time and maintains a consistent deployment frequency, making it a standout in these areas. However, variations in First Response Time are holding it back from a perfect score.

When compared to Google's Annual Dora Report, OpenVINO holds up impressively, especially in terms of speed and efficiency. With some tweaks to improve those response times, it could easily leap into the elite tier.

Curious to see how your project stacks up? Middleware's OSS tools can help you dive into your metrics, find areas for improvement, and fast-track your project's success!

Also Check: The Dora Rating for another God-level Repo - React Native

Conclusion: Is OpenVINO's AI Speed Hiding a Need for More Precision?

OpenVINO knocks it out of the park with its impressive Cycle Time management, making its development process highly efficient and consistent. However, there's a bit of a snag---improving First Response Time could unlock even greater potential, speeding things up and driving more engagement from contributors. For teams looking to fine-tune their processes, OpenVINO's journey offers a great lesson in striking that perfect balance between speed and quality. If you are looking to bring more predictability & visibility to your engineering pipeline, then you should check out Middleware Open Source.

If you're excited to explore these insights further and connect with fellow engineering leaders, come join us in The Middle Out Community. Feel free to subscribe to the newsletter as well if you'd like some exclusive case studies too.

Let's Wrap This Up

OpenVINO was developed by Intel to enhance AI applications, particularly in edge computing, bringing AI processing closer to where the data is generated---think smart cameras, drones, and IoT devices.