Forem: Wakatepe-mergify

Managing Monorepos with a Merge Queue: a Game Changer

Wakatepe-mergify — Thu, 02 Nov 2023 15:28:42 +0000

Handling large codebases is a tough job, and it's even harder when dealing with monorepos. A monorepo is a single code repository that contains multiple projects or modules. It's a handy way to share code across teams and manage dependencies.

But dealing with monorepos can be tricky. You have to make sure changes in one module don't mess up things in another. This is where a merge queue steps up.

Monorepos: Why Are They Challenging?

Monorepos have several benefits, such as easier dependency management, better code sharing, and simpler code refactoring. However, managing a monorepo can be challenging, especially as the codebase grows. Some of the main challenges of managing monorepos include the following:

1️⃣ Difficulties with Dependency Management

With so much code in one repository, it can be challenging to ensure that dependencies are correctly managed across all projects and services.

2️⃣ Slow Build and Test Times

Building and testing the entire monorepo can take a lot of time, especially as it grows.

3️⃣ Managing Conflicts and Overlapping Changes

With many developers working on the same codebase, it can be challenging to manage conflicts and ensure that changes made in one module do not conflict with changes made in another.

Why Managing Monorepos with a Merge Queue

To help overcome these challenges, betting on a merge queue sounds like a great idea.

Here are some of the benefits of using a merge queue for managing monorepos:

1️⃣ Reduced Risk of Conflicts and Overlapping Changes

A merge queue ensures that changes made in one module do not conflict with changes made in another by managing the merge order and dependencies.

2️⃣ Increased Efficiency with Automatic Dependency Management

A merge queue automatically manages dependencies, making it easier for developers to work on their individual projects and modules.

3️⃣ Faster build and test times

By managing the merge order, a merge queue reduces build and test times by ensuring that changes that depend on each other are merged together.

Using a merge queue for managing monorepos is a real game changer. Developers can set up a merge queue for their monorepo and customize it to fit their specific needs. For example, they can set up different merge queues for different projects or modules and specify the order in which changes should be merged.

Mergify for Monorepo: How Does it Work?

When you submit a pull request to a monorepo, it can be put in one or multiple queues at the same time. With its feature, Partition Rules, Mergify will not merge the pull request until it has been validated by each queue in which it is. This collaboration ensures that the pull request meets all the requirements of each queue.

If the pull request is in multiple queues and fails to meet the requirements of at least one queue, it cannot be merged. To merge, the pull request must meet all the criteria of each queue it is in.

You can also use queue rules to handle different projects in the monorepo that have different criteria for merging pull requests. To do this, you can replicate the queue rules logic for each project and use the attribute queue-partition-name.

A partition rule has the following parameters:

Handle Different Projects in a Monorepo Using Partition Rules

If a pull request contains a modification on any file in the folder projectA/ it will be added in the partition projectA, if it contains a modification on any file in the folder projectB/ then it will be added in the partition projectB, if it contains a modification in both folder projectA/ and projectB/, then it will be added to both partitions.

If none of the two partitions' rules matches, then the PR will be added in both partitions. The queue_rules will still determine in which queue in the partition(s) the pull request will be added in.

Here is a table representing the partition and queues with the code below:

Project A	Project B
Hotfix	Hotfix
default	default

shared:
    priority_rules: &priority_rules
        - name: hotfix PR detected
          conditions:
            - label=hotfix
          priority: high
        - name: lowprio PR detected
          conditions:
            - author=dependabot[bot]
          priority: low

pull_request_rules:
   - name: queue PR with queue label
     conditions:
       - label=queue
     actions:
       queue:

partition_rules:
  - name: projectA
    conditions:
      - files~=^projectA/

  - name: projectB
    conditions:
      - files~=^projectB/

queue_rules:
  - name: hotfix
    priority_rules: *priority_rules
    routing_conditions:
      - label=hotfix
    merge_conditions:
      - or:
        - and:
          - queue-partition-name=projectA
          - check-success=ciA
        - and:
          - queue-partition-name=projectB
          - check-success=ciB

  - name: default
    priority_rules: *priority_rules
    merge_conditions:
      - or:
        - and:
          - queue-partition-name=projectA
          - check-success=ciA
        - and:
          - queue-partition-name=projectB
          - check-success=ciB

Conclusion

Managing monorepos can be challenging, but with a merge queue, and especially Mergify's one, developers can work more efficiently and with more confidence.

A merge queue allows developers to manage their monorepo as a single codebase while still being able to work on individual projects and modules. This improves collaboration and communication between teams, reduces the risk of conflicts and errors, and speeds up the development process. If you're struggling with managing a monorepo, consider giving Mergify's new feature a try.

How to Get Rid of Flaky Tests? Best Practices

Wakatepe-mergify — Thu, 26 Oct 2023 14:16:39 +0000

Flaky tests are a thorn in the side of most software development teams.
Even more so when they have set up an automated CI/CD workflow.

In addition to the security holes they create, flaky tests also lead to the need for manual intervention.

Getting rid of them and cleaning up your test suite should therefore be a priority. Luckily, this article focuses on the different ways of eradicating flakiness in your tests.

Understanding Flaky Tests

Flaky tests are like the chameleons of the testing world, as they change their behavior depending on the context. A test that intermittently fails and passes poses a challenge for developers, who often spend countless hours chasing down false negatives.

These tests lack reliability and consistency, making it difficult to determine the true quality of the software being tested. The variability in test results can stem from a variety of sources, including environmental factors, race conditions, non-deterministic code, or external dependencies.

If you want to know more, take a look at Fabien's article explaining what is a flaky test and how to classify them.

Flaky Tests Remediation: Best Practices

Addressing flaky tests requires a proactive approach and continuous effort.

This subject has already been covered by Fabien in his article Flaky Tests: How to Fix Them? If you haven't read it, you should.

In it, he shares with you the best practices to adopt to get rid of your flaky tests:

Split your test suites.
Execute tests on VMs or controlled environments.
Log and document.
Separate and quarantine.
Retry your tests (the good way).
Monitor and alert.
Develop processes of investigations.
Run tests multiple times before merge.
Write and apply good practices.
Without getting too repetitive, here are other best practices you should also keep in mind.

🐞 Debugging and Root Cause Analysis

Invest time in understanding and diagnosing the causes behind flaky tests. Analyze test logs, gather relevant data, and identify patterns or dependencies that may contribute to their instability.

🌍 Test Environment Stability

Ensure that the test environment is consistent and stable. Eliminate external factors that can introduce variability and cause intermittent failures, such as network issues, timing dependencies, or resource constraints.

🎯 Test Isolation and Determinism

Design tests to be independent and deterministic, ensuring they don't rely on external resources or produce different outcomes with the same inputs. Utilize techniques like mocking or stubbing to create controlled and predictable testing environments.

⚙️ Test Retry Mechanisms

Implement retry mechanisms for flaky tests to reduce false positives. By re-running tests that fail intermittently, you can increase the likelihood of obtaining consistent and reliable results.

📊 Test Monitoring and Reporting

Establish a robust test monitoring and reporting system to identify flaky tests in real time. Regularly review test results, track their stability over time, and prioritize their resolution as part of ongoing test maintenance efforts.

You can also take advantage of various tools, such as those from CircleCI, Jetbrains, or CI Monitoring solution from Mergify.

Conclusion

Flaky tests are a real pain in the @$? and they pose a significant challenge in the realm of CI/CD.

Their inconsistent behavior undermines the reliability and efficiency of software engineering teams.

However, by adopting the good practices we've just presented in this article and "Flaky Tests: How to Fix Them?", you'll limit the nuisance.

Putting them into practice quickly and regularly will save you a lot of trouble. Problems that are trivial today, but could become major threats tomorrow.

Why Should You Get Rid Of Flaky Tests? Unmasking the Hidden Nuisance in your CI/CD Pipeline

Wakatepe-mergify — Thu, 26 Oct 2023 13:58:42 +0000

Amidst the quest for automation and speed, a common obstacle emerges: flaky tests. These seemingly innocent but troublesome tests can undermine the entire CI/CD optimization process.

To understand the nuisance of flaky test, it's like having a roommate you can more or less count on.

When you ask him for a hand, you never know how he'll handle your request.

He may or may not pay the rent, and the same goes for chores. When you get home from work, your last yoghurt may be waiting for you, or it may be digesting in his belly.

That's what we call an unreliable roommate. Despite identical rules, conditions and environment, he can make random decisions.

In fact, a flaky test is just the same. Under identical conditions, a flaky test will either pass or fail. A sort of Russian roulette.

In this article, we will dive into the concept of flaky tests, understand their implications, and explore why they are a nuisance.

What Are Flaky Tests?

Flaky tests, also known as unreliable or intermittent tests, are automated tests that exhibit inconsistent behavior.

These tests are characterized by their ability to produce both passing and failing results for the same codebase without any modifications. In simpler terms, running a flaky test multiple times can yield different outcomes, leading to inconsistent feedback regarding the stability and correctness of the system.

If you want to know more about what are flaky tests and their classification, you should read Fabien's article about flaky tests definition and classification.

The Nuisance of Flaky Tests

Flaky tests can be detrimental to CI/CD optimization in several ways:

✅ False Positives

Flaky tests can generate false positives, reporting failures when no actual defects exist. This can create unnecessary delays and frustration for developers, who have to investigate and fix non-existent issues, leading to wasted time and effort.

❌ False Negatives

Conversely, flaky tests can also produce false negatives, where actual defects go undetected. When tests sporadically pass, potential issues can slip through the cracks, jeopardizing the quality of the software being delivered.

🤨 Unreliable Feedback

The primary purpose of automated tests is to provide reliable feedback on the stability and quality of the codebase. Flaky tests undermine this objective, making it difficult to distinguish between genuine failures and transient issues, eroding developers' trust in the test suite.

📉 Decreased Productivity

Flaky tests hinder the efficiency of the development process. Developers spend precious time investigating and re-running tests to identify and reproduce failures, detracting them from focusing on crucial tasks like implementing new features or fixing genuine bugs.

🤔 Impaired Confidence

Flaky tests erode the confidence of both developers and stakeholders. Developers become skeptical of the test results, and stakeholders may lose faith in the reliability of the software being produced. This lack of confidence can have far-reaching consequences, affecting team morale and the overall success of the project.

How Can I Get Rid of Them?

Flaky tests are a real nuisance, and their impact is often underestimated. This problem, if not properly dealt with, will be dragged along like a ball and chain by the developers on your team. And the consequences can be dire:

Velocity at half-mast,
Vulnerable code base,
Loss of trust and consideration among developers...

Suffice it to say, it doesn't look good! Faced with this situation, it's time to adopt a proactive stance and look for concrete solutions. Fabien has already dealt with this subject in his article Flaky Tests: How to Fix Them?

You can also take advantage of various tools, such as those from CircleCI, Jetbrains, or CI Monitoring solution from Mergify.

The Developer Experience, from your Own Team to your End-Users

Wakatepe-mergify — Thu, 21 Sep 2023 15:34:09 +0000

Developer experience, or DX, refers to all a developer's perceptions, feelings, and experiences in interacting with a tech product or software. In this instance, 'interaction' can be two-fold.

A developer could use your product, such as a software development kit or an API, to create a product for the end user. Two, the developer could be creating a product for you, such as an app, working as your employee.

Since the term 'user experience' is more commonly known than DX, it helps to compare the two. Do they both tie in? A DX vs UX comparison will show that user experience concerns the end users, while developer experience concerns the developers who create the product for end users.

Both have one main thing in common; the aim to create a good product. That makes DX important for both the external users and the internal teams. For the former, developer satisfaction means better products, increased user satisfaction, and higher user retention. For the latter, it results in faster development and higher productivity.

So how do you go beyond the DX vs UX differences and balance both in your organization? Let's find out.

Developer Experience for Your Team

Internal developer experience comprises the tools, processes, and culture you create for your development team. Simply put, it's the experience developers have while working on an internal project.

For instance, a team of developers may be creating a new app for your product. Many factors, such as the workflows, level of support, tools, technology, mentorship, collaboration, etc., will impact their experience.

Benefits of Prioritizing Internal DX

When you prioritize internal DX, you improve employee retention and increase the number of success stories from your organization. Here are some other benefits:

Increased Productivity: When developers can access effective training, streamlined workflows, and modern tools, they work better. Higher productivity results in faster project delivery and better efficiency.
Better Innovation: With proper support, responsiveness, and mentorship, there's a greater chance of creativity and innovation. An organization that focuses on internal DX can create an environment that fosters risk-taking and experimentation.
High-Quality Code: The prioritization of internal DX also results in better code quality. Developers can leverage feedback mechanisms to write scalable, maintainable, and high-quality code.

Tips to Improve Internal DX

McKinsey recommends focusing on the following to improve developer velocity: ''tools, culture, product management, and talent management''. Organizations can heed this advice by:

Offering growth and learning opportunities to developers to help them stay up-to-date with new technologies
Providing user-friendly and modern technologies for the development process
Creating streamlined workflows, such as code testing, review, and deployment, to lower friction
Creating a collaborative culture where developers can work with their colleagues for better outputs
Listening to developer feedback and using it to make improvements in the development environment

Developer Experience for Your End-Users

External DX is the experience developers have when they use a company's products or services to build their software. It comprises the following:

API usability and ease of integration
Quality of support and documentation
Tools, libraries, and frameworks available for development

A comparison of StackOverflow Developer Surveys over the years shows that the time developers spend per project has been decreasing. That's only possible with a decent external DX.

Importance of End-Users DX

It's imperative to provide a good external DX for users since it results in increased use and adoption of a company's software development products. A higher adoption helps drive revenue generation and business growth.

It also helps users by allowing them to create robust applications and enjoy better integrations. Developers who use these products can also create more diverse applications in shorter timeframes.

Best Practices for Improving External DX

A reference book on the subject: the Devrel Book. It presents a mapping of the developer's journey and the external developer experience. This book gives you the contact points you need to optimize.

Some good practices for improving external DX include:

Publishing clear and understandable landing pages and product pages,
Creating FAQ page,
Providing clear documentation,
Helping developers to get their first win with Quick Start Guides that include explanations of endpoints, examples, and code snippets,
Offering libraries and sample codes to give developers a starting point for the development,
Making it easy to sign up or register,
Offering a sandbox to "play" with the tool, without any risk,
Communicating about news, features and changes through a changelog,
Introducing all the integrations and extensions.

Balancing Internal and External DX

If a company has poor internal DX, it can lead to errors and delays in API improvement. That would further reduce experience quality for external developers using the company's product.

Likewise, external DX can also impact internal DX. Suppose external developers run into various issues with the SDK (software development kit) or API.

They would need increased support from the company's internal developers. It could divert resources away from a company's internal development processes.

A balance between the two is necessary to keep developers satisfied on both ends. Google is a good example of such a company. On one end, Google offers SDKs and APIs for Google Cloud Platform, Google Maps, and Google Drive. These products allow developers to integrate Google's tech into their products.

Internally, Google builds software like Google Chrome and Google Ads. Google's internal developers often use the same tech and tools the company offers to external developers.

In doing so, Google strikes a balance as the collective use of resources requires consistency and reliability to keep both internal and external projects running smoothly.

Other organizations can follow suit by:

Gathering insights and feedback from internal and external developers to learn about improvement points
Aligning internal and external goals to create synergy between the two
Ensuring consistency across products when creating APIs, SDKs, and libraries
Encouraging user advocacy for internal and external projects.

Improve DX With Automation

Developer experience is an important component of measuring success today. While improving the developer experience can take time, one step can make some quick improvements. That is automation.

With a tool like Mergify, workflow automation becomes a breeze. Mergify optimizes the CI/CD pipeline by queuing, updating, and merging your pull requests automatically. It saves time and money by eliminating outdated PRs and manual rebase, allowing faster project delivery and better code security.

Most importantly, Mergify can improve the developer experience by providing reliable workflows with minimal friction.

Testing your Pull Requests Smartly with a Merge Queue

Wakatepe-mergify — Thu, 21 Sep 2023 15:20:20 +0000

Before we start, let's agree: all your pull requests must be tested. If you don't agree with this first point, reading this article may seem like a waste of your time.

Unless you're ready to change your mind?

While it's important to test all the changes you wish to integrate into your code base, it's important to test them intelligently, and at the right time.

Good news: you can do this with a merge queue!

Intelligent PR testing: CI savings and exacerbated velocity

To test your pull requests as efficiently as possible, you should do it in two stages:

tests to validate your PR and get it into the queue,
tests to merge your pull request into main.

Split your Test Suite

First, divide your test suite into two distinct categories, differentiating between unit tests and more global tests such as functional or end-to-end tests.

Queue Entry Conditions and Unit Tests

For a pull request to be included in the merge queue, it must be valid and of good quality. After all, why attempt to merge poor-quality changes?

With this in mind, you can define entry conditions. In general, we recommend choosing the requirements:

More than 1 approval,
Unit Test CI.

If these checks pass, then your PR is validated and can enter the queue. Here's what your YAML file might look like.

queue_rules:
  - name: default
    queue_conditions:
      - check-success = mycijob
      - "#approved-reviews-by >= 1"

Here, the pull request enters the queue called default if the CI named mycijob has passed. Moreover, to enter the queue, the PR needs to be approved by at least one contributor.

Pull Requests Rules and Conditions to Merge

Your PR is now in the merge queue. It will make its way to the top of the queue. This moment corresponds to the merge momentum. The PR has just been updated; it's ready to be tested and merged.

So, it's at this precise moment that you should launch all your integration tests. Not before (and certainly not after!)

You can define new merge conditions for your pull request, in other words, new checks, corresponding to the CI job of your functional and end-to-end tests.

Here's how it might look.

queue_rules:
  - name: default
    queue_conditions:
      - check-success = mycijob
      - "#approved-reviews-by >= 1"
    merge_rules:
      - check-success = mycijob-extra

The merge_rules define the condition required to merge a pull request once it is in the merge queue. Here, the PR will be merged if the CI called "mycijob-extra" has passed successfully.

Note: Mergify injects all branch protection settings defined by GitHub, so there's no need to repeat them.

Why Should You Test Your PR this Way?

Following this strategy, each pull request is fully tested, but above all intelligently, by launching the right type of test at the right time to match your standards and workflows.

But the coolest thing about all this is the benefits you'll reap. Using this testing strategy for your pull requests ensures that:

Limit your CI costs, since you only run your tests once, and when you need to,
Speed up your project and improve your velocity. Launch large tests only when the PR is ready to be merged, and avoid relaunching your tests because of out-of-date PRs,
Secure your code base: only up-to-date and thoroughly tested PRs can be integrated.

So, are you convinced? If not, we invite you to try Mergify and its merge queue.

Updating, Testing, and Merging Several Pull Requests at Once with a Merge Queue

Wakatepe-mergify — Mon, 21 Aug 2023 14:52:11 +0000

The rapid expansion of a project can lead to an exponential increase in the number of pull requests requiring processing. This surge in pull requests brings an inevitable, heightened demand for Continuous Integration (CI), leading to a consequent surge in both the time needed for CI and, subsequently, the associated costs. So how can this be mitigated?

Implementing a merge queue offers a feasible solution, as it can effectively manage and reduce the number of CI jobs initiated per pull request. This is a natural and direct advantage of a merge queue. But to enhance efficiency even further, you might consider formulating a smart testing strategy and utilizing a distinct feature known as batches.

Avoid Rerunning Tests

In the context of a large-scale project, a growing number of pull requests can have an exponential effect on your CI times. This is especially true if you refuse to merge out-of-date pull requests — and you should.

Let's take the example of a project with 20 open pull requests.

Without a merge queue, after the first pull request has been merged, the other 19 would have to be rebased as they're out-of-date, and 19 CI jobs would have to be restarted.

After the second merged PR, 18 PRs would have to be rebased, and 18 CI jobs would restart. And so on.

With a merge queue, if you have 20 open pull requests, you'll only have to rebase 20 and schedule 20 CI jobs.

A merge queue solves this issue by processing your pull requests sequentially and automatically, one by one. After automatically rebasing it, a merge queue triggers a CI job to run on that pull request. The same principle applies to the second pull request and then to all the others.

To sum up, without a merge queue, the number of CI jobs could be equal to:
20+19+18+17+...+1=210 CI jobs! As opposed to just 20 jobs in all with a merge queue.

Smart Test Strategy: Reduce CI Time, Increase Velocity

By using a merge queue, you can go even further to optimize your CI times and improve the velocity of your project.

Let's say some of your tests are very long and therefore require a lot of CI time.

If you launch them as soon as you create your PR, you need to be sure to merge them right away if they pass.

However, without automation, you are likely to have to rebase this PR and therefore have to run your tests several times. Each iteration represents time and money.

With a merge queue, it is possible to split your tests to optimize your workflow.

When a pull request is created, it only runs the unit tests. If they pass, the pull request will be included in the merge queue. Once it's at the top of the queue, it's time to go: just after being rebased, this is the best time to launch your longest and most expensive tests (functional tests, end-to-end tests...).

If those tests pass, the pull request will be merged, and you only had to run your tests once, at the right time.

Going Further: Introducing Batches Feature

Now that you know a little more about the merge queue and how it can help you reduce your CI time, it's time to look at a real game changer.

We're talking about Mergify's Merge Queue's exclusive feature: batches.

With batches, you can check the mergeability of multiple pull requests at once. To do so, you have to use the batch_size option.

If you set it to 5, the merge queue will create a rebased draft pull request and merge the commits of the five next queued pull requests.

Then, to test all the commits from the five pull requests, Mergify will only run one single CI job instead of 5!

If the CI passes: Mergify merges the five queued pull requests and closes the draft one.

If the CI reports a failure: Using a binary search, Mergify builds two smaller batches of pull requests to check in order to find the culprit; for example, a batch of 3 and a batch of 2. Once the guilty pull request is spotted, Mergify kicks it out of the queue, and the rest of the pull requests are merged.

Look at the demo below to see what it looks like when used.

Conclusion

In conclusion, the exponential increase in pull requests and consequent demand for Continuous Integration (CI) in a rapidly expanding project can present significant challenges, most notably increased CI times and associated costs. However, tools and strategies like the implementation of a merge queue, smart testing, and batch processing can effectively mitigate these challenges.

A merge queue helps process pull requests sequentially, reducing the need for rebasing and the restart of CI jobs. This makes your CI process far more efficient and cost-effective. Additionally, a smart test strategy, which includes running unit tests immediately after creating a pull request and running more extensive tests only after a pull request reaches the top of the queue, can significantly reduce CI times and increase project velocity.

The batch feature in Mergify's Merge Queue offers a game-changing advantage by checking the mergeability of multiple pull requests at once, reducing the number of required CI jobs even further. It smartly handles any CI failures by identifying and eliminating the problem pull request while seamlessly merging the others.

With these features and strategies, the challenges of a growing project can be effectively managed, keeping your CI times and costs in check and ensuring the project's smooth and efficient progress.

What's a Merge Queue and Why Use it?

Wakatepe-mergify — Thu, 17 Aug 2023 09:41:22 +0000

That's a good question, isn’t it? Indeed, if it was only a little-known term a few months ago, merge queues would be more and more talked about. Between announcements from leaders like GitHub and real technical solutions, merge queues are increasingly adopted by software development teams.

So, you'll be able to dive into this topic to understand what a merge queue is, in what situations they are useful, and how they work in practice.

Are you ready for it? Let’s get started — you can also watch our video at the end of this article.

What Do We Mean by “Merge Queue”?

Before wanting to know why you should work with a merge queue it is important to define what it is.

Its name says all: a Merge Queue is simply a queue of Pull Requests waiting to be merged.

Each member of your team creates Pull Requests, like a lot each day. Then the maintainers of the repository add it to the queue. Simple right?

To be accurate, you are not just putting basic PRs in a queue. All the PRs present in the queue have been approved by maintainers. It means that they passed all the required checks.

So you get a queue full of validated Pull Requests. Sounds cool, not really useful though. Why not just merge them, one by one? To give you an answer, let see first what common issues you will encounter if you do not use a merge queue.

Why Do You Need a Merge Queue?

To be honest, there are plenty of reasons and arguments to explain why you should bet on a merge queue. In this part, you will discover a really problematic situation and how you can solve it by using a merge queue.

The Common Issue: Merging Outdated Pull Requests

To understand the problem queues resolve, you first have to understand the issue. To do so, imagine the following situation:

The main branch passes its continuous integration testing correctly.
A pull request is created, which also passes the CI. Call it PR1.

At this time, here is how you can represent the state of your repository:

Right now everything is totally OK, but it will not last. Let’s see.

While PR1 is still open, another commit is pushed to the main branch. No matter if this newly created commit is pushed directly to the main branch or merged from another pull request. In fact, you just have to understand that the main branch has been modified.

Then the tests are run against the main branch by the CI, and they pass again. At this point, you can describe the state of your repository and its continuous integration system like this:

You can see the PR1 is still marked as valid by the continuous integration system. Indeed, it is normal since it did not change. Only the main branch changed.

As there is no code conflict, GitHub considers PR1 as mergeable: the merge button turns green.

All happy you are, you smash that green button.

But you saw that coming, you might have a surprise — and not a good one.

Now that you want to merge PR1, a new merge commit is created and the continuous integration tests fail, but why?

In fact, when PR1 was marked as valid, the CI did not test — again — PR1 with the new commit added to the main branch (remember it?).

However, the last commit brought new tests in the main branch with him and now PR1 does not have the correct code to pass this new test. As sad as logical.

How to Deal with this Situation?

Basically, the problem is all about rebase and the necessity for each pull request to be up-to-date. If you do not use a Merge Queue, you have two options:

Only run your CI on the head of your feature branch — and do not require your feature branch to be up-to-date with the main branch.

→ The main drawback here is that your feature branch might be or not compatible with the main branch.

Require all feature branches to be up-to-date with the target.

→ The main drawback is that you waste a lot of time and money.

That’s a quite common situation for organizations and teams working with CI/CD processes. If it is an issue, do not worry, you have found the real solution!

The Real Solution: a Merge Queue

A promise is a promise, you’re not going to stay in this dead-end situation.

Your solution: using a merge queue. Indeed, it solves that problem by updating any pull request before being merged. But only the ones that are not up-to-date with its main branch. Actually, a merge queue forces the CI system to test the PR again but with the new code from its main branch.

In the previous situation, if you use a merge queue, it would automatically merge the main in the feature branch.

As you can see below, the CI would have rerun. Then the pull request would have been marked as failing, and removed from the queue. Of course, if the PR was valid, and all the checks passed, it would have been merged.

Another case of scenario: multiple pull requests are validated and ready to be merged.

A merge queue will schedule their merging sequentially and ensure you that they are all updated on top of each other. Obviously, these updates are only done if the pull requests complete all the conditions.

Nevertheless, what happens when you merge an updated pull request and then get another one which is still outdated. To ease your understanding, what about another illustration?

A merge queue will make sure the second pull request is updated with the latest tip of the main branch before merging. By doing this, there is no way to merge an outdated and broken pull request into the main branch.

And you repeat this as many times as necessary, for each outdated pull request in the queue.

Life is not always easy, but a merge queue can make it a lot easier.

How Does a Merge Queue Work?

Now that you know what kind of issue a merge queue can solve, let’s summarize its functioning.

As you probably understood, a merge queue looks more or less like a complex workflow we can analyze step by step.

1️. Queue Valid PRs

Run the engine on your pull requests. All valid pull requests — the ones which validated all the conditions can be added to the queue.

2. Update and CI

The merge queue will make sure that each PR in the queue is up-to-date by merging the main branch into the feature branch.

Then the CI rerun to make sure the PR is mergeable.

3. To Merge or not to Merge, that’s the Question

Two quite distinct cases:

All checks passed → merge the PR.
Failing test → remove the PR from the queue.

What About Mergify’s Merge Queue?

In concrete terms, Mergify’s Merge Queue does everything you just read.

It is one of the first merge queues launched on the market and already has thousands of happy users.

While the common features presented before can solve many headaches, you may find yourself facing even more complex and specific situations. These situations require very specific functionalities.

Good news, with Mergify, you already have them at your fingertips!

✅ Speculative Checks: Testing Different PRs in Parallel

The first pull requests from the queue are embarked in a merge train and tested together in parallel so they can be merged faster.

✅ Batches: Check and Merge Multiple PRs at Once.

Mergify allows checking the mergeability of multiple pull requests at once using the batch_size option.

✅ Multiple Queues: Put PRs in a Dedicated Queue.

By using multiple queues, it’s possible to put some pull requests in a higher priority queue.

✅ Queue Freeze: Stopping All Merging Processes.

Mergify allows freezing the merge of one or several queues simultaneously to provide maximum control and flexibility on how and when you want the code to be merged.

✅ Priority Management: Prioritize your Pull Requests.

Choose which PR should be merged first, depending on labels, owners, etc. Your rules ultimately!

Demo Video

Conclusion

Here you are folks, you should know everything about the concept of a merge queue. From how it works to why you should use one, merge queues have no secrets for you.

It’s now time to try it and you know what? You can enjoy a 14-day trial for free with Mergify. I guess it’s your chance to say bye-bye to this outdated pull request issue.