Forem: Ilya

What is inside Rate Limiting for .NET

Ilya — Tue, 19 Nov 2024 15:05:00 +0000

The Rate Limiting API debuted in .NET 7. It implements several popular algorithms for limiting the number of requests to a shared resource. This API is typically promoted as a part of a built-in rate-limiting ASP.NET Core middleware. However, the API itself doesn't depend on ASP.NET and has a broader scope. This API is recently written and may reflect the current state of concurrency in .NET. Finally, it's a production-ready library, not a book example of a semaphore in a loop with sleep. So let's take a look inside and see if we can learn something.

Rate Limiting API overview

Disclaimer: In a mature system, and for the sake of horizontal scaling, rate limiting is often implemented at the infrastructure layer (e.g. Rate Limiting with NGINX), except for some tricky quota partitioning driven by domain logic.

Middleware

As mentioned above, there is a built-in RateLimitingMiddleware in ASP.NET Core. Its basic usage is extensively covered in Microsoft Learn and community blogs, so allow me to skip it. There is not much inside: the midlleware basically does two things:

Choosing a limiter by the endpoint's metatada and global settings.
Trying to acquire the protected resource (i.e. the endpoint).

Things get more interesting at the rate limiter level.

Limiting algorithms

System.Threading.RateLimiting namespace contains limiters that implement the following algorithms:

Each limiter, except ConcurrencyLimiter, caps the number of requests in a time period. ConcurrencyLimiter caps only the number of concurrent requests. All limiters are derived from the abstract RateLimiter class and implement the following public API:

// Fast synchronous attempt to acquire permits.
RateLimitLease AttemptAcquire(int permitCount);

// Await until the requested permits are available
// or can no longer be acquired.
ValueTask<RateLimitLease> AcquireAsync(int permitCount);

We can see how both methods are used at the middleware level. First, a fast sync attempt is made. If it fails, then the async method is called, possibly hitting a wait queue (I discuss this queue in the next section).

Partitioning and chaining

Another API worth mentioning is PartitionedRateLimiter<TResource>, which is responsible for:

Partitioning: selecting a limiter at runtime by the TResource value.
Chaining: combining limiters into a sequence.

An example of partitioning:

using var limiter = PartitionedRateLimiter.Create<string, int>(
    resource =>
{
    if (resource == "foo_resource_id")
    {
        return RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: 1, 
            _ => new FixedWindowRateLimiterOptions {...});
    }
    else
    {
        return RateLimitPartition.GetConcurrencyLimiter(
            partitionKey: 2,
            _ => new ConcurrencyLimiterOptions {...});
    }
});

var lease = limiter.AttemptAcquire("foo_resource_id");

An example of chaining:

using var limiter1 = PartitionedRateLimiter.Create<string, int>(
    resource =>
{
    return RateLimitPartition.Get(partitionKey: 1, 
        key => concurrencyLimiter1);
});

using var limiter2 = PartitionedRateLimiter.Create<string, int>(
    resource =>
{
    return RateLimitPartition.Get(partitionKey: 2, 
        key => concurrencyLimiter2);
});

using var chainedLimiter = PartitionedRateLimiter
    .CreateChained<string>(limiter1, limiter2);

Note that chaining is only supported for partitioned limiters. Non-partitioned limiters must be wrapped into partitioned ones. This wrapping doesn't prevent or otherwise affect the direct use of original limiters, i.e. the same instance can be used as a part of a chain and independently at the same time.

What is inside a limiter?

Let's discuss the main components of a limiter. It may seem obvious, but to implement a public API as in Limiting algorithms we need a permit counter, and a wait queue. For time-based limiters, we also need a timer.

Counter

Each limiter must track the amount of available permits. There is no fancy data structure behind this: _permitCount counter is just an integer private field. The real challenge is to correctly update that little cute field in a multi-threaded environment. For example, the inner logic of the AttemptAcquire method of ConcurrencyLimiter looks pretty simple...

RateLimitLease AttemptAcquire(int permitCount)
{
    if (_permitCount >= permitCount)
    {
        lock (Lock)
        {
            // it tries to reduce _permitCount inside:
            // _permitCount -= permitCount;
            if (TryLeaseUnsynchronized(permitCount, 
                out RateLimitLease? lease))
            {
                return lease;
            }
        }
    }
}

Ignoring the fact that I've omitted 99% of the supplementary code, which covers thread synchronisation and various edge cases. A LOT of edge cases. Ensuring secure write access to shared data is inherently complex, and the limiter's complete code is no joke. And there is even more to discuss.

Wait queue

Any limiter from above supports queuing permit requests, which can be de-queued (FIFO or LIFO) at some point in the future when the permits become available. Suitable job for a double-linked list, or a double-ended queue! The developers chose the latter, and built the wait queue on top of the internal Deque collection. There is a long-term discussion about implementing a public version of Deque, but unfortunately it is not even close to a deal.

This double-ended queue has many similarities with such well-known collections as Stack<T> or Queue<T>:

It is built on top of... a simple T[] array with some index math around it.
It grows in size by allocating a new array of x2 size when it reaches the current array capacity.
It is not thread-safe.

The last point is the most important: again, lock statements are heavily used to ensure exclusive access to a shared Deque instance.

Timer

Another important building block is a timer. Indeed, except for ConcurrencyLimiter, each of the rate limiting algorithms considered has to repeat an action over time (to replenish the bucket, move the window and so on). This is where System.Threading.Timer comes in handy, it's a lightweight class that can execute a callback method on a thread from the thread pool at regular intervals.

For TokenBucketRateLimiter, implementing bucket replenishment is pretty straightforward:

_renewTimer = new Timer(
                    Replenish, 
                    this, 
                    _options.ReplenishmentPeriod);

Things get more interesting in DefaultPartitionedRateLimiter<TResource, TKey>, where TimerAwaitable is introduced as an internal async-over-sync wrapper over the synchronous System.Threading.Timer API:

_timer = new TimerAwaitable(timerInterval, timerInterval);
_timerTask = RunTimer();

private async Task RunTimer()
{
    _timer.Start();
    while (await _timer)
    {
        try
        {
            // handle a cache of created limiters
            await Heartbeat().ConfigureAwait(false);
        }
        catch { }
    }
    _timer.Dispose();
}

TimerAwaitable is a good reminder of the fact that not every awaitable expression has to be a Task. In fact, the compiler requirements for 'await' are quite loose.

Let's have a look at what's in TimerAwaitable:

private void Tick()
{
    Interlocked.Exchange(ref _callback, _callbackCompleted);
}

public TimerAwaitable GetAwaiter() => this;

public bool IsCompleted => ReferenceEquals(_callback, _callbackCompleted);

public bool GetResult()
{
    _callback = null;
    return _running;
}

Here is a System.Threading.Timer instance with only one job: on every "tick" set _callback field to its "completed" value. This field, despite the name, is mostly used to determine whether this quazi-task is completed or not. The value is later reset to null in the GetResult call. These methods plus the implementation of the ICriticalNotifyCompletion interface allow to await the timer instance itself, without allocation of a new task!

Usage Example

You might have expected an example of how to use the Rate Limiting API at the end of this post. I was going to, but then I had a better idea. In this post I've mostly shown code written by someone else (probably by a better software engineer than me), so let's continue that exploitation.

This API is extensively covered by tests, like this one. These tests are a better example than I could ever provide. They are isolated and run locally, assuming you have cloned and built the dotnet/runtime repo on your machine. I strongly recommend that you give it a try. Building the entire .NET runtime from source may seem intimidating at first, and it will take some time. But if you follow the Workflow Guide, this mega project shall compile without a single warning.

Summary

In this post I discussed the internals of the built-in .NET Rate Limiting API. I showed how the limiter behavour can be tailored to your application's needs by using techniques such as partitioning and chaining. I examined the source code in the System.Threading.RateLimiting namespace and discussed the key components of a limiter: a counter, a wait queue, and a timer.

The most popular examples of using the Rate Limiting API are also the most controversial. It's natural to enable rate limiting for inbound HTTP requests at the infrastructure layer, while outbound limiting is easily implemented within an application using Polly. I would love to hear about other real-world uses of the Rate Limiting API, so please let me know if you have one.

Introducing Awesome .NET Testing ✅

Ilya — Mon, 14 Oct 2024 13:30:00 +0000

I have a confession to make. I love writing automated tests. That's why I've created awesome-dotnet-testing – a curated list of awesome .NET libraries and tools that help ensure code quality through automated testing. While my source of inspiration was awesome-dotnet-core, sadly that massive project doesn't seem to be actively maintained anymore, with some notable projects missing, and some obsolete. This is why I have started a new categorised list with a focus solely on testing, and I'm going to keep it up to date and comprehensive. I've even learned a few new things along the way, even though I've been working with testing on .NET for a long time. So I would be happy if it helps someone.

Contents so far

Designing HTTP API clients in .NET

Ilya — Wed, 21 Aug 2024 08:00:00 +0000

In this post, I describe a scalable architectural approach to building typed HTTP API clients for ASP.NET Core applications. I discuss how following this approach can contribute to API testability and maintainability. I provide some practical examples and highlight common pitfalls.

Problem definition

Distributed systems are popular. HTTP APIs are very popular. And .NET is a reasonable tool for implementing both, starting from the classic RESTful APIs and microservices-oriented architecture.

While one of the arguments in favour of microservices is sharing the workload between multiple development teams using different programming languages (and who said anything about Conway's Law), in this article I'd like to talk about a homogeneous, pure .NET ecosystem. Most of the stuff below is relevant to ASP.NET Core Web API applications. Over the years, I have designed, implemented, or overseen many of these, and I want to share the best practices I have learned.

Let's start with a published REST API. On the consumer side, managed by another team, the typed HTTP client code is written. Hopefully, its implementation is based on some docs shipped with the API. Then it just works, right? Well, yes and no. A typical enterprise-grade distributed system may have thousands of endpoints and hundreds of respectful consumers, developed and maintained by dozens of teams. Independently re-creating HTTP clients from scratch in each and every consumer project is out of the question, because it doesn't scale at all.

There are several ways to save time and reduce the amount of repeated boilerplate code:

Reuse existing client code (NuGet).
Use a high-level client builder for each client.
Generate client code on each consumer using some form of specification from the producer (OpenAPI).

And while this post revolves mostly around NuGet, it would be wrong of me not to mention the others.

High-level client builders

There are many third-party REST client builders for .NET on GitHub, and new ones keep popping up for some reason. Some of the alive and popular are:

Part of the community is quite happy using these libraries. However, based on my experience so far, and considering the occasional problems I've encountered (limitations, missing features, or questionable design), I wouldn't make a high-level builder the default choice in a large system. But never say never.

OpenAPI / Swagger

Swagger is a widely used toolset based on the OpenAPI specification that needs no introduction. For instance, an OpenAPI/Swagger schema can be generated from ASP.NET Core app using Swashbuckle, NSwag, or the new fancy .NET 9 built-in generator. And then, given that document, the corresponding client can be generated in any popular programming language. Here is a detailed list of .NET libraries for client code generation.

However, OpenAPI's declarative resource specification does not cover your implementation details. It's simply designed to do the opposite: provide a language-agnostic way to express an API contract. And what are implementation details of an HTTP client? Authentication, authorization, serialization, validation, error handling, and so on.

Don't get me wrong: Swagger is great, use it, and keep the documents up to date. Your colleagues will appreciate it. But for the .NET parts of a distributed system we can smooth backend-to-backend integrations... with good old NuGet packages.

Solution structure

Here is the proposed dependency diagram for a typical ASP.NET Core application that implements POST /orders REST API:

As the root, there is Orders.Client class library containing a typed client and, naturally, all its input/output DTOs. This library is also published as NuGet package.
Web project depends directly on Orders.Client, and reuses ApiModels as endpoints' inputs/outputs.
Tests project reuses the typed client in API tests (I explain my definition of "API tests" in the next section).
Finally, an external consumer of POST /orders references the Orders.Client NuGet package.

An example of an input/output DTO could be OrderCreateApiModel, into which the POST /orders request body is deserialized. By the way, I prefer public contracts like this one to follow some global naming convention, i.e. "ApiModel" in this case. Such a rule serves well as a big red sign that any thoughtless change of such type can break the API backward comparability.

Typical code for each project might look like this:

Orders.Client:

//omitted for brewity: interfaces, cancellation tokens, attributes etc. 
public class OrdersClient(HttpClient httpClient) : IOrdersClient 
{
    public async Task<OrderApiModel> CreateOrderAsync(
        CreateOrderApiModel input);
}

Orders.Web:

public OrderController : ApiController
{
    async Task<OrderApiModel> CreateOrderAsync(
        CreateOrderApiModel input);
}

Orders.Web.Tests (based on WebApplicationFactory):

public class VeryBasicTests(WebApplicationFactory<Program> factory) 
    : IClassFixture<WebApplicationFactory<Program>>
{
    private readonly OrderClient _client = new(factory.CreateClient());

    /// <see cref="OrderController.Create" />
    [Fact]
    public async Task CreateOrderAsync_Default_OK()
    {
        await _client.CreateOrderAsync(new CreateOrderApiModel() {...});
        //omitted for brewity: Act, Assert steps
    }
}

Advantages of sharing a typed client project:

Explicit contract. The Orders.Client project declares and maintains a public contract. All API changes start from here. During code review, this project is your starting point for hunting API-breaking changes.
Reusable code. As you can see in the above code, the same ApiModel types are used in a typed client, a controller, a NuGet package, even in API tests!
Tested code. You write API tests against the typed client, which means you also test the public client itself.
Code scaffold assistance. Having only the client interface, it is possible to generate a stub for the client itself, for the referenced ASP.NET Core controller/action (and vice versa). Generate a fully functional API test for each method of the typed client (i.e., for each public endpoint you declare).
Easy control over the dependencies of your contracts. You can even automate it by writing architectural tests (NetArchTest, ArchUnitNET).

Frankly, I want to have the same project structure even when there are no NuGet package consumers planned, see below.

API tests

Disclaimer: An effective approach to testing is to balance efforts across different levels of testing, i.e. to follow The Test Pyramid paradigm. And while this section focuses on in-memory unit/integration testing, nothing I say below implies in any way that system/end-to-end tests are not needed (they are the absolute must).

Advantages of using a typed client in testing

In-memory HTTP API testing can be tricky. Fortunately, ASP.NET Core ships with an in-memory test server that effectively reduces the cost of client-server emulation, and has full support for typed clients. Let's examine the benefits of reusing a typed client in tests:

Unified codebase. If you don't specify a team-wide convention for declaring a public contract in the form of a well-designed typed HTTP client... Then, it's an invitation to take a quick-and-dirty approach to writing API tests. Such as hardcoded calls to HttpClient, or at best a homemade typed client with potentially wrong serialization settings or a messed up HTTP handler pipeline.

Code coverage insights. Without a facade for making API calls, uncovered endpoints can only be reliably tracked at the coverage analysis stage. This is not bad, but we can do better. If the public method of a typed client has no usages, this usually means that it is not explicitly called in a test. In effect, it means that the referenced API endpoint itself is not covered. These client's unused methods are easy prey for static code analyzers, giving you compile-time test coverage insights before you even run a code coverage collector.

Therefore, even if there are no .NET consumers for your APIs, building a typed client is still recommended.

Giving API tests the right purpose

Then let's talk about what logic API tests should cover. Assuming you have separate sets of:

End-to-end language-agnostic tests (this is where Swagger can help a lot), ideally run somewhere in a test environment.
Unit and integration tests that cover business logic and data access layer.

Then there is little reason to play in their domain. On the contrary, at the API testing level, I'd like to have tested only the implementation details of the HTTP APIs:

Endpoint address.
(De)serialization of input/output data.
Application-level error handling (usually implemented via ASP.NET exception handler or middleware).
Middleware pipeline.
Authorization policy.

Thus the less business logic involved in API tests, the better. Mock it. Make the tests simple, fast, isolated, and suitable for continuous testing. This approach fits well with thin controllers (using CQRS and/or the Mediator pattern) if your controller's actions have a single, easy to mock dependency a'la IRequestHandler<TInput, TOutput>:

[HttpPost("/orders")]
public Task<OrderApiModel> Create(
    [FromServices] 
    IRequestHandler<CreateOrderApiModel, OrderApiModel> handler,
    [FromBody] 
    CreateOrderApiModel input)
{
    //...
    return handler.InvokeAsync(input);
}

Key points:

Business logic is effectively decoupled from the communication layer.
API tests cover only the communication layer.
No messing with controllers in tests. Testing a fat controller may be tricky because of one big implicit dependency: HttpContext. In short, I recommend abstaining from manual controller creation in tests.

The other decorating handler

Custom HTTP handlers are well known as a mechanism to manage cross-cutting concerns around HTTP requests. The calling application has control over the HTTP handler pipeline, so it can be reconfigured, reordered, or even rebuilt from scratch. Decorating a client with a Token Management Handler or a custom Polly policy is easy... assuming the client accepts an HttpClient parameter in its constructor, and you haven't messed with the natural order of things by obstructing the client customization in some way (I really don't want to show how).

One thing to emphasize here is that a consumer may have their own reasons for decorating HTTP calls. So even if you are driven by good intentions, keep the public constructor. Even if your API needs that bunch of client-side decorators so badly that you bundled it with the client, at least provide an extension point for the handler pipeline. Otherwise, one day you will find that your "smart" factory method has just been decompiled and rewritten on the consumer side. Just because it was cheaper and faster than asking your team for changes.

DI registration

As mentioned above, keep the registration simple and open for extension. For instance, such an unobtrusive registration could be a wrap over a built-in AddHttpClient<TInterface, TImpl> call:

public static IHttpClientBuilder AddClient<TInterface, TImpl>(
    this IServiceCollection services,
    Func<IServiceProvider, string> getBaseAddress) // important
    where TImpl : class, TInterface
    where TInterface : class
{
    //any registration method should be self-contained if possible
    services.AddYourCustomHttpHandlers(provider => {...});

    var builder = services.AddHttpClient<TInterface, TImpl>(
    (provider, httpClient) =>
    {
        httpClient.BaseAddress = new Uri(getBaseAddress(provider));
    })
    .SetYourCustomHttpHandlers();

    //builder is returned to allow further extension
    return builder;
}

Caller side:

services.TryAddSimpleClient<IFooClient, FooClient>(
    provider => provider
        .GetRequiredService<AppSettings>()
        .Middleware.Foo.Url);

As you can see from the code above, instead of passing a settings object directly to the AddClient method, a delegate is passed. This doesn't force the caller to provide the dependency immediately and works well with ASP.NET Core configuration.

Key points:

Make the registration method self-contained.
Explicitly declare dependencies.
Never ask for configuration values before the ServiceProvider is built.
Let the caller decide how to resolve dependencies.

Adding client-side logic

How about something tightly coupled to the typed client itself? Like input validation:

public Task<OrderApiModel> CreateAsync(CreateOrderApiModel input)
{
    _validator.ThrowIfInvalid(input);

    var response = await httpClient.PostAsJsonAsync("orders", input);

    // ...
}

Calling ThrowIfInvalid(input) promises great benefits indeed:

No HTTP call on invalid input.
Full error data is provided to the caller: we avoid the pain of stripping an exception of all security-sensitive info, fitting it into a ProblemDetails compliant response (a'la ValidationProblemDetails), and deserializing it on the client side.

What could go wrong? Well, depending on the validator, we may have just duplicated our business logic to an unknown number of external systems, effectively reducing the effort of building a distributed application almost to zero. While validating the input data makes some sense, business rules (I have covered the difference in more detail here) should under no circumstances leak to the caller.

With this in mind, I would recommend:

Keeping business logic where it belongs: in the domain.
Reducing a typed client's inner logic to the bare minumum.
Treating any typed client's auxiliary inner logic as a "high risk – low reward" move.

Making the client dependent on another package

Another questionable design decision is adding external dependencies to a typed client assembly. For example, referencing a core package with helper methods, default serialization settings (which I highly recommend defining), and all that shared sfuff. Another example might be using a third-party library. Like Newtonsoft.Json was the preferred way to handle JSON input/output in a typed client before we got System.Text.Json. And there is nothing wrong with that.

However, any client dependency can lead to a future case of the notorious "DLL hell" problem. While multiple major versions of the same package are transitively referenced by the root application, we have exactly one package version bound at runtime. Then, when we call the other version transitively, we can get a nasty runtime error. In general form, it is unsolvable on the root application side. More technical details can be found, for example, in this thread and its follow-ups: Referencing multiple package versions within one project with extern aliases.

Things often get ugly in a service that implements Saga or Facade patterns:

Of course, you could just update all your (N+1) typed clients at once in case of a breaking change in the core library, but this approach doesn't scale well.

Again, avoid external dependencies unless it absolutely necessary, and think twice before adding anything to the library.

Summary

In this post, I described a scalable architectural approach to building typed HTTP API clients on top of ASP.NET Core applications. I outlined a number of practices that contribute to API testability and maintainability. I advocated the use of NuGet packages over autogenerated client code. I also talked about customizing the HTTP handler pipeline, adding client-side logic, and avoiding the "DLL hell" problem.

Nested validation in .NET

Ilya — Wed, 03 Jul 2024 08:39:29 +0000

In this blog's opening post, I discuss the problem of validating nested Data Transfer Objects in modern .NET. Nesting simply means that the root object can reference other DTOs, which in turn can reference others and so on, potentially forming a cyclic graph of unknown size. For each node in the graph, its data properties are validated against a quite typical rule set: nullability, range, length, regular expressions etc.
And for DTO types, let's declare the following conventions:

It may have DataAnnotation attributes, including custom ones.
It may implement IValidatableObject.
It should avoid third-party dependencies if possible.

You may have guessed that the graph is the tricky part. Indeed, a built-in DataAnnotations.Validator doesn't do nested validation by design, and this was a default behaviour for decades. But the fix is trivial, right? Just implement any kind of graph traversal with cycle detection! Well, yes and no. In this post, I compare popular third-party libraries that support nested validation. Looking ahead, there is a big performance difference even among robust production-ready solutions.

There are many ways to define validation rules in .NET, each with its own advantages and disadvantages. For example:

Attributes: explicit, useful for OpenAPI document generation.
IValidatableObject: more flexible yet still self-contained.
External: This is a jack of all trades. It leaves DTOs clean and provides maximum flexibility (FluentValidation is the best example of this approach).
Manual validation: the most naive approach, it simply has inlined if clauses without declaring validation rules at all. As a result, it gives unbeatable performance at the cost of scalability, and it doesn't apply to a graph of unknown length/topology. Later it is used as a benchmark baseline.

To finish this long intro and save everyone's time, let me highlight what is not covered in this article:

ASP.NET Model Validation. Although it comes with full support for DataAnnotations attributes, it is still an inseparable part of a large and complex framework that deals with both server-side application and Web APIs, ModelState, version backward comparability, etc... a topic that undoubtedly deserves its own article.
IOptions<T> validation. Ironically, with the arrival of [ValidateObjectMembers] and [ValidateEnumeratedItems] in .NET 8, OptionsBuilder<TOptions> now supports validation of nested options. And there are now at least 3 different validation algorithms shipped with ASP.NET.

What is validation?

Let's say we're processing a user's registration email address. What should we check?

The address should be in the correct format. This is validation.
The address domain should not be on our blacklist. This is a business rule.
The address should be unique in our database. This is a business rule.

What is the difference? Validation is a pure function. It is deterministic (same input - same output) and has no side effects. That's why looking for a domain in a list is not validation: such lists are subject to change, so they're not deterministic. A good rule of thumb for mere enterprise developers like me:

Validation: self-contained (we only need the data from the DTO itself)
Business rule: anything that touches mutable data (database, API, file system etc.)

And my advice is: don't mix them up. Validate your input before the control flow even reaches your business domain. Just like ASP.NET does with model binding. Regardless of the application architecture, in many cases you actually want fail fast on invalid/malicious input and avoid unnecessary allocation of your scoped and transient services. Then, testing: covering pure functions with tests is trivial. Well, at least it is way easier to do separately, than mocking a database and couple of APIs for all-at-once validator. Put some effort into the quality of the data coming into your domain, and you'll get a clearer and more concise domain logic.

To go deeper, please read Mark Seemann's Validation and business rules post, discussing the topic in great detail. Let me say a few things about the libraries under consideration, and we can finally get on with the benchmarking.

DataAnnotationsValidator

Our first contender is the DataAnnotationsValidator.NETCore package. It is long dead and has performance issues, so strongly not recommended. However, this library illustrates well the idea behind many home-made solutions:

Reflection to read metadata.
Recursive depth-first search for traversing a graph.
A hash set for cycle detection.

MiniValidation

Alive and well-designed, MiniValidation offers smooth experience in nested validation. While implementing a similar depth-first search for visiting a DTO graph, it adds metadata caching to the mix, resulting in much better performance.

FluentValidation

FluentValidation is undoubtedly the most popular third-party validation library on .NET. It is a robust choice if you need clean POCOs or multiple validation maps per type. However, its performance may surprise you.

Benchmark: DataAnnotation and FluentValidation

Our first benchmark is to validate a fairly typical DataAnnotation-marked DTO, containing both a single nested object and a collection of them (each is expected to be validated):

public class Parent
{
    [Range(1, 9999)]
    public int Id { get; set; }

    [Required(AllowEmptyStrings = false)]
    [StringLength(12, MinimumLength = 12)]
    public string? Name { get; set; }

    [Required]
    public Child? Child { get; set; }

    [Required]
    public List<Child> Children { get; init; } = new(0);
}

public class Child : IChild
{
    [Required]
    public DateTime? ChildCreatedAt { get; set; }

    [AllowedValues(true)]
    public bool ChildFlag { get; set; }
}

Of course, FluentValidation has no use for these attributes, so its validators are created separately while repeating the same rules:

public class ParentValidator : AbstractValidator<Parent>
{
    public ParentValidator()
    {
        RuleFor(x => x.Id).InclusiveBetween(1, 9999);
        RuleFor(x => x.Name).NotEmpty().Length(min: 12, max: 12);
        RuleFor(x => x.Child).NotNull().SetValidator(new ChildValidator());
        RuleForEach(x => x.Children).NotNull().SetValidator(new ChildValidator());
    }
}
public class ChildValidator : AbstractValidator<Child>
{
    public ChildValidator()
    {
        RuleFor(x => x.ChildCreatedAt).NotNull();
        RuleFor(x => x.ChildFlag).Equal(true);
    }
}

Finally, the Manual benchmark uses explicit if checks and serves as a baseline. Each benchmark is runned against of the same Parent collection. There are the results depending on the collection size:

Method	Size	Mean	Allocated	Alloc Ratio
Manual	100	3 μs	34 KB	1
MiniValidation	100	162 μs	427 KB	12
DataAnnotationsValidator	100	302 μs	614 KB	17
FluentValidation	100	314 μs	946 KB	27
Manual	1000	33 μs	343 KB	1
MiniValidation	1000	1586 μs	4260 KB	12
DataAnnotationsValidator	1000	3084 μs	6150 KB	17
FluentValidation	1000	3300 μs	9586 KB	27
Manual	10000	342 μs	3437 KB	1
MiniValidation	10000	16237 μs	42619 KB	12
DataAnnotationsValidator	10000	31223 μs	61480 KB	17
FluentValidation	10000	32364 μs	95911 KB	27

Well, DataAnnotationsValidator is expectedly bad, but FluentValidation... is even worse in both time and space! At first I thought there was a bug (there was not). Then I did my best to look for FluentValidation settings that might help to optimise its performance (there weren't any, except "fail fast", see below). The overall result distribution remains the same.
But look at MiniValidation! The same algorithm, but optimised for performance, gives a quite impressive 2x boost over DataAnnotationsValidator.

Benchmark: IValidatableObject

As you probably know, IValidatableObject is an alternative to explicit DataAnnotations attributes, with all the validation logic encapsulated within DTOs. This benchmark uses the same validation rules but implemented in Validate method, so it's all about traversing a graph and calling Validate at each node. FluentValidation is not on the list this time.

public class ChildValidatableObject : IValidatableObject
{
    public DateTime? ChildCreatedAt { get; set; }
    public bool ChildFlag { get; set; }

    public IEnumerable<ValidationResult> Validate(ValidationContext validationContext)
    {
        if (ChildCreatedAt == null)
        {
            yield return new ValidationResult("foo error message #2", 
                new[] { nameof(ChildCreatedAt) });
        }

        if (ChildFlag == false)
        {
            yield return new ValidationResult("foo error message #3", 
                new[] { nameof(ChildFlag) });
        }
    }
}

Method	Size	Mean	Allocated	Alloc Ratio
'Manual with IVO.Validate call'	100	21 μs	109 KB	1.00
'MiniValidation + IVO'	100	59 μs	199 KB	1.82
'DataAnnotationsValidator + IVO'	100	151 μs	442 KB	4.04
'Manual with IVO.Validate call'	1000	206 μs	1093 KB	1.00
'MiniValidation + IVO'	1000	565 μs	1992 KB	1.82
'DataAnnotationsValidator + IVO'	1000	1511 μs	4421 KB	4.04
'Manual with IVO.Validate call'	10000	2141 μs	10937 KB	1.00
'MiniValidation + IVO'	10000	6608 μs	19921 KB	1.82
'DataAnnotationsValidator + IVO'	10000	16254 μs	44219 KB	4.04

Again, MiniValidation wins by an even larger margin. Now let's merge the results and look at the overall performance (values rounded for readability):

Method	Size	Mean	Allocated
Manual	10000	342 μs	3437 KB
Manual with IVO.Validate call	10000	2141 μs	10937 KB
MiniValidation + IVO	10000	6608 μs	19921 KB
MiniValidation	10000	16237 μs	42619 KB
DataAnnotationsValidator + IVO	10000	16254 μs	44219 KB
DataAnnotationsValidator	10000	31223 μs	61480 KB
FluentValidation	10000	32364 μs	95912 KB

You may notice that MiniValidation + IValidatableObject give the best results of all third party libraries.

Benchmark: Fail fast

And yet FluentValidation has the feature that other competitors lack: CascadeMode.Stop. It's flexible and can be set at different levels (rule, class, global):

public class FailfastChildValidator : AbstractValidator<Child>
{
    public FailfastChildValidator()
    {
        ClassLevelCascadeMode = CascadeMode.Stop;
        //All the rules are declared as usual
        //...
    }
}

Method	Size	Mean	Allocated
FluentValidation + Fail Fast	10000	9012 μs	38556 KB
FluentValidation	10000	32364 μs	95911 KB

Of course, the fail-fast version is much faster. Most of the time I prefer the full validation report, but fail-fast is an option worth mentioning when talking about performance.

Summary

In this post I discussed the problem of validating nested objects in .NET. Since the built-in DataAnnotations validator doesn't traverse complex properties, we have to rely on third-party libraries for this. I explained the difference between validation and business rules, and why this is important.

As for the benchmark results:

The MiniValidation library shows the best overall performance.
FluentValidation, despite its popularity, is generally 2x slower. There are some faster alternatives, such as Validot, but I would like to leave the burden of benchmarking to its maintainers.

And don't get me wrong. If you want to decouple your rules from DTOs and get a simple, stable and production-tested solution - just take FluentValidation, because its performance difference is negligible in many cases. If you need self-describing DTOs - stick with MiniValidation. And for performance driven code - inline your checks where possible.

The obvious next step in the development of general purpose validation libraries, is, of course, the adoption of ~~ChatGPT~~ source generators. A validation generator such as this one would potentially eliminate the performance gap between general usage validation libraries and inlined validation. In fact, we already have all the necessary technology shipped with .NET, so stay tuned for news!

All the code from the article is available on Github: https://github.com/ilya-chumakov/PaperSource.DtoGraphValidation.