Forem: Russ Hammett

Microsoft Orleans: Grain Caches

Russ Hammett — Thu, 15 Dec 2022 16:00:00 +0000

Holy moly it’s been a while. Time for some more Orleans goodness, hopefully? Caches!

Overview

What are caches and what do they do? As usual, going to lean on Wikipedia to help me out:

From https://en.wikipedia.org/wiki/Cache_(computing):

In computing, a cache is a hardware or software component that stores data so that future requests for that data can be served faster; the data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere.

So basically a means of getting data faster, sign me up!

Often, you’ll see multiple levels of caching on varying pieces of infrastructure, whether those pieces are all contained within a single piece of hardware (like a computer), or spread across multiple pieces of hardware (like a network). We have the need for speed, and caches gonna give it to ya… ideally!

Problems with caching

https://aws.amazon.com/builders-library/caching-challenges-and-strategies/ has a lot of good information, but just to name a few:

Stale data
Inconsistent data across caches
In the case of local caches - memory and/or additional CPU pressure on a system that is otherwise acting as a webserver
In the case of external caches - additional system complexity, additional hop to get to data (though seemingly still better than having to rerun some sort of querying/aggregating of data from potentially numerous data sources when not caching)

A (naive) Orleans Cache

So, it’s been a while since I’ve written anything, much less about Orleans. It’s Hacktoberfest though so I at least gotta try to get that shirt!

Like in past posts where I’m demonstrating something Orleansly, we’ll be working from my github repo starting here https://github.com/Kritner-Blogs/OrleansGettingStarted/releases/tag/v0.62.0. Prior to us creating another IOrleansFunction, we actually need to write the thing we’re be demonstrating, the cache!

A cache abstraction can (and perhaps should?) have more methods available to it than what we’ll be doing in this post, but this isn’t meant to be a full fledged solution.

public interface IOrleansCache<TValue> : IGrainWithStringKey, IGrainInterfaceMarker
{
    Task AddOrUpdate(string key, TValue value);
    Task<TValue> Get(string key);
}

The above should look pretty straight forward, but just to go over it anyway:

TValue is the type of value our cache will be storing
IGrainWithStringKey states that cache will have a string primary key
AddOrUpdate - for adding/updating values based on a string key (not to be confused with the grain primary key)
Get for retrieving a value from the cache based on a string key

In case it isn’t obvious, we have two “keys” in the above declaration. A grain “primary” key, and a “key value pair” key. What’s the difference? Well by having the primary key be a string, we can get different grain instances for each individual string key we provide. Why would we do that? You may have different caches with different value types you want to store, bring up a different cache per customer, or a bit of both.

As for the implementation of the grain, I’m leaning heavily on an LRU cache from the package https://github.com/bitfaster/BitFaster.Caching/.

public class OrleansLruCache<TValue> : Grain, IOrleansCache<TValue>
{
    private readonly ConcurrentLru<string, TValue> _cache;

    public OrleansLruCache()
    {
        _cache = new ConcurrentLru<string, TValue>(50);
    }

    public Task AddOrUpdate(string key, TValue value)
    {
        _cache.AddOrUpdate(key, value);
        return Task.CompletedTask;
    }

    public Task<TValue> Get(string key)
    {
        if (_cache.TryGet(key, out var result))
            return Task.FromResult(result);

        TValue nullResult = default;
        return Task.FromResult(nullResult);
    }
}

Again this should all be pretty self explanatory, just a plain old consuming of a cache and not (currently) worrying about things like parameterizing configuration, providing cache eviction methods, persisting the cache to disk in any way, etc.

Wrap up

Keeping this one pretty short and sweet, I implemented within the final code (found here https://github.com/Kritner-Blogs/OrleansGettingStarted/releases/tag/v0.63.0) our IOrleansFunction so it can be played around with. I might have another post or two this month elaborating on this some more, just cuz I need to do some more PRs! (And get back into the habit of writing, but this is at least the 50th time I’ve said that)

References

Microsoft Orleans - Orleans.SyncWork now a part of OrleansContrib

Russ Hammett — Wed, 01 Dec 2021 18:40:00 +0000

The package I wrote about in this post is now a part of the OrleansContrib organization!

This is a pretty short post, I think, but after a tweet about the release of Orleans.SyncWork, there was a bit of back and forth in the thread:

I reached out to Reuben shortly after, and now the repository is a part of the OrleansContrib organization! I don’t think this actually means anything of super significance, but it’s pretty cool! Maybe the package will now get better visibility!

That’s it!

Resources:

CI/CD for .net 6, using GitHub actions

Russ Hammett — Wed, 01 Dec 2021 01:50:51 +0000

With the publishing of Orleans.SyncWork, I’ve had the opportunity to explore GitHub actions - which is a way to automate workflows. Here’s some of my first experience into the “action” (groan).

Automated workflow

First things first, what even is a workflow, and what does it mean to automate one? Well dear potential reader, a workflow is nothing more than a set of steps taken to complete a task.

From Wikipedia:

A workflow consists of an orchestrated and repeatable pattern of activity, enabled by the systematic organization of resources into processes that transform materials, provide services, or process information. It can be depicted as a sequence of operations, the work of a person or group, the work of an organization of staff, or one or more simple or complex mechanisms.

You can think of a workflow as the steps taken to accomplish “something”. That “something” can be any number of things, related to any number of subjects. In the context of this post, we’ll be mostly covering workflows as it relates to a build and release pipeline, also commonly referred to as continuous integration (CI) and continuous delivery (CD).

I’d like to cover both the CI and CD aspects of the Orleans.SyncWork, so let’s get started.

What you’ll (probably) need

Experience working with a CLI
Unit tests
An idea of the steps that you need to take in order to build, test, and deploy your code. If these steps are already in your head in the form of CLI commands, then you’re already most of the way there!
Some patience in getting your workflow properly laid out

Continuous Integration

Before you’re able to deploy code through a work flow (continuous delivery), you need to be able to integrate it safely into your main/trunk. For dotnet, through a handful of CLI commands, the building and testing of code is pretty straight forward. Doing CI has the added benefit of bringing up a brand new environment for builds, each and every time, a similar idea to why I’ve been a proponent of build servers for so long.

Build

dotnet build

The above command is the minimum you need to build either a solution file or project file on the dotnet side of things. From a continuous integration perspective, you may want to throw a few flags onto the command, such as:

dotnet build --configuration release

and things of that nature, see what’s available to you with the dotnet build documentation.

That could look like this from the CLI:

Test

Next is testing. I’ve probably already said it too many times, but test your code! Especially if you’re building libraries! Tests help ensure that the code you’re writing does what you say it does. Additionally tests be used as “documentation” in a way, if the tests are named well, and are invoking the code in a similar manner to how your consumer will use it, they’ll be in a better place to get started using what you’ve delivered.

Like the build command, the test command is quite straightforward as well:

dotnet test

Of course the above is the absolute bare minimum command, there are lots of parameters that can be passed to dotnet test as well.

A test run could look like this from the CLI:

CI Action

With the above dotnet build and dotnet test commands, we have most of what we’ll need to put together an action to build and test our code, automatically!

There is lots of good information, even some specific to .net testing on the documentation. I pretty much used the documentation as a starting point, and ended up with this…

.github/workflows/ci.yml:

name: Build and test

on:
  pull_request:
    branches: [ main ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: Setup .NET
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 6.0.x

    - name: Restore dependencies
      run: dotnet restore

    - name: Build
      run: dotnet build -c Release --no-restore

    - name: Test
      run: dotnet test -c Release --no-build --verbosity normal --filter "Category!=LongRunning"

The above file should be mostly straightforward, first we give a name to our workflow with name, specify the triggers for the workflow, in this case “on pull requests against the main branch”. The file then goes on to define the job “build”, which specifies an OS to run on, then steps. The steps do a few things of note:

Checkout the code, placing it into the ubuntu instance that is being utilized from the step previous
Setup .NET with a prebuilt action
restore dependencies
- This is an explicit step, rather than implicit from our previous dotnet build command, as if this explicit restore step were to fail, we’d more quickly know at what point of the build there is a failure
build the solution file at the project root
finally test the code

We have a few new flags in our build and test commands, namely specifying a configuration of release, and don’t restore/build on steps after those steps having already occurred. One final note is the --filter "Category!=LongRunning" - I was having trouble with the test runner getting through the tests I had laid out. They took 3 minutes to run locally, but ran for over 25 minutes on the build agent. Due to this fact, I added some classifications of “category” to the longer running tests, and excluded them from the test run in the above ci.yml file.

Continuous Delivery

Continuous delivery is a lot like continuous integration, and builds on top of it. I’m of the thinking that CD should do everything CI should do, or perhaps even better, actually rely on the CI, rather than redefining the steps in your CICD like I ended up doing. That was a bit rambly, but CD should do everything CI should do, except with the additional step of actually delivering (deploying/pushing) the code as a part of its workflow.

Delivery Complexities

That delivery part can have a lot of nuance to it that ups the complexity by a significant amount when compared to just “CI”. What does it mean to actually deliver code? Well, that could depend a lot on what type of code you’re actually delivering. In my case, I’m delivering a NuGet package, which has its own complexities, but what else is there? Well the other obvious thing that comes to mind is a web site / web api, one which could potentially have database changes to roll out in addition to the code. This to me, has the potential to be worlds more complex than just pushing a NuGet package up. How do you not only handle failures, but detect them and roll back, in the case of something going wrong with either you web push or database push? Perhaps I’ll be able to explore that one day, but for now, let’s get back to the NuGet package.

So, is there a complexity with delivering a NuGet package? Yes. NuGet package versioning can be a big undertaking when it comes to manual deployments, much less CD; as there is a requirement of NuGet packages being immutable. Does this mean that for every check in, on every potential branch that will be pushed to NuGet, you need to update some text file or code to indicate the next built version? That was my initial thinking, but thankfully that is not the case with the help of Nerdbank.GitVersioning.

I don’t think I have my CICD set up exactly how I’ll end up having it, but for right now it works. I installed the NerdBank.GitVersioning tool and package, and now for each build, I get a unique version number for the NuGet package upon build. I can toggle between prerelease or release packages, and can even publish “nightly” builds that contain a commit hash on them, all in the name of uniquely identifiable NuGet packages.

There was a fair amount of setup, that in some ways I’m still working through, but this article is already getting long enough, take a look at the PR(s) if you’re curious: https://github.com/OrleansContrib/Orleans.SyncWork/pull/8 and https://github.com/OrleansContrib/Orleans.SyncWork/pull/13. The “tldr” of it is, nbgv tooling uses the git history to rev the version number being used during builds, allowing for unique build numbers each time the CI/CD fires.

CD Action

There’s been a fair amount of information so far, but between our CI action and the information about GitVersioning, we have everything we need to put together a “first pass” cicd.yml. For CI, we were doing builds/tests against PRs to main. For CD, we’ll want to do delivery when code is pushed to main, as well as branches that begin with “RELEASE/v*”. My thinking here is that since we’ll be integrating often into main (theoretically), we don’t necessarily want to create full “new release packages” for every commit to main. We could however, create “prerelease” NuGet packages to main for each commit, making those changes available to the NuGet feed, but them not being labeled as a release version. Otherwise I have Nerdbank.GitVersioning set up to release “release” versions of packages from the “RELEASE/v*” branches.

The CD action file itself will look very similar to the CI one, just with the few additions of dotnet nuget... commands, shown below:

.github/workflows/cicd.yml

name: Build, test, and deploy

on:
  push:
    branches:
      - 'main'
      - 'RELEASE/v**'


jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: Setup .NET
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 6.0.x

    - name: Restore dependencies
      run: dotnet restore

    - name: Build
      run: dotnet build -c Release --no-restore

    - name: Test
      run: dotnet test -c Release --no-restore --no-build --verbosity normal --filter "Category!=LongRunning"

    - name: Pack
      run: dotnet pack src/Orleans.SyncWork/Orleans.SyncWork.csproj -c Release --no-restore --no-build --include-symbols -p:SymbolPackageFormat=snupkg -o .

    - name: Push to NuGet
      run: dotnet nuget push *.nupkg --skip-duplicate -s https://api.nuget.org/v3/index.json -k ${{secrets.NUGET_API_KEY}}

In the above, you’ll notice that it’s more than 50% “the same file” as CI. I was going to potentially look into composite actions as some point, so see if I could instead “chain” CI and CD, rather than redefining the CI within the CD file; but I’ve not yet had a chance to explore that.

Aside from the change to the “on” event (pull_request -> push), there are two new commands at the bottom dotnet pack and dotnet nuget push. The dotnet pack command is used to “package” up the project specified into a “.nupkg” file (and snupkg in this case for symbols). Finally the dotnet nuget push command is used to push those newly packed NuGet package(s) to the feed specified of nuget.org. On this command you’ll also see the {{secrets.NUGET_API_KEY}} portion of the command, this is defined as a repository secret and it can be used to pass “secret” information to things like workflows, in this case it’s my NuGet API key. These secrets can be set from the repository “Settings” -> “Secrets”:

Still to do

The “putting out a release branch” is still a bit of a manual step for me. I need to run nbgv prepare-release from my local environment, then push up the subsequently created “RELEASE/v*” branch and updated new pre-release version that is created under main.

That may not have made sense.

If I’m working in main with a prerelease version of “1.0-prerelease”, when I nbgv prepare-release, main will be (as an example) updated to “1.1-prerelease” with a branch called “RELEASE/v1.0” created having a release version of “1.0”. The push of these two changes will currently build a new prerelease package of “1.1-prerelease” and release package of “1.0”, both of which will contain “the same content” at the time of being pushed.

I’m not sure how I feel about the above. I like the automatic build and deploying of packages, but I don’t like having to create the release locally. I could conceivably create a manually dispatched workflow that did this release preparation for me, but then there’d still be the slight strangeness around immediately pushing out a prerelease package with no changes in comparison to the “previous” pre-release package built, and the new release package being built. I’m not sure what the “right flow” is quite yet, what I have right now does work, it just seems a bit messy.

Perhaps I’ll eventually look into workflows more like this:

CI - continues to be run on PRs to main
CICD - can be executed against the main branch and “RELEASE/v*” branches, but is not done automatically as it currently is
- I’m not sure about this part, main and the release should theoretically always be deployable, but do I really want to deploy every time there’s a change to main…?
Prepare release workflow - with this workflow, I’d want to do the “local” steps I currently take for preparing a release, but do it through a github action.

My Workflow

I wasn't sure if the literal sections were required, so here they are, "My Workflow" I think was pretty well described above, hopefully!

Submission Category:

Maintainer Must-Haves
DIY Deployments

Yaml File or Link to Code

OrleansContrib / Orleans.SyncWork

This package's intention is to expose an abstract base class to allow https://github.com/dotnet/orleans/ to work with long running CPU bound synchronous work, without becoming overloaded.

This package's intention is to expose an abstract base class to allow Orleans to work with long running, CPU bound, synchronous work, without becoming overloaded.

Built with an open source license, thanks Jetbrains!

Building

The project was built primarily with .net3 in mind, though the varying major version releases support .net6, .net7, and .net8; depending on the package version (should mirror the .net versions).

Requirements

Project Overview

There are several projects within this repository, all with the idea of demonstrating and/or testing the claim that the NuGet package https://www.nuget.org/packages/Orleans.SyncWork/ does what it is claimed it does.

Note that this project's major revision is kept in-line with the Orleans major version, so the project does not necessarily abide by SemVer, but we try as much as possible to do so. If breaking changes are introduced, descriptions of the breaking change…

View on GitHub

Additional Resources / Info

Microsoft Orleans - Long running CPU bound work

Russ Hammett — Wed, 01 Dec 2021 01:47:21 +0000

I’ve been talking about the idea of long running, CPU bound, synchronous work for a long time. Now, I’ve finally taken the time to build it out into a NuGet package!

Been talking about it for years at this point. Between my other posts on Orleans, or that one time (so far?) I was on a podcast to talk about it, I never got around to writing the thing down that I was always talking about. That has changed now in the form, and you too can solve problems (maybe!) with a small amount of code exposed by this NuGet package!

As a refresher, I needed to come up with a means of distributing compute calls for 10s of thousands of crypto calls, all hitting within a single moment. I had done some experimenting/proof of concepting, and at the time settled on Orleans to accomplish the distribution. Though I (still) haven’t used Orleans in more of an “intended use-case” of doing lots of distributed asynchronous work, using it in the way we are works pretty well for our use case.

If I were to do it all over again, perhaps it would make more sense to utilize a message queueing system with tried and true infrastructure. I’m not sure if I didn’t know of these concepts at the time, but this route may have been a better option. Even this option seems like it would “conceptually” work, but I’ve still not personally work bringing up a queue system with workers and notifications.

Anyway, onto the NuGet package!

The package

The package repository can be found: https://github.com/OrleansContrib/Orleans.SyncWork
And on NuGet: https://www.nuget.org/packages/Orleans.SyncWork/

There are only a few moving parts when it comes to the package:

ISyncWorker
SyncWorker
LimitedConcurrencyLevelTaskScheduler

ISyncWorker

All your CPU bound, long running, and synchronous work will be implementing from this class, though it will be through the extension of the SyncWorker<TRequest, TResult> abstract base class.

This interface exposes a few separate methods/contracts that allow for the interaction with the long running grain work:

Start(TRequest)
- This method is the “Entry point” into dispatching work to the grain. This method is the only one that takes in a parameter in the form of the arguments needed to perform the work. In the case of PasswordVerifier, those parameters are both “the password” and “the password hash”, both of which are needed to actually do the verification of a password.
GetWorkStatus()
- This method is used as a sort of “polling” mechanism, where callers can retrieve the current status of the grain work being worked on. The statuses Completed or Faulted will then allow for the return of data from one of the following two methods, depending no which status the grain is in.
GetResult()
- This method gets the result of a grain’s execution when the grain is in a Completed state, in the form of TResult.
GetException()
- This method gets the exception from a grain’s execution when the grain is in a Faulted state.

SyncWorker

This is the abstract base class of the grains. This class implements ISyncWorker<TRequest, TResult> and provides implementation details for the methods describe from the previously mentioned interface, as well as a few other methods:

CreateTask(TRequest request)
- This private method is invoked by the Start(TRequest request) method, which sets the _task state on the instance, and enqueues that long running work onto a LimitedConcurrencyLevelTaskScheduler
PerformWork(TRequest request)
- This is the abstract method to this abstract class, it is invoked through the creation of the _task, and is the “thing” that needs to be implemented within the implementations of this class.

LimitedConcurrencyLevelTaskScheduler

This is a class that was more or less copied from here. Its intended use is to limit the amount of work that can be performed at any one time as it relates to the work queued against this scheduler.

Without making use of this scheduler, queueing a massive amount of work on the “normal” scheduler quickly overwhelms the Orleans silo in such a way that there are no resources available to accommodate the asynchronous messaging calls required for Orleans to operate. Using this scheduler, at some configurable level under the “amount of work that conceivably be done concurrently”, the Orleans cluster is able to continue asynchronous communication, while also performing these long running tasks being worked through the limited concurrency task scheduler.

Not the package

Aside from the package itself, the repository has a few other projects:

Orleans.SyncWork.Demo.Api
Orleans.SyncWork.Demo.Api.Benchmark
Orleans.SyncWork.Tests

Orleans.SyncWork.Demo.Api

a “sample” implementation of a set of Orleans grains, as well as the cluster hosting. In this project, several long running grains are implemented and registered to the Orleans silo. Those grains are then exposed through api endpoints, or are used as a means of testing some of the logic of SyncWorker<TRequest, TResult> within the Orleans.SyncWork.Tests project.

This project exposes an Orleans Dashboard, as well as SwaggerUI, for keeping an eye on the test cluster and invoking calls to the API respectively.

Orleans.SyncWork.Demo.Api.Benchmark

A Benchmark project has been set up to get an idea of the timing differences between serial execution, Parallel.For, and the parallel execution offered through the SyncWork<TRequest, TResponse> implementation. The latter can be slower then Parallel.For, but faster then the serial execution - though keep in mind this benchmarking testing is done completely locally, where in a real world scenario, you’d be bringing up multiple silo hosts to make up a highly available cluster. At this point it stands to reason that the functionality exposed by this package will far exceed the performance accomplished with a single machine.

An initial running of the benchmark gave the results:

Method	Mean	Error	StdDev
Serial	12.284 s	0.0145 s	0.0135 s
MultipleTasks	12.274 s	0.0073 s	0.0065 s
MultipleParallelTasks	1.723 s	0.0185 s	0.0144 s
OrleansTasks	1.118 s	0.0080 s	0.0074 s

Though keep in mind this can be further improved with an actual cluster of orleans silos, rather than just my one locally.

Orleans.SyncWork.Tests

Unit testing project for the work exposed from the NuGet package. These tests bring up a “TestCluster” which is used for the full duration of the tests against the grains.

One of the tests in particular throws 10k grains onto the cluster at once, all of which are long running (~200ms each) on my machine - more than enough time to overload the cluster if the limited concurrency task scheduler is not working along side the SyncWork base class correctly.

Wrap up

So that’s it. This is quite a bit simpler of an implementation that I ended up originally making for work. The same basic idea is there, but the abstraction is quite different, if it holds up well enough, maybe using this package will be in order!

You can use this package to enable your Orleans cluster to handle your long running sync work, like 10s of thousands of crypto calls!

Resources

Getting Started with NDepend

Russ Hammett — Sat, 20 Nov 2021 15:00:00 +0000

One of the folks over at NDepend reached out to me to provide a license for their product; let’s do some exploring!

I’ve never actually used NDepend, and have only otherwise scratched the surface using other code analysis tools. Perhaps this will be a mis-categorization, but the few I’ve worked with previously include:

FxCop
StyleCop
VS Enterprise code analysis (I forget if this is the actual name, I no longer have an enterprise license)
SonarQube

The idea of tools like NDepend seems pretty straight forward, look through your source code, and identify potential pain points that could use another look. Sounds pretty simple in theory, but I’d imagine the details of such analysis goes way over my head.

Getting started in Visual Studio

Visual Studio is no longer my IDE of choice when working in .net core, but I dusted it off to play around with NDepend. It was quite simple to get installed as a plugin for visual studio, and here’s a helpful video if it’s not otherwise obvious on how to get started: https://www.ndepend.com/docs/getting-started-with-ndepend

I didn’t really know what to expect, and I don’t have any real significant (personal) code bases that I can hook this up to, so for now I’ll run it against my Design Patterns repository to get a baseline.

To kick off the analysis I clicked the circle icon in the bottom right of my visual studio instance, and was quickly presented with a dashboard that looked like this:

Note that this wasn’t actually my first run, I think it was my second or third after having changed a few lines of code in between runs to get a delta in the analysis. Looking at the dashboard it became apparent the analyzer must keep “state” as to what the issues are over time; which is a great sounding feature as it will allow you to track your code “smells” as your code continues to be developed!

The Dashboard

Upon first gazing at the dashboard (pictured above), I was honestly a bit overwhelmed. It was a lot of information, and I didn’t see an immediately obvious “starting point” of what I should be concerned with.

Twice weekly I meet with a few friends over a video/screen share session and we work through plural sight courses together, this week I actually had them on to do a “first impressions” of the NDepend tool; unfortunately my OBS setup was hosed up, and I lost most of the audio. I say this as it is not only my own “first impression” but others as well - namely several who have (slightly) used static analysis tools previously, and others who have not.

Sections

Here I’ll go over what I considered to be the “sections” that were present when going through the NDepend tool.

Dashboard

Sorry for the awful quality of the above image, it was taken from the recording that I couldn’t end up using due to lack of audio.

This seemed to be more or less the “meat and potatoes” of it all. We can see numerous metrics, some of which are pretty self explanatory, but several of which perhaps not so much for someone who is coming from a code base that has no testing; which granted is itself kind of a problem.

I did make a few adjustments to the code from the previous time I ran analysis, I am a big fan of tracking the code quality over time - so this is a huge benefit in my book. It was not immediately obvious to me how this information is tracked over time in a “multi developer project” situation. I am under the assumption that some of the files that were added to the project itself would need to be added to source control in order to track the information over time, but I’m not really a big fan of that (assuming my assumption is even correct). NDepend can be installed in some build systems, which I would guess would keep the analysis separate from the source, but tied to that repository in some manner. I believe I saw integrations for at TFS and TeamCity, I wonder if it could also be integrated with something like TravisCI or GitHub actions.

The sections:

Lines of code - pretty obvious what this metric is, it shows the total number of lines of code in your analyzed solution, including a delta from the last time of analysis (or when comparing to a specific baseline).
Debt - based on what I (think I) know about what technical debt, this metric would go up or down based on a potential variety of factors, namely method complexity and code coverage. This was one that we all had questions around.
Quality Gates, Rules, Issues - None of us were sure what separates these issues from one another, but they seemed to be driven in some manner by the “Queries and Rules” that are enabled within a separate section, but that was another point of confusion coming to this for the “first time”. The gist of these sections seemed to be based around a configurable set of rules that you can (or not) tailor to your project. On the first run through it seemed like we were getting some false positives around use of System.Random being used in a “non security setting”. Seems that rules can be enabled or disabled on a rule level, or by utilizing attributes to ignore specific NDepend rules for a particular member within your code.
Types - a breakdown of the various assemblies, namespaces, files, etc. that are a part of your codebase. It would be interesting to see how this changes over time within a code base.
Coverage - I did not have any coverage in this particular project, and it was definitely asked about by the folks I was doing the walk through with. The way I assume it works it “Code Coverage” as a function of unit testing and exhausting code branches. A very useful metric to have, as long as you have meaningful tests and asserts.
Method complexity - I’d imagine this is a similar metric to the Cyclomatic Complexity that is a part of the enterprise Visual Studio analysis tool. The more complex your methods are, the harder your code is to grok and/or maintain. This metric likely plays a large part in the “Debt” metric mentioned above.

Overall I found the dashboard pretty useful, if not a bit overwhelming. One thing I think the dashboard could use that would benefit the users of NDepend greatly, especially when they’re just starting out, is a little “i” popup per metric:

The popup could at a minimum describe the metric, but probably better, still link to the NDepend documentation on what exactly the metric means, and what goes into its calculation.

Queries and Rules Explorer

I was not really sure where to start here. When poking around with checking and unchecking things within this window, it seemed to impact what is “in scope” of the analysis. It seems like you can probably also define your own rules to look for in your code base, but we did not get into that in our first excursion into NDepend.

Queries and Rules Edit

When clicking on an action item within the dashboard be it “good or bad”, this pane was updated. In the case of failed rule checks, you’d be given a means of fixing it, with navigation to the code block that failed the check. This section was quite useful for helping get more “green lights” on the dashboard, though it was quite cramped in the tab it showed up in, though I understand why it showed up there, so you could view the dashboard and/or problem code while keeping the failed rules list visible as well.

Review

Overall I find the information provided through NDepend to be quite useful, having a tool like NDepend running as a part of your build process, even automatically rejecting PRs if certain quality gates aren’t met sounds fantastic.

I did have a few concerns about “what exactly I’m looking at” which could be remedied via “?” or “i” links on the (granted already quite busy) interface; but I think overall would be a welcome change, especially for first time users (perhaps a feature toggle?).

How does this static analysis compare with other, now free options like Roslyn analyzers, or other tools like sonar, or perhaps even ReSharper? I would imagine there’s some overlap, but that’s not necessarily a bad thing if the multitude of tools bring something unique to the table. The analysis as compared to other baselines I found very nice, though I’m curious how it works in a “multi developer” setting, and with source control and build systems.

I look forward to actually implementing some tests in my analyzed project, and/or putting the tool up against something “more real” than my Patterns repository. Overall it seems like NDepend is a very powerful tool!

References

dotnet centralized package versioning

Russ Hammett — Fri, 08 Oct 2021 14:00:00 +0000

In a previous post I talked about using variable for keeping NuGet packages across multiple projects; now there is a (potentially) better way.

Overview

In the previous post, it was demonstrated how to utilize “repository wide” variables located in a Directory.Build.props folder. With these variables defined, you could use the variable, rather than a literal value, in your csproj to keep dependencies throughout the repository on the same version.

There is now a feature (in review) at https://github.com/NuGet/Home/wiki/Centrally-managing-NuGet-package-versions which allows for doing the same thing, just in a more elegant way, at least in my opinion.

Setting it up

The in review feature relies on a new “repository wide” like file called “Directory.Packages.props”. Using https://github.com/Kritner-Blogs/DirectoryBuildProps/releases/tag/v1.1 as a starting point, we’ll introduce the centralized package versioning.

Update Directory.Build.props

First, we’ll update the “Directory.Build.props” file to contain the following:

<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>

This tells visual studio, rider, the dotnet cli, etc., that the current folder, as well as its children (so repository wide if used at the root of the repo) that central package versioning is used.

Create Directory.Packages.props

Within the same folder as the “Directory.Build.props”, we’ll be creating a new file “Directory.Packages.props”. This file is used to specify the NuGet package version used for all packages used across the repository. Continuing to use the “Kritner.SolarProjection”, we’ll set up our new “Directory.Packages.props” file to specify the version.

<Project>
    <ItemGroup>
        <PackageVersion Include="Kritner.SolarProjection" Version="1.0.2" />
    </ItemGroup>
</Project>

In the above, we’re just defining the package as well as a version that should be used when that package is listed within any csproj file at the same level or as a child of the directory where the “Directory.Build.props” file is located.

Note that the above file does not include the package automatically in csproj files, it only defines the version to use when the package is listed in the csproj file.

Update csproj files

Now that central package versioning is being used in our repository, we’ll need to make updates to our csproj file to make use of it; namely removing the version declaration from our csproj files.

Currently, we have the following in our csproj file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Kritner.SolarProjection" Version="$(NuGet-Kritner-SolarProjection)" />
  </ItemGroup>

</Project>

In the above you’ll note that the old “variable style” version is being used, now with central package management, we don’t even need that:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Kritner.SolarProjection" />
  </ItemGroup>

</Project>

Additional notes

In the previous post we learned how to utilize variables to control our nuget package versions used across the repository, where desired. In this post we explored how to define a package version in one place, and for that same package version to be used across the whole repository.

There are a few other things to note when using central package versioning however:

If a package version is defined in the csproj while central package versioning is turned on, you will be a compile error
- This is done (I believe) to help force the developer “into the pit of success”, making them aware of the fact that the package version is either already defined on a central level and you don’t need to specify the version, or that the package version needs to be defined in the “Directory.Packages.props” file.
Related to the above point, you are still able to override the package version within a csproj by making use of the “OverrideVersion” property (as opposed to “Version” property) within the csproj file, as so:
You can still make use of variables within your props files, as demonstrated in the above image. I like using these variables for defining a “package version” for a suite of packages, where they’re all released with a similar version number all at the same time, even one something didn’t necessarily change for each package in the suite. Examples of where this can be useful include the aspnetcore repository itself, as well as Microsoft Orleans - define the version of the package suite you want to use as a variable, then in your “Build.Packages.props” file, rely on that variable for each package used, rather than defining the literal package version for each package.

Resources

How to build a "tamperproof cookie"

Russ Hammett — Tue, 17 Aug 2021 00:00:00 +0000

All right! It’s been a minute! Tamperproof cookies, I needed one, it’s pretty simple after thinking it through.

I feel like I always say it’s been a while since my last post… but this time it has in fact been a while.

The problem

So… working on a multi factor authentication setup for a freelance client. Have the multi factor one time password all set up, but now I need a way to implement a “remember me for 30 days” feature. That should be relatively simple, right? Can’t we just use cookies?

Not so fast there past me prior to thinking it through! Cookies can easily be modified by the client, because they’re stored on the client local storage and aren’t signed with a digital signature!

Here’s some context into the situation:

Basically, I needed a way to store local to the user a secure (tamper proof) means of indicating the user has been dual factor authenticated for 30 days. The simplest way to do this would seemingly be to store a cookie with a value of the expiration date and user name. However, this first idea I had falls apart quickly, since a user could just change the cookie value (either expiration date and/or user name) and be able to get past the second factor. The second factor one time password (OTP) entry should only be presented to the user if the cookie is valid, for the specific user, and is not yet expired (30 days from the cookie creation).

Outline of Requirements

There are a few things that are needed for a client side solution:

Something stored client side that can be securely used as a means of indicating the user does not need to enter a OTP
- Should have some sort of unique attribution to the user - either through a user ID, email address, or something similar
- Should have an expiration date that is taken into account when checking the validity of the thing
- Should not validate successfully in instances where the user information or the expiration date is changed

The solution

Once documenting the above requirements coupled with the above screenshot, it became pretty clear to me that some sort of cryptography could be used to ensure the tamperproof part of the thing being stored on the client side - in this case a cookie.

HMAC - Hash-base Message Authentication Code - is a cryptographic operation that utilizes a “key” and “message” to produce a “mac” which is (more or less) the combination of the two pieces. The general hmac signature looks something like this:

digest = hmac(key, message)

The key portion is secret between one or more parties - in my case one party, and the message is data that is to be validated as genuine. In my case, the key will be kept secret from the user, as that key is what verifies the message is what we put into the cookie.

What’s a good key to use you may be asking yourself? Well, it’s not the user name, though it’s something similar. The user already knows their own username, so it’s not a good candidate for a key. There is however, their password, specifically their password hash. The user would not know the hash of their password that is stored in the remote system, so it is a good potential candidate for a key.

The user’s password is computed basically like this:

password_digest = H(salt||plaintext_password, work_factor)

Where password_digest is the output of running salt concatenated with plaintext_password through a one way function H, work_factor times. The password_digest is not something the user would know, as their salt changes with each password change, nor do they know the underlying one way function applied to their plaintext_password.

So, now we have a key candidate, what about the actual message? The message is thankfully very straightforward, and we can use the datetime of the OTP cookie expiration. The full HMAC call will end up being: cookie_hash_content = hmac(H(password_hash), expiration_date_string).

This entire cookie_hash_content can be stored in the body of the cookie, though we will also need to stored the expiration_date_string within the cookie as well, so that the server side can reconstruct the inputs into the hmac call, and verify they match.

Thankfully, this is easy enough to do with a pipe (or other) delimited set of fields within the cookie. I will be using this format:

{expiration_date_string}|{cookie_hash_content}

Validating the above cookie at authentication time is as simple as:

Try to split the cookie value into two pieces, if less than two pieces exist, return false
Try to parse the first element in the split as a date, if fails, return false
Ensure the parsed date is after the current datetime, if it is not, return false
Validate the cookie_hash_content by comparing the cookie’s second part of the split, to a newly generated hmac(H(password_hash), expiration_date_string)
- if they don’t match, return false
- if they match, return true

The above pseudocode also has the added benefit if the user changes their password, they will immediately fail the OTP cookie check, as the mac using their old password hash will no longer match the newly generated/checked password hash upon login. This in a few ways makes things just a bit more secure.

Here is a quick run through of the HMAC (and a few unit tests) using fake expireDates and passwordHashes: https://replit.com/@Kritner/TamperProofCookies#main.py

Final thoughts

This works because the key is not known (or knowable) to the end user
The salts, while not necessarily secret, are not known to the end user, so they would have no chance of being able to recreate the same conditions to come up with the key being utilized
We verify the cookie_hash_content by re-computing the value based on the values stored in the cookie itself, along with the additional data of key that isn’t in the cookie, and can’t be known by the user, so there’s no chance the user would be able to come up with “the right information” that could pass the verify step.

Resources:

Design Patterns: Decorator

Russ Hammett — Fri, 09 Apr 2021 15:00:00 +0000

The decorator pattern is a structural design pattern that can be used to add functionality to classes without modifying the original class or its interface.

The Decorator Pattern

From Wikipedia:

In object-oriented programming, the decorator pattern is a design pattern that allows behavior to be added to an individual object, dynamically, without affecting the behavior of other objects from the same class. The decorator pattern is often useful for adhering to the Single Responsibility Principle, as it allows functionality to be divided between classes with unique areas of concern. The decorator pattern is structurally nearly identical to the chain of responsibility pattern, the difference being that in a chain of responsibility, exactly one of the classes handles the request, while for the decorator, all classes handle the request.

“Prefer composition over inheritance” might be something you’ve heard of before, and I feel like the decorator pattern is basically the epitome of this saying. Decorators are themselves implementations of an abstraction, that also depend on the abstraction itself. This allows for “composable” pieces of behavior in that you would likely have a single “main” implementation of an abstraction, then separate “decorator” implementations that build on top of it.

I’m not sure if that made sense, so hopefully an example will help clear things up.

Implementing the decorator pattern

The abstraction

For our abstraction, we’re going to create a service that retrieves the weather for us:

public interface IWeatherGettinator
{
    Task<Weather> GetWeather();
}

public class Weather
{
    public Guid Id { get; }
    public int FreedomUnits { get; set; }
    public bool ItGonRain { get; set; }

    public Weather()
    {
        Id = Guid.NewGuid();
    }

    public override string ToString()
    {
        var sb = new StringBuilder()
            .AppendLine("The current weather:")
            .AppendLine($"\tId: {Id}")
            .AppendLine($"\tTemperature (in freedom units): {FreedomUnits}")
            .AppendLine($"\tIt gon rain? {ItGonRain}");

        return sb.ToString();
    }
}

In the above, we have an interface IWeatherGettinator that has a single method, which takes no arguments, and returns the weather; simple enough.

The base implementation

For this example, we’re going to pretend that to get the weather is an expensive operation, just because it makes one of the decorators of IWeatherGettinator more interesting and easier to see the point of it all (IMO).

So for the implementation of the IWeatherGettinator:

public class WeatherGettinator : IWeatherGettinator
{

    private readonly Random _random;

    public WeatherGettinator(Random random)
    {
        _random = random;
    }

    /// <summary>
    /// Gets the weather, mimics a long running task.
    /// </summary>
    /// <returns><see cref="Task"/> of <see cref="Weather"/></returns>
    public async Task<Weather> GetWeather()
    {
        await Task.Delay(5000);

        return new Weather()
        {
            FreedomUnits = _random.Next(0, 100),
            ItGonRain = _random.Next(0, 1) == 1
        };
    }
}

On getting the weather (which is a ~5 second operation), we get a random temperature and a value that indicates if it’s going to rain or not. Each time GetWeather is invoked, it takes another 5 seconds to retrieve a random “weather” instance.

The above implementation should look something like this when you ToString the retrieved Weather via a GetWeather invoke:

Our first decorator

Getting the weather seems to take a pretty long time! Perhaps we can introduce our first decorator to get some timing information on the implementation of IWeatherGettinator:

public class StopWatchDecoratorWeather : IWeatherGettinator
{
    private readonly IWeatherGettinator _weatherGettinator;

    public StopWatchDecoratorWeather(IWeatherGettinator weatherGettinator)
    {
        _weatherGettinator = weatherGettinator;
    }

    public async Task<Weather> GetWeather()
    {
        Stopwatch sw = Stopwatch.StartNew();
        var weather = await _weatherGettinator.GetWeather();
        sw.Stop();

        Console.WriteLine($"Decorated IWeatherGettinator ran for {sw.ElapsedMilliseconds}ms");
        return weather;
    }
}

This might look a little strange, so let’s go over it a bit. The StopWatchDecoratorWeather is an implementation of IWeatherGettinator that also depends on an implementation of IWeatherGettinator. You can see that the method implementation GetWeather starts a stop watch, calls the injected implementation of IWeatherGettinator, stops the stopwatch, and writes to the console the length of time the operation took.

Now you probably wouldn’t actually write a decorator exactly like the above, but you could do something like it, especially if you were to make use of a logging framework. Logging the above could make more sense, especially if the invoke took more than a certain amount of time.

How does the above get used? Well, you could “compose” your object like so:

IWeatherGettinator weatherGettinator = new StopWatchDecoratorWeather(new WeatherGettinator(new Random()));

You would actually want to be constructing these objects via an IOC container or something similar, this was to just get the point across… also there is some complexity to registering decorated objects with at least the .net core built in IOC container; though you could also use a factory to accomplish it more easily.

What does the above construction mean? Well, we’re instantiating a new instance of a StopWatchDecoratorWeather which itself depends on a IWeatherGettinator instance, in this case we’re passing in a concrete implementation of WeatherGettinator.

Running the IWeatherGettinator while making use of the StopWatchDecoratorWeather would look like this:

The issue with inheritance

Inheritance has an issue; an issue that can be solved through composition. I’ve mentioned this several times now, but it’s kind of hard (at least for me) to convey what I mean.

Say we wanted to introduce another piece of functionality; in this post we’re going to be doing it through the use of decorators, but just imagine for a moment that we weren’t.

In c# a class can only ever extend a single other class. If we ever wanted to “mix and match” behaviors of an abstraction that relied on inheritance, that can be extremely difficult to do. If we have an interface IFoo, with an implementation Foo, then had separate implementations A and B, both of which extended Foo, how would we go about introducing yet another implementation C, that needed some of its own unique characteristics, as well as the additional functionality provided by both A and B? This would be challenging with an inheritance scenario, but is really trivial when making use of decorators.

To demonstrate that, let’s introduce another decorator to our IWeatherGettinator.

The second decorator

We’ve done some testing with our StopWatchDecoratorWeather and determined that we could save a lot of time getting the weather if we introduced some caching. Thankfully, introducing caching via a decorator is very simple, and should look pretty similar to our first decorator!

public class CachedWeatherGettinator : IWeatherGettinator
{
    private readonly IWeatherGettinator _weatherGettinator;

    private Weather _cachedWeather;
    private DateTime _cachedOn;

    public CachedWeatherGettinator(IWeatherGettinator weatherGettinator)
    {
        _weatherGettinator = weatherGettinator;
    }

    public async Task<Weather> GetWeather()
    {
        var now = DateTime.Now; // this should be injected for testability, but we're demonstrating a decorator so...
        var timeDiffSecondsSinceCachedValue = now.Subtract(_cachedOn).TotalSeconds;

        if (_cachedWeather == null || timeDiffSecondsSinceCachedValue > 30)
        {
            Console.WriteLine("Cached value does not exist or is expired, get a new one.");
            _cachedOn = now;
            _cachedWeather = await _weatherGettinator.GetWeather();
        }
        else
        {
            Console.WriteLine("Using cached value...");
        }

        return _cachedWeather;
    }
}

In our new CachedWeatherGettinator, we’re making use of some instance state to return the previously retrieved weather if it’s less than 30 seconds since it was retrieved. You’ll notice this implementation, like our first decorator, both implements and depends on IWeatherGettinator.

We can now try out our new decorator like this:

IWeatherGettinator weatherGettinator = new CachedWeatherGettinator(new WeatherGettinator(new Random()));

And you’ll see that the “first” call to the IWeatherGettinator will take the 5 seconds, but another call made immediately after will return much faster.

But even more interesting that that, is we can make use of multiple decorators for the same abstraction, the construction of which is considered composition.

What could that look like?

IWeatherGettinator weatherGettinator = new StopWatchDecoratorWeather(new CachedWeatherGettinator(new WeatherGettinator(random)));

We’re “composing” our object by decorating our base WeatherGettinator with multiple decorators! We have decorated the base object with both a StopWatch as well as Caching layer, mostly to demonstrate that the caching layer is in fact working. Let’s take a look!

Reasons to use this pattern

Hopefully it should be pretty clear after going through the post why this pattern can be quite powerful, but just to reiterate:

Adding additional functionality to an abstraction without changing the abstraction or its concretions.
Keeping concerns of caching or logging separate from the base implementation. These are taken care of via decorators, allowing your core “business logic” to stay pure, and uncaring about the functionality provided by the decorators.

References

Prettifying HealthChecks

Russ Hammett — Fri, 04 Dec 2020 19:12:11 +0000

Previously I wrote about creating Health Checks for Microsoft Orleans, but the JSON response was too minimal. In this post we’ll see about prettifying that output!

In the previous post we learned a bit about health checks, how to create them, and view their “health” from the perspective of Microsoft Orleans. The end result was a single word response of “Healthy”, “Degraded”, or “Unhealthy”; not very exciting stuff.

In this post, I’d like to quickly go over how you’d go about not only reporting on the “overarching status”, but giving details on the individual health checks that make up that overarching status.

(Note: I did have an issue out there with a hacktoberfest label, that I did get a PR for, but I wanted to go a slightly different route in the end, though I did have it integrated into master for some time.)

The Health check documentation does go into some detail about how to accomplish this health check prettifying, but I’m not a huge fan of “manually” writing out JSON; instead I opted for an anonymous object.

Prettifying Property Potentials

A few new things I want to report on from the response to the health check GET endpoint:

Health Check Name
Health Check Description
Individual Health Check Status
Some additional information specific to the health check that describes what makes the health check return “Degraded” or “Unhealthy”

Luckily all of this information can be made available to us via the HealthReport generated as a part of the health check.

Health Check Response Writer

We’re going to introduce a new method that writes a custom response for our health check endpoint. From startup, we’ll want to provide a custom ResponseWriter within MapHealthChecks:

app.UseEndpoints(endpoints =>
    {
        endpoints.MapHealthChecks(
                "/health",
                new HealthCheckOptions
                {
                    AllowCachingResponses = false,
                    ResponseWriter = HealthCheckResponseWriter.WriteResponse
                })
            .WithMetadata(new AllowAnonymousAttribute());
        endpoints.MapControllers();
    });

Where the referenced HealthCheckResponseWriter is a new static class we’ll be introducing next.

the ResponseWriter expects a method with the following signature:

You’ll notice above the method receives an HttpContext as well as HealthReport. This HealthReport will make available to us several pieces of data that we can report on, specific to each individual health check.

As for our actual response writer implementation, here is the original that was merged into master from L-Dogg‘s PR:

private static Task WriteResponse(HttpContext context, HealthReport result)
{
    context.Response.ContentType = "application/json; charset=utf-8";

    var options = new JsonWriterOptions
    {
        Indented = true
    };

    using var stream = new MemoryStream();
    using (var writer = new Utf8JsonWriter(stream, options))
    {
        writer.WriteStartObject();
        writer.WriteString("status", result.Status.ToString());
        writer.WriteStartObject("results");
        foreach (var entry in result.Entries)
        {
            writer.WriteStartObject(entry.Key);
            writer.WriteString("status", entry.Value.Status.ToString());
            writer.WriteString("description", entry.Value.Description);
            writer.WriteStartObject("data");
            foreach (var item in entry.Value.Data)
            {
                writer.WritePropertyName(item.Key);
                JsonSerializer.Serialize(
                    writer, item.Value, item.Value?.GetType() ??
                                        typeof(object));
            }
            writer.WriteEndObject();
            writer.WriteEndObject();
        }
        writer.WriteEndObject();
        writer.WriteEndObject();
    }

    var json = Encoding.UTF8.GetString(stream.ToArray());

    return context.Response.WriteAsync(json);
}

The above definitely works, but I’m not huge on writing the json “manually” (if that makes sense). I wanted to write another blog post on this anyway, as I already had a branch going (and didn’t actually expect a PR :O), so here’s my solution:

internal static class HealthCheckResponseWriter
{
    public static Task WriteResponse(HttpContext context, HealthReport healthReport)
    {
        context.Response.ContentType = "application/json; charset=utf-8";

        var result = JsonConvert.SerializeObject(new
        {
            status = healthReport.Status.ToString(),
            details = healthReport.Entries.Select(e => new
            {
                key = e.Key,
                description = e.Value.Description,
                status = e.Value.Status.ToString(),
                data = e.Value.Data
            })
        }, Formatting.Indented);

        return context.Response.WriteAsync(result);
    }
}

I find it a bit more concise working with the anonymous object.

Health Check updates

We’re not currently generating “data” information from the health checks that the HealthCheckResponseWriter would be able to make use of, so let’s take a look at what we could do there.

My intention for the “data” property of the anonymous object is to describe what would make the specific health check return a “Degraded” or “Unhealthy”, anything aside from those two statuses can be assumed to be “Healthy”.

If you recall, we already built thresholds into the health checks to represent the degraded and unhealthy statuses, now we’ll just need to provide those available to the health report.

Taking a look at the HealthCheckResult class:

you’ll see that method takes in an optional IReadOnlyDictionary<string, object> data = null, which happens to be the “data” member we made sure to return from our WriteResponse method in the previous section of the post.

We will make use of this IReadonlyDictionary to provide our “threshold” information on a per grain basis. I will be putting this threshold information into both the CPU and Memory grains, but just as an example here’s what one of those will look like:

[StatelessWorker(1)]
public class CpuHealthCheckGrain : Grain, ICpuHealthCheckGrain
{
    private const float UnhealthyThreshold = 90;
    private const float DegradedThreshold = 70;
    private readonly ReadOnlyDictionary<string, object> HealthCheckData = new ReadOnlyDictionary<string, object>(
        new Dictionary<string, object>()
        {
            { "Unhealthy Threshold",  UnhealthyThreshold},
            { "Degraded Threshold",  DegradedThreshold}
        });

    private readonly IHostEnvironmentStatistics _hostEnvironmentStatistics;

    public CpuHealthCheckGrain(IHostEnvironmentStatistics hostEnvironmentStatistics)
    {
        _hostEnvironmentStatistics = hostEnvironmentStatistics;
    }

    public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = new CancellationToken())
    {
        if (_hostEnvironmentStatistics.CpuUsage == null)
        {
            return Task.FromResult(HealthCheckResult.Unhealthy("Could not determine CPU usage.", data: HealthCheckData));
        }

        if (_hostEnvironmentStatistics.CpuUsage > UnhealthyThreshold)
        {
            return Task.FromResult(HealthCheckResult.Unhealthy(
                $"CPU utilization is unhealthy at {_hostEnvironmentStatistics.CpuUsage:0.00}%.", data: HealthCheckData));
        }

        if (_hostEnvironmentStatistics.CpuUsage > DegradedThreshold)
        {
            return Task.FromResult(HealthCheckResult.Degraded(
                $"CPU utilization is degraded at {_hostEnvironmentStatistics.CpuUsage:0.00}%.", data: HealthCheckData));
        }

        return Task.FromResult(HealthCheckResult.Healthy(
            $"CPU utilization is healthy at {_hostEnvironmentStatistics.CpuUsage:0.00}%.", data: HealthCheckData));
    }
}

You should notice in the above that we introduce a ReadOnlyDictionary with the thresholds for degraded and unhealthy, then passed that ReadOnlyDictionary to the data parameter of the static method within HealthCheckResult.

Testing it out

The only thing left to do is test it out! You may have seen the cover image which contained spoilers, but just to wrap things up, here’s what it looks like when hitting the “/health” endpoint after our changes:

References

Microsoft Orleans - Health Checks

Russ Hammett — Fri, 04 Dec 2020 15:00:00 +0000

Health Checks are a means of seeing how a system is running at the time of performing the check. Let’s see how we can apply them to Orleans!

Overview

Health Checks are generally exposed via HTTP endpoints, and when hit (often at a “/hc” or “/health” endpoint) they are able to report of the “health” of the current system.

The health checks, at least in .net land, are comprised of an enum HealthStatus which indicates Health, Degraded, or Unhealthy. The health checks themselves are created by implementing concretions of the IHealthCheck interface.

Any system can include one or more health checks, and what “a health check” means is completely up to you as the implementer. You could as an example have 1 health check that “checks”:

All contained within a “single” health check called “muhSystem” (or whatever), or you could implement the three above “checks” as their own individual health checks; so a single check vs multiple checks, all of which represent “the same thing”. Why would you choose one over the other? Well, going the single check route allows you to check on Foo, Bar, and Bazes health, without “leaking” any information about what’s being checked. This sort of check could be useful if you needed to be careful about revealing information on some of the internal workings of your system.

In the “three separate checks” scenario, it might not matter to you if you leak some information about your system, you want to give your users (or perhaps your watchdog) information on a more detailed level.

Getting set up

We’ll be starting the code section of this post from the v0.58 tag on my OrleansGettingStarted repository.

I did not write a blog post about the changes that were performed in the update to the v0.58 tag, but one of the changes was getting the silo host running under the next UseOrleans extension method. In the newer version of .net core and Orleans, you’re able to host multiple “processes” from the same IHostBuilder. What this allows us to do is host a small API that will serve the health check endpoint through http requests.

First thing we’ll do is add a default web host to our host builder - which after the change will be hosting both our silo host, as well as our api:

hostBuilder.ConfigureWebHostDefaults(builder => { builder.UseStartup<Startup>(); })

(Note there are going to be other varying changes that I may not be specifically calling out, but the end code is here and in the references at the bottom of the post.)

We’ll also introduce a Startup class:

public class Startup
{
    public Startup(IConfiguration configuration)
    {
        Configuration = configuration;
    }

    public IConfiguration Configuration { get; }

    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddControllers();
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }

        app.UseHttpsRedirection();

        app.UseRouting();

        app.UseAuthorization();

        app.UseEndpoints(endpoints =>
        {
            endpoints.MapControllers();
        });
    }
}

The above class should more or less be the “default” Startup class present when creating a new web API project from template.

Health Checks

Basic Health Check

The first health check we’re going to do will be a basic one - in fact that’s what we’ll name it. For this health check, we’ll use an IClusterClient and ensure it can get an instance of a grain, and get a result from that grain. If it can get a result from the grain the health check should return “Healthy”, otherwise “Unhealthy”.

As always, we first need our Orleans grain interface:

public interface IBasicHealthCheckGrain : IHealthCheck, IGrainWithGuidKey { }

The above is quite simple, we’re creating a class that implements both a IHealthCheck and IGrainWithGuidKey. IGrainWithGuidKey should be familiar from some of my other Orleans posts, and the IHealthCheck grain was mentioned earlier in this post, it’s an interface that describes a health check. We’re not adding anything to this interface that isn’t already provided via the IHealthCheck or IGrainWithGuidKey.

Our basic health check grain implementation looks like this:

[StatelessWorker(1)]
public class BasicHealthCheckGrain : Grain, IBasicHealthCheckGrain
{
    public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = new CancellationToken())
    {
        return Task.FromResult(new HealthCheckResult(HealthStatus.Healthy));
    }
}

That’s it! Just return a healthy result. If our actual IHealthCheck implementation is unable to get an instance of this grain and an exception is encountered, the exception handler will return “Unhealthy” for us.

Now that we have a health check grain, we’ll need an actual IHealthCheck implementation that will utilize our newly created “health check grain”. I know we’ll be creating several health checks here, all of which do “a lot of the same thing”, so this seems like the perfect opportunity to introduce an abstract class OrleansHealthCheckBase:

public abstract class OrleansHealthCheckBase : IHealthCheck
{
    protected readonly IClusterClient _client;

    protected OrleansHealthCheckBase(IClusterClient client)
    {
        _client = client;
    }

    /// <summary>
    /// Entry into health check, ensures the client is initialized, if it is not returns a healthy status.
    /// </summary>
    /// <param name="context">The health check context.</param>
    /// <param name="cancellationToken">The cancellation token.</param>
    /// <returns><see cref="Task"/> of <see cref="HealthCheckResult"/></returns>
    public virtual async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = new CancellationToken())
    {
        if (!_client.IsInitialized)
        {
            return HealthCheckResult.Healthy($"{nameof(_client)} not yet initialized.");
        }

        return await CheckHealthGrainAsync(context, cancellationToken);
    }

    /// <summary>
    /// Perform the actual health check work within this implemented method.
    /// </summary>
    /// <param name="context">The health check context.</param>
    /// <param name="cancellationToken">The cancellation token.</param>
    /// <returns><see cref="Task"/> of <see cref="HealthCheckResult"/></returns>
    protected abstract Task<HealthCheckResult> CheckHealthGrainAsync(HealthCheckContext context, CancellationToken cancellationToken);
}

All of our health checks will depend on a connection to the cluster, so the above will take in a IClusterClient, and ensure the cluster is initialized prior to proceeding with the to be implemented actual check.

As I mentioned previously, for our “Basic” health check, we’ll just be checking that we can get an instance of a grain, and return a value. Such a health check will look like:

public class BasicOrleansHealthCheck :  OrleansHealthCheckBase
{
    public BasicOrleansHealthCheck(IClusterClient client) : base(client)
    {

    }

    protected override async Task<HealthCheckResult> CheckHealthGrainAsync(HealthCheckContext context, CancellationToken cancellationToken)
    {
        try
        {
            return await _client.GetGrain<IBasicHealthCheckGrain>(Guid.Empty)
                .CheckHealthAsync(context, cancellationToken);
        }
        catch (Exception e)
        {
            return HealthCheckResult.Unhealthy($"Health check failed.", e);
        }
    }
}

Performance Health Checks

Now that the basic health check is out of the way, we can implement some more meaningful ones. The following health checks require a registered IHostEnvironmentStatistics (which you can find out more about here).

These health checks will be especially useful for gauging the utilization of an Orleans node over time, which would allow you to make decisions on questions such as “should I spin up or down additional nodes for this cluster?”. Answers to such questions, especially if running your Orleans cluster in a k8s environment, are much simpler when you have performance metrics exposed via a health check endpoint, and are making use of a watchdog.

Going to go through these fast, they should be mostly self explanatory but you can view the completed code for anything I don’t specifically cover.

CPU Health Check

public interface ICpuHealthCheckGrain : IHealthCheck, IGrainWithGuidKey
{

}

[StatelessWorker(1)]
public class CpuHealthCheckGrain : Grain, ICpuHealthCheckGrain
{
    private const float UnhealthyThreshold = 90;
    private const float DegradedThreshold = 70;

    private readonly IHostEnvironmentStatistics _hostEnvironmentStatistics;

    public CpuHealthCheckGrain(IHostEnvironmentStatistics hostEnvironmentStatistics)
    {
        _hostEnvironmentStatistics = hostEnvironmentStatistics;
    }

    public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = new CancellationToken())
    {
        if (_hostEnvironmentStatistics.CpuUsage > UnhealthyThreshold)
        {
            return Task.FromResult(HealthCheckResult.Unhealthy(
                $"CPU utilization is unhealthy at {_hostEnvironmentStatistics.CpuUsage}%."));

        }

        if (_hostEnvironmentStatistics.CpuUsage > DegradedThreshold)
        {
            return Task.FromResult(HealthCheckResult.Degraded(
                $"CPU utilization is degraded at {_hostEnvironmentStatistics.CpuUsage}%."));
        }

        return Task.FromResult(HealthCheckResult.Healthy(
            $"CPU utilization is healthy at {_hostEnvironmentStatistics.CpuUsage}%."));
    }
}

New grain interface, new grain implementation for CPU health checking. We’re going to return Unhealthy if above 90% CPU, Degraded if above 70%, Healthy otherwise.

Memory Health Check

Same basic idea for the memory health check, again making use of our registered IHostEnvironmentStatistics:

public interface IMemoryHealthCheckGrain : IHealthCheck, IGrainWithGuidKey
{

}

[StatelessWorker(1)]
public class MemoryHealthCheckGrain : Grain, IMemoryHealthCheckGrain
{
    private const float UnhealthyThreshold = 95;
    private const float DegradedThreshold = 90;

    private readonly IHostEnvironmentStatistics _hostEnvironmentStatistics;

    public MemoryHealthCheckGrain(IHostEnvironmentStatistics hostEnvironmentStatistics)
    {
        _hostEnvironmentStatistics = hostEnvironmentStatistics;
    }

    public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = new CancellationToken())
    {
        if (_hostEnvironmentStatistics?.AvailableMemory == null || _hostEnvironmentStatistics?.TotalPhysicalMemory == null)
        {
            return Task.FromResult(HealthCheckResult.Unhealthy("Could not determine memory calculation."));
        }

        if (_hostEnvironmentStatistics?.AvailableMemory == 0 && _hostEnvironmentStatistics?.AvailableMemory == 0)
        {
            return Task.FromResult(HealthCheckResult.Unhealthy("Could not determine memory calculation."));
        }

        var memoryUsed = 100 - ((float)_hostEnvironmentStatistics.AvailableMemory / (float)_hostEnvironmentStatistics.TotalPhysicalMemory * 100);

        if (memoryUsed > UnhealthyThreshold)
        {
            return Task.FromResult(HealthCheckResult.Unhealthy(
                $"Memory utilization is unhealthy at {memoryUsed:0.00}%."));

        }

        if (memoryUsed > DegradedThreshold)
        {
            return Task.FromResult(HealthCheckResult.Degraded(
                $"Memory utilization is degraded at {memoryUsed:0.00}%."));
        }

        return Task.FromResult(HealthCheckResult.Healthy(
            $"Memory utilization is healthy at {memoryUsed:0.00}%."));
    }
}

In this case, I’m going a slightly different route and returning “Unhealthy” if the memory information cannot be determined, this should probably be consistently done between this and the CPU health check, but I wanted to show how you as the implementer is able to choose what “Healthy” vs “Unhealthy” means. For this memory health check, we’re unhealthy if above 95% memory utilization, degraded if above 90, healthy otherwise.

Wiring up the health checks

Now that we have our health check grains, we’ll introduce new IHealthChecks very similar to the BasicOrleansHealthCheck which extended OrleansHealthCheckBase.

public class CpuOrleansHealthCheck : OrleansHealthCheckBase
{
    public CpuOrleansHealthCheck(IClusterClient client) : base(client)
    {
    }

    protected override async Task<HealthCheckResult> CheckHealthGrainAsync(HealthCheckContext context, CancellationToken cancellationToken)
    {
        try
        {
            return await _client.GetGrain<ICpuHealthCheckGrain>(Guid.Empty)
                .CheckHealthAsync(context, cancellationToken);
        }
        catch (Exception e)
        {
            return HealthCheckResult.Unhealthy($"Health check failed.", e);
        }
    }
}

public class MemoryOrleansHealthCheck : OrleansHealthCheckBase
{
    public MemoryOrleansHealthCheck(IClusterClient client) : base(client)
    {
    }

    protected override async Task<HealthCheckResult> CheckHealthGrainAsync(HealthCheckContext context, CancellationToken cancellationToken)
    {
        try
        {
            return await _client.GetGrain<IMemoryHealthCheckGrain>(Guid.Empty)
                .CheckHealthAsync(context, cancellationToken);
        }
        catch (Exception e)
        {
            return HealthCheckResult.Unhealthy($"Health check failed.", e);
        }
    }
}

Now we need to wire all of these health checks up to our “/health” endpoint within our webhost. Luckily, this is pretty easy. The earlier Startup:

Becomes:

public class Startup
{
    public Startup(IConfiguration configuration)
    {
        Configuration = configuration;
    }

    public IConfiguration Configuration { get; }

    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddControllers();
        services.AddHealthChecks()
            .AddCheck<BasicOrleansHealthCheck>("basicOrleans")
            .AddCheck<CpuOrleansHealthCheck>("cpuOrleans")
            .AddCheck<MemoryOrleansHealthCheck>("memoryOrleans");
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }

        app.UseHttpsRedirection();

        app.UseRouting();

        app.UseAuthorization();

        app.UseEndpoints(endpoints =>
        {
            endpoints.MapHealthChecks("/health").WithMetadata(new AllowAnonymousAttribute());
            endpoints.MapControllers();
        });
    }
}

The difference being we’re adding the health checks (and giving them names) within ConfigureServices, and mapping the health checks to a “/health” endpoint within Configure.

Testing it out

Let’s fire up the silo, and test this thing out!

Well, that was pretty anticlimactic… we’ll have to see about prettying up that health check response, hopefully in another post that I’ll totally write real soon!

References

Self Notes - React Router Getting Started

Russ Hammett — Tue, 08 Sep 2020 13:30:33 +0000

My notes about trying to get started with React.

I’m mostly a backend-dev. The bit of front-end dev I’ve done was on web-forms with asp.net. Getting started in this whole ecosystem seems pretty daunting coming from a background where I have access to strongly typed, easily testable code.

This is probably rehashing things I’ve stated before, but with front end I feel like it’s more of an “opinion” on if something is implemented in an “good” way.

Anyway, I’m trying to mess about with React. I want to hopefully at some point, replace the mostly placeholder site I have at kritner.com, with something a bit more substantial.

The first thing I thought would make sense to get started with is a general shared layout for pages.

I’ll be making use of Code Sandbox for my experimentation. Code sandbox, if you’ve never used it, allows for a playground (or sandbox…) for putting together sample apps with varying tech stacks like React, Vue, with or without TypeScript, etc.

Stubbed Components

Starting out with the template React Typescript. First thing I’ll do is introduce a few page stubs that my navigation will link to:

About.tsx

import React from "react";

export default function About() {
  return <div>About me</div>;
}

Contact.tsx

import React from "react";

export default function Contact() {
  return <div>Contact</div>;
}

Home.tsx

import React from "react";

export default function Home() {
  return <div>Home</div>;
}

In the above I’m utilizing function components, rather than class components https://reactjs.org/docs/components-and-props.html#function-and-class-components as they seem to be preferred unless otherwise needed; they look a lot simpler as well.

Routing

Url Routing

Not sure if this is the best place to introduce routing (I’d guess not), but not knowing a whole lot about the conventions/standards of React yet, this is what I’m going with. I’m going to update the App.tsx to include routing.

The starting code for App is:

export default function App() {
  return (
    <div className="App">
      <h1>Hello CodeSandbox</h1>
      <h2>Start editing to see some magic happen!</h2>
    </div>
  );
}

For routing, I’ll pull in the “react-router-dom” package into code sandbox:

We’ll now add routing to our App component, which supports URL routing. first our imports (there are a few extra that we’ll be getting to):

import { BrowserRouter, Route, Switch, Link } from "react-router-dom";

// Also importing the components that will be routed to
import Home from "./pages/Home";
import Contact from "./pages/Contact";
import About from "./pages/About";

I’ll be making use of BrowserRouter, Switch, and Route for the URL routing. The App component will need to be updated to route to our earlier created “Home”, “About”, and “Contact” components.

export default function App() {
  return (
    <BrowserRouter>
      <Switch>
        <Route path="/" component={Home} exact />
        <Route path="/contact" component={Contact} />
        <Route path="/about" component={About} />
      </Switch>
    </BrowserRouter>
  );
}

The above should look pretty straightforward, though the “exact” on the root route seems to be necessary, at least in the order I have the routes defined. It seems that “/contact” matches first on “/“, and would show the Home component rather than Contact, unless the exact is specified (or if the “/“ is the last route defined).

Now the application should be responsive to the routes defined above:

Navigation Routing

Next to handle link based navigation, we’ll make use of Link.

<div>
    <Link to="/">Home </Link>
    <Link to="/about">About </Link>
    <Link to="/contact">Contact </Link>
</div>

The full App.jsx now looks like:

import * as React from "react";
import { BrowserRouter, Route, Switch, Link } from "react-router-dom";

import Home from "./pages/Home";
import Contact from "./pages/Contact";
import About from "./pages/About";

import "./styles.css";

export default function App() {
  return (
    <BrowserRouter>
      <div>
        <Link to="/">Home </Link>
        <Link to="/about">About </Link>
        <Link to="/contact">Contact </Link>
      </div>
      <Switch>
        <Route path="/" component={Home} exact />
        <Route path="/contact" component={Contact} />
        <Route path="/about" component={About} />
      </Switch>
    </BrowserRouter>
  );
}

I tried playing around with some of the routing portions being their own component, but was getting errors around trying to include things outside of a router. I’m guessing this can still refactored a bit, and am curious how others do it!

Obviously the above has no styling or anything of that nature, which is something else I need to learn more about ^^

References:

Setting up mTLS and Kestrel (cont.)

Russ Hammett — Mon, 17 Aug 2020 00:00:00 +0000

In the last post we started talking about mTLS. In the post I pointed out that the client cert’s signing CA was not verified, let’s fix that!

The problem

The main thing accomplished in the previous post was getting a CA up and running with a client certificate signed by that CA. We updated our web api to require client certificates, and successfully connected to the application using our new client certificate.

The flaw in the previous post was though we were validating a client certificate was present, we weren’t actually validating that the client certificate was signed by the CA we created. If the client certificate were valid for some other reason (like if it were signed by another trusted root CA on the system, like an actual internet CA) it would still “get in”.

Steps to Remedy

Set up for depicting the problem
Demonstrate the problem
Solve the problem with code

Set up for depicting the problem

So first things first, we need to show that this is in fact a problem. I’ve already described the problem, but how can we go about proving it?

Easiest thing IMO, is to just create another CA and client cert with the new CA. The web api should accept both the original “client” cert, as well as the “badClient” cert.

I’m going to do all the steps from the previous post for setting up the CA, including trusting our “badCa”. The reason this is being done has already been covered above, but just to reiterate, we want to show that any valid cert is currently getting in; where we want only valid certs from our intended CA to get in.

So now we should have the following:

A “good” CA - the one we want to ensure checked out client certs
A “good” client cert - the one that should be accepted via our app
A “bad” CA - a valid CA, but not one that should be allowed to sign certificates that can get into our app
A “bad” client cert - signed by the “bad” ca, should not (but currently will be able to) get into our app

It’ll look something like this if you’re inspecting the certificates:

Demonstrate the problem

Now all there is to do is hit our web api with both the “good” cert and the “bad” cert, and confirm we are in fact able to get output from both.

Just like previously, let’s make sure the web api is running with a dotnet run from the “Kritner.Mtls” project.

Then hit the application first w/o a cert:

curl --insecure -v https://localhost:5001/weatherForecast

With the “good” cert:

curl --insecure -v --key client.key --cert client.crt https://localhost:5001/weatherForecast

With the “bad” cert:

curl --insecure -v --key badClient.key --cert badClient.crt https://localhost:5001/weatherForecast

You can see, as expected, the request w/o a client cert is rejected, and the requests with both the “good” and “bad” client certs get through. Now, we need to figure out how to go about restricting the app to only accept client certificates signed by our “good” CA

Solve the problem with code

So first things first, we need to identify something about what makes “good” client certificate “good”, and what makes “bad” client certificate “bad”. If you inspect the certificates on your system, you’ll see there is an “Authority Key Identifier” as an attribute. This “Authority Key Identifier” on the client certificate matches the “Authority Key Identifier” and/or “Subject Key Identifier” on the CA that signed the certificate:

Apologies about all different CA labels and whatnot if you’ve noticed them in the screenshots, I’m switching around computers like a madlad!

In the above, you’ll be able to see that the “good” cert “belongs” to the “good” CA, and the “bad” cert “belongs” to the “bad” CA - this is the information we need! Now we just need a way to get to the information in code.

Introduce an additional service to validate the CA

Note the starting point of the code I’m working with is https://github.com/Kritner-Blogs/Kritner.Mtls/releases/tag/v0.9.1

Let’s review the current code within Startup I even left a little note for myself and others from the last post:

Implementation Stub

OnCertificateValidated = context =>
{
    var logger = context.HttpContext.RequestServices.GetService<ILogger<Startup>>();

    // You should implement a service that confirms the certificate passed in
    // was signed by the root CA.

    // Otherwise, a certificate that is valid to one of the other trusted CAs on the webserver,
    // would be valid in this case as well.

    logger.LogInformation("You did it my dudes!");

    return Task.CompletedTask;
}

Let’s introduce a service into this section of code, its abstraction will look like this:

public interface ICertificateAuthorityValidator
{
    bool IsValid(X509Certificate2 clientCert);
}

In the above abstraction, we’re taking in a client certificate, and returning whether or not it’s valid (obviously). the context within OnCertificateValidated has access to the ClientCertificate and it’s already in the form of X509Certificate2.

Let’s stub out our implementation:

public class CertificateAuthorityValidator : ICertificateAuthorityValidator
{
    private readonly ILogger<CertificateAuthorityValidator> _logger;

    public CertificateAuthorityValidator(ILogger<CertificateAuthorityValidator> logger)
    {
        _logger = logger;
    }

    public bool IsValid(X509Certificate2 clientCert)
    {
        _logger.LogInformation($"Validating certificate within the {nameof(CertificateAuthorityValidator)}");
        return true;
    }
}

The above is obviously just a “starting point”, where we’re always saying its valid. We’ll wire it up by registering it as a service, and plugging it into our OnCertificateValidated. The writing up of the services I’ve covered several times in other posts but if you need help, take a look at the finished code (TODO put a link here… if I miss this on my review, there’ll probably be a link at the bottom).

OnCertificateValidated = context =>
{
    var logger = context.HttpContext.RequestServices.GetService<ILogger<Startup>>();
    logger.LogInformation("Within the OnCertificateValidated portion of Startup");

    var caValidator = context.HttpContext.RequestServices.GetService<ICertificateAuthorityValidator>();
    if (!caValidator.IsValid(context.ClientCertificate))
    {
        const string failValidationMsg = "The client certificate failed to validate";
        logger.LogWarning(failValidationMsg);
        context.Fail(failValidationMsg);
    }

    return Task.CompletedTask;
}

Above, we’re getting an instance of our ICertificateAuthorityValidator once the client certificate is (otherwise) validated, then running out additional validation procedure on it. If the validation fails, it will mark the validation as failure, otherwise it will still be successful.

With our stubbed implementation returning true from IsValid, let’s see what that looks like:

Changing the stubbed implementation to return false from IsValid:

Actual Implementation

Now we can work on our actual implementation of the CertificateAuthorityValidator. You’ll recall that we can (hopefully) rely on the “Authority Key Identifier” to ensure only our intended CA’s signed certificates can make it through validation.

Shall we do some debugging?

The screenshot above shows that the information we need is in fact present in the data presented to us from the X509Certificate2. To save a bit of time and writing, know that the “raw data” on this extension does represent the same value on the CA cert, but there’s a few additional bytes of information, namely “KeyID=” (as seen in the screenshots earlier). I could not actually get this data from the bytes to confirm (tried getting the byte string as ascii, utf8, and several others), but that’s what it seemed to be. This means for our implementation, we need the “raw data” from this extension, minus a few of the first bytes to account for what I can only assume is “KeyID=”.

The full CertificateAuthorityValidator:

public class CertificateAuthorityValidator : ICertificateAuthorityValidator
{
    private readonly ILogger<CertificateAuthorityValidator> _logger;

    // this should probably be injected via config or loaded from the cert
    // Apparently the bytes are in the reverse order when using this BigInteger parse method,
    // hence the reverse
    private readonly byte[] _caCertSubjectKeyIdentifier = BigInteger.Parse(
        "e9be86f64eb53bc12c1b5fe0f63df450274811da",
        System.Globalization.NumberStyles.HexNumber
    ).ToByteArray().Reverse().ToArray();

    private const string AuthorityKeyIdentifier = "Authority Key Identifier";

    public CertificateAuthorityValidator(ILogger<CertificateAuthorityValidator> logger)
    {
        _logger = logger;
    }

    public bool IsValid(X509Certificate2 clientCert)
    {
        _logger.LogInformation($"Validating certificate within the {nameof(CertificateAuthorityValidator)}");

        if (clientCert == null)
            return false;
        foreach (var extension in clientCert.Extensions)
        {
            if (extension.Oid.FriendlyName.Equals(AuthorityKeyIdentifier, StringComparison.OrdinalIgnoreCase))
            {
                try
                {
                    var authorityKeyIdentifier = new byte[_caCertSubjectKeyIdentifier.Length];
                    // Copy from the extension raw data, starting at the index that should be after the "KeyID=" bytes
                    Array.Copy(
                        extension.RawData, extension.RawData.Length - _caCertSubjectKeyIdentifier.Length, 
                        authorityKeyIdentifier, 0, 
                        authorityKeyIdentifier.Length);

                    if (_caCertSubjectKeyIdentifier.SequenceEqual(authorityKeyIdentifier))
                    {
                        _logger.LogInformation("Successfully validated the certificate came from the intended CA.");
                        return true;
                    }
                    else
                    {
                        _logger.LogError($"Client cert with subject '{clientCert.Subject}' not signed by our CA.");
                        return false;
                    }
                }
                catch (Exception e)
                {
                    _logger.LogError(e, string.Empty);
                    return false;
                }
            }
        }

        _logger.LogError($"'{clientCert.Subject}' did not contain the extension to check for CA validity.");
        return false;
    }
}

curl --insecure -v --key badClient.key --cert badClient.crt https://localhost:5001/weatherForecast

curl --insecure -v --key badClient.key --cert badClient.crt https://localhost:5001/weatherForecast