Forem: Dalibor Kundrat

From .Net Metrics to Graphql Sub.

Dalibor Kundrat — Wed, 02 Nov 2022 07:10:59 +0000

Metrics scenario:

You are collecting metrics using .Net Metrics API
Then process, aggregate and export metrics using Opentelemetry SDK

Under the hood, Opentelemetry SDK connects to the .Net Metrics API by listening to registered meters.

A combination of both is recommended!

If you are creating an internal library, you probably do not want to equip all your class libraries with opentelemetry packages.

.Net Metric != Opentelemetry Metric.

It is important to point out that things work a little differently on each side.

.Net Metrics API

Provides just abstraction and events for telemetry/metrics.
No value store
No aggregation
No ready-made solutions

Opentelemetry SDK

Provides extended implementation
Aggregation
Value store.
Less code control (cant manipulate with the store)

Counter as example

Let's take a look at how the .Net counter is handled from start to finish...

1) We define Meter.
2) We register concrete counter under Meter.
3) We trigger a counter from code based on some event/action.

    // Meter instance
    var _meter = new Meter("MeterName","v0.1");

    // Counter creation
    var counter = _meter.CreateCounter<long>("InstrumentName", "Unit", "Description");

    // Triger on Event
    counter.Add(1)

It is important to understand that the .Net counter has no value and has no idea about the previous state

You cannot query the current value, reset it, or interact with it.

What happens if you call `counter.Add(Value)` multiple times?

Registered listener callbacks are called with the current Add(..) value.
No previous value or state is passed to the listener.
No value is present between the calls

Then why is it called a counter?

It`s just an abstraction with information about the type and how much to increase/decrease the value. (+1 ,+10, -5) by concrete listeners.

How to get real counter behaviour?

A: Manual - > Write code yourself

You need to implement a custom metric listener
Based on the counter event callback, you need to update a variable that holds the counter reading for you.

Simple example:

Advantage is that you have full code control.

B: ReadyMade -> Use Opentelemetry SDK

Use Opentelemetry SDK to do this automaticaly for you and access values by custom readers/exporters.

Advantage is that someboady else wrote this for you

Opentelemetry SDK contains AgregateStore. For each metric, a cumulative, a sum, or a histogram value is automatically stored, depending on the type.

For this reason, readers/exporters can access counter value.

All aggregates are stored in memory! When the process is restarted, all values are lost

Value representation

All metric values are treated internally as double or long.

Depending on the source type, Opentelemetry SDK treats them as double/long and assigns the value to a structure. This ensures the same memory allocation for all types.

So if you create a counter, its value will end up as long in the aggregate memory.

From sources:

This has simplified many SDK internal processes.

Forget the idea that a counter of type int saves some memory :)

Unusal custom reader+exporter

Readers are responsible for manipulating data and passing metrics to the exporter.
Exporters receive a batch of metrics, and each metric may contain 1 or more metric points that must be exported.

Why do we need a custom?

In a project I was working on, I was looking for the easiest way to pass some real-time metrics to GraphQL subscriptions.

Lets have a look on example. The servers are IoT brokers for specific communication protocol.

There are several independent parts in the code and it was not possible to create a worker service for each to propagate values.
In the future, there will be more "servers" and API normalization was necessary.

I end up creating a custom reader + exporter that reads metrics and propagate those values to Graphql subscriptions.

Each part simply uses the abstract .Net Metric API + a base class to hold the metric in the required form.
Reader and Exporter collect and push changes to Graphql Sub.
All exported values are wrapped by a custom unified object with a dynamic value type property

Clients and Portal users can now easily access specific real-time metrics data. All other metrics/traces are exported using the default built-in exporters.

The metric name is converted to a subscription topic and allows filtering based on server UID or metric name.

You can find me on GitHub.and feel free to ask questions. For this reason, readers/exporters cant access counter value.

API Testing with Postman and Newman from CI/CD

Dalibor Kundrat — Mon, 06 Dec 2021 08:50:47 +0000

I`ll show you how to use Postman to test protected GraphQL API and run test collection using Newman from GithubActions!

Note: This article is part of opensource fullstack app

Postman as GraphQL IDE?

Postman is a well-known tool that helps you test your APIs.

Postman supports:

REST
WebSockets
GraphQL

Not yet, but I see the possibility of it becoming a powerfool tool for GraphQL.

Import GraphQL SDL

This has to be done manually! (Unfortunately) Postman does not support schema introspection.

Pre-requests scripts

The Pre-Request and Tests can be found in several scopes.

Scope is just place, where you can find them.

Scope level Collection
Section level Folder
Scope level Request

Is there a initial script?

Postman does not have init script that runs once on startup! You can workaround this and implement it using variables!

How to sign-in with OAuth?

This depends on whether you are doing the tests manually or automated

If you run heedless from CI/CD you need to write scripts and that can be difficult with some oidc providers who have strict rules and check all security aspects.

For testing purposes, you can define a specific oidc client_id:Postman. You can simply protect it with ResourceOwnerPassword and create a test user for it.

Image from Okta

The ResourceOwnerPassword is fully secured in this scenario:

You have a special client (oidc client) and you need Secret to use it.
The test user still needs to provide his name and password.
Will only be activated under a specific flag
Will not be used publicly
Can be restricted to a specific origin

Request access token

To request a new token, you must make an asynchronous call to the oidc token endpoint.

Pre-scripts:

1) Check if you have the token from the previous run
2) Validate the lifetime of the token.

Token is valid - > uses the existing one
Token has expired - > requests a new one

The OAuth_Token_Exchange triggers the request with the function pm.sendRequest(...).

Validation of the GraphQL response

GraphQL returns json and you need to create a helper script to validate it. This script validates property by property with the option to ignore some.

Export test collection

For that you need to export the collection and the variables and use exported data in headless ci/cd runner.

Export collection:

Export enviroment:

Newman headless runner

Newman is CLI runner for postman collections. You can find it on Github.

Install with: npm install -g newman.

Run test collection from console (terminal):

(We use the --insecure flag because we use untrusted developer certificates).

This is an example of the output

Integration in Build and CI/CD

The demo application uses Nuke for build automation.

Nuke is a console application that contains all build logic defined as Targets and allows you to run cross-platform and generate .yaml for CI/CD pipelines.

Let us place Newman in front of the Cypress tests.

To better understand what Target_E2E_Tests is. Let us take a look at the whole pipeline and the dependencies between the different targets.

Targets define named operation with specific actions.

Github-Action run:

Repository

You can find the full source code of the app, including identity, distributed logging, tracing and monitoring, in the open source

Github repo:
https://github.com/damikun/trouble-training

Newman sources sub-link (under postman folder):
https://github.com/damikun/trouble-training/tree/main/Src/Tests

Why GraphQL IDEs on the market are not what you want

Dalibor Kundrat — Mon, 29 Nov 2021 14:07:16 +0000

😕 This will not be an educational article, but only a murmur about the state of the market.

There are a few GraphQL IDEs on the market today that support various features, but I am not satisfied with any of them. (True state)

These tools should be our internal applications/tools and make it easier to work with GraphQL!

Todays GraphQL IDEs

Graphiql - Facebook's oldest original still looks like it's from the 90s and does not support tabs. The tool itself is great because it's stable and provides separate packages under Monorepo that are used by different tools. Let us say it's a tool used to build other tools :)
Playground - The good old playground was widely used. It was developed by Prisma, but then moved under Graphiql (Graphql Foundation). With that, the project was discontinued and there is nothing as simple and good anymore. Playground was just good for the old days.
Apollo Studio - Apollo has several products to work with, sometimes I do not even know what is deprecated and what is current (Devtools, Studio and whatever) have you tried it? Not really my taste. It's like with the Apollo client, nice to look at but then you realize there's Relay too :) But they have advanced functionality that is true!
Altair - No really good UX experience
Insomnia - I was surprised by the functionality. I like plugins a lot. It is close to the requirements I will talk about.
Firecamp - Another IDE, which has the possibility be good, if they implement what they have planned. But it is not yet finished!
And whatever else - I am just not happy with them.

Let me mention one other option that exists. The Postman.

Postman as GraphQL IDE? Not yet. But I see the possibility of it becoming a powerful tool for GraphQL.

This is something I do not understand. They have a great feature set and also support the basics of GraphQL, but they just do not care! Important features like introspection are not supported. If they would invest more time, they could become a gamechanger, but they just do not care. Just check requested PRs.

But back to the topic: if you are a business, you'll probably end up building your own tool.

The requirements

Readable UI experience and easy to work with (most current IDEs are garbage for this UX theme)
Ability to store your state in the cloud / locally / on your own backend (having your own storage is a really good idea, especially if you are in the private sector)
Prepare test collections, import/export work with them.
Headless cli to be able to run collections in CI/CD.
Voyager built in! Why not? I do not want to have multiple tools.
You want to monitor your requests and view telemetry data, you want to understand your requests.
Auth, SSL are standard requirements today
Query optimization, maybe too much, but it works.

Result

The company ends up developing a custom tool to get all this done, because it's really not hard to do. (Only time matters). We use Graphiql as a base and everything else is just WebApp/cli programming as we know it from our daily work.

Keep things under control.

Goodbye and have a nice day!

Cross-platform Build Automation with Nuke

Dalibor Kundrat — Thu, 11 Nov 2021 05:56:29 +0000

Intro

The build process involves a series of automated tasks performed on raw source code to prepare that code for future deployment.

Instead of manually calling the compiler and package manager as dotnet restore/dotnet build, you can automate this process, especially if you are working with multiple solutions/projects that depend on each other and require certain steps to be built.

Build automation is not just about building the code

Rather, it provides many features and rich tools that help with:

Cleanup
Restore packages and dependencies
Compilation
Test automation
Post-build processing
Generating production builds
Packaging, creating Docker images, etc.
Deployment between different environments
Notifications and reporting
Integration of 3-party applications
Version control

GithubActions, AzurePipelines, TeamCity and what else?

You know these build servers are widely used, but there are also many online tools as SonarCloud etc. used by companies and OSS projects.

😎They are cool, no reason to stop using them, but the way how you configure them is not a daily pleasure.

Problems:

They provide several pre-built actions, and each time you have to decide what to use and how to configure it, what arguments to enter, etc.
Yaml as a common configuration definition really sucks. Once I read that it's there to make developers feel at home... 🤣 Where is intelisense and strong typecheck? i am .Net developer.
You can not build software locally and on CI/CD server equally. We need to create helper scripts for local development.
Stack changes require studying new documentation and bending the existing solution.
Cross-platform issues with custom scripts and paths
Configuring the build process is something you do not want to do every day. It's best not to touch it until it's working because it's so complex and you can not remember exactly what each part does because you picked it up from the first post on the internet.

Nuke build

Nuke is hire to provide a nice typed and cross-platform experience by setting up the build process using console application. The oficial documentation can be found on www.nuke.build

It helps you define dependencies between different actions (targets) and automatically generate code for CI/CD in advance, from your C# code using the code base you know from your daily work.

Since nuke runs as console app it provides nice strongly typed intellisense with documentation.

Nuke is not a compiler, it's just an abstraction that gives you a pleasant experience with your code. It provides built-in functionality with source generators driven by attributes and builder patterns.

It's also extensible and currently offers a variety of integrations. So you get a beautiful API that works across platforms and is not just for C# programmers!

Quick setup

To install Nuke.GlobalTool, run the following command:

dotnet tool install Nuke.GlobalTool --global

Now you can run the quickstart setup with nuke:setup from your project root directory. This is usually the folder where you have the .gitignore file.

⠀
NOTE: As you can see, you can customize the root folder and the location of the build project! Make sure that the defined paths/folders are not part of .gitignore. They must be part of source-control!
⠀

You can skip the QuickStart if you want, but it generates some important cross-platform execution scripts that help install the SDK!

Let us take a look at the generated files:

.build - Contains a C# console application where you can define your build process.
.nuke - Contains the nuke schema and temporary files (These are generated).
build.cmd - Is a cross-platform script (runs on Mac/Win/Linux) that triggers a concrete OS script to install the .Net SDK and required dependencies to get Nuke running properly as a console application.
build.sh - Linux/MacOS init script
build.ps1 - Windows init script

The ./build/Build.cs file is the entry point of the Nuke console applicati.

This is some dummy example of build:

[CheckBuildProjectConfigurations]
[ShutdownDotNetAfterServerBuild]
class Build : NukeBuild
{
    public static int Main() => Execute<Build>(x => x.Compile);

        Target Clean => _ => _
        .Before(Compile)
        .Executes(() => {
            // Action you wanna perform in clean
        });

        Target Compile => _ => _
        .Executes(() => {
            // Action you wanna perform in copile
        });

        Target Pack => _ => _
        .DependsOn(Compile)
        .Executes(() => {
            // Action you wanna perform in pack
        });
}

It contains the console Main() with the default Target.

In the abaw example, the default target is Compile = > (Run(x = > x.Compile)). This means when you run the script ./build.cmd, it will use Compile and create the execution plan for it!

To visualize the dependency between the defined targets, you can run nuke --plan and get the following visualization in your browser:

More complex Demo example

To show an example of a build system, I will use free opnesource demo app which I have in my github.

The demo contains several projects that are interconnected. I want to build the frontend and the backend separately and do this on each push on Github to verify that the code build is possible and bug-free, and to ensure that all backend tests can be run OK.

This is the Nuke build plan for this project (nuke --plan):

As you can see, there are multiple Targets and dependencies between them. Sometimes you just want to build the frontend, other times the backend, or all together.

⠀
NOTE: With Nuke you define tarets and it is up to you how you define the relationships between the tarets with the builder patterns .DependsOn(...), .Before(...) , .After(...) and so on..
⠀

Since the demo consists of several parts (frontend, backend, tests), you want to avoid putting all the code in one file. Instead, you can use partial class.

.build/Build.cs

partial class Build : NukeBuild {
    // All concat targets
 }

.build/Build.Backend.cs

partial class Build : NukeBuild { 
    // All related to backend Clen/Build/Compile
}

.build/Build.Backend.Test.cs

partial class Build : NukeBuild { 
    // All related to backend Unit/Integration/API Tests
}

.build/Build.Frontend.cs

partial class Build : NukeBuild { 
    // All related to frontend Clean/Restore/Compile/Build
 }

The main Build class derived from base NukeBuild provides some event callbacks that can be overridden:

// Method that is invoked after the instance of the build was created.
protected internal virtual void OnBuildCreated();

// Method that is invoked after the build has finished (succeeded or failed).
protected internal virtual void OnBuildFinished();

// Method that is invoked after build instance is initialized. I.e., value injection
protected internal virtual void OnBuildInitialized();

// Method that is invoked when a target has failed.
protected internal virtual void OnTargetFailed(string target);

// Method that is invoked before a target is about to start.
protected internal virtual void OnTargetRunning(string target);

// Method that is invoked when a target is skipped.
protected internal virtual void OnTargetSkipped(string target);

// Method that is invoked when a target has been executed successfully.
protected internal virtual void OnTargetSucceeded(string target);

Example:

protected override void OnBuildInitialized()
{
    Logger.Info("🚀 Build process started");

    base.OnBuildInitialized();
}

Let us take a look at Main Build.cs for this demo:

.build/Build.cs

[GitHubActions(
    "backend-restore-build-and-test",
    GitHubActionsImage.WindowsLatest,
    GitHubActionsImage.MacOsLatest,
    InvokedTargets = new[] { nameof(Backend_All) },
    OnPushIncludePaths = new[] { "Src/**" },
    OnPushBranches = new[] { "main" },
    AutoGenerate = true)]
[GitHubActions(
    "frontend-restore-and-build",
    GitHubActionsImage.WindowsLatest,
    GitHubActionsImage.MacOsLatest,
    InvokedTargets = new[] { nameof(Frontend_All) },
    On = new[] {
         GitHubActionsTrigger.PullRequest,
         GitHubActionsTrigger.Push
    },
    OnPushBranches = new[] { "main" },
    AutoGenerate = false)]
[CheckBuildProjectConfigurations]
[ShutdownDotNetAfterServerBuild]
partial class Build : NukeBuild
{
    public static int Main() => Execute<Build>(x => x.Backend_Compile);

    //---------------
    // Params and Definitions
    //---------------

    [Parameter("Configuration to build - Default is 'Debug' (local) or 'Release' (server)")]
    readonly Configuration Configuration = IsLocalBuild ? Configuration.Debug : Configuration.Release;

    [Solution] readonly Solution Solution;

    //---------------
    // Enviroment
    //---------------

    AbsolutePath SourceDirectory => RootDirectory / "Src";

    //---------------
    // Build process
    //---------------

    protected override void OnBuildInitialized()
    {
        Logger.Info("🚀 Build process started");

        base.OnBuildInitialized();
    }

    Target Backend_All => _ => _
        .DependsOn(
            Backend_Clean,
            Backend_Test
        );

    Target Frontend_All => _ => _
        .DependsOn(
            Frontend_Clean,
            Frontend_TryBuild
        );

    Target All => _ => _
        .DependsOn(
            Backend_All,
            Frontend_All);
}

The GitHubActions attribute ensures that the correct Github workflow .yaml is created under the root directory ./github/workflows/.

This is very handy as you do not have to write it yourself. Same goes for AzurePipelines, SonarCloud or any other tools out there! (supported by nuke)

[GitHubActions(
    "backend-restore-build-and-test",  // <-- Workflow Name
    GitHubActionsImage.WindowsLatest,  // <-- Runs on Windows
    GitHubActionsImage.MacOsLatest,    // <-- Runs on MacOs
    InvokedTargets = new[] {
         nameof(Backend_All)           // <-- What targets we wanna triger
    },  
    On = new[] {
        GitHubActionsTrigger.Push      // <-- Trigger event
    },
    AutoGenerate = true)]              // <-- Regenerate `.yaml` on change

This informs Nuke to generate the source code as .github/workflows/backend-restore-build-and-test.yaml.

name: backend-restore-build-and-test

on:
  push:
    branches:
      - main
    paths:
      - "Src/**"

jobs:
  windows-latest:
    name: windows-latest
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v1
      - name: Cache .nuke/temp, ~/.nuget/packages
        uses: actions/cache@v2
        with:
          path: |
            .nuke/temp
            ~/.nuget/packages
          key: ${{ runner.os }}-${{ hashFiles('**/global.json', '**/*.csproj') }}
      - name: Run './build.cmd Backend_All'
        run: ./build.cmd Backend_All
  macOS-latest:
    name: macOS-latest
    runs-on: macOS-latest
    steps:
      - uses: actions/checkout@v1
      - name: Cache .nuke/temp, ~/.nuget/packages
        uses: actions/cache@v2
        with:
          path: |
            .nuke/temp
            ~/.nuget/packages
          key: ${{ runner.os }}-${{ hashFiles('**/global.json', '**/*.csproj') }}
      - name: Run './build.cmd Backend_All'
        run: ./build.cmd Backend_All

As you can see from the .yaml definition, Nuke uses the coross-platform script ./build.cmd that I mentioned at the beginning of this article.

After you commit your code, Github will use this file to perform the build process for you.

Let us go ahead and look in detail at how the backendtargets Clean, Build, and Compile are configured:

.build/Build.Backend.cs

partial class Build : NukeBuild
{
    Target Backend_Clean => _ => _
        .Before(Backend_Restore)
        .Executes(() =>
        {
            SourceDirectory.GlobDirectories("**/bin", "**/obj")
                .Where(e => !e.ToString().Contains(
                    "node_modules",
                    StringComparison.OrdinalIgnoreCase))
                .Where(e => !e.ToString().Contains(
                    "ClientApp",
                    StringComparison.OrdinalIgnoreCase))
                .ForEach(DeleteDirectory);

            Backend_TestsDirectory.GlobDirectories("**/bin", "**/obj")
                .Where(e => !e.Contains("node_modules"))
                .ForEach(DeleteDirectory);
        });

    Target Backend_Restore => _ => _
        .Executes(() =>
        {
            DotNetRestore(s => s
                .SetProjectFile(Solution));
        });

    Target Backend_Compile => _ => _
        .DependsOn(Backend_Restore)
        .Executes(() =>
        {
            DotNetBuild(s => s
                .SetProjectFile(Solution)
                .SetConfiguration(Configuration)
                .EnableNoRestore());
        });
}

These are backend test targets

The project contains unit, integration and API tests. They need to be executed together or separately!

.build/Build.Backend.Test.cs

partial class Build : NukeBuild
{

    //---------------
    // Enviroment
    //---------------

    AbsolutePath Backend_TestsDirectory => RootDirectory / "Src" / "Tests";
    AbsolutePath Backend_Unit_Tests_Directory => RootDirectory / "Src" / "Tests" / "APIServer.Aplication.Unit";
    AbsolutePath Backend_Integration_Tests_Directory => RootDirectory / "Src" / "Tests" / "APIServer.Aplication.Integration";
    AbsolutePath Backend_API_Tests_Directory => RootDirectory / "Src" / "Tests" / "APIServer.API";

    //---------------
    // Build process
    //---------------

    Target Backend_Test => _ => _
        .DependsOn(
            Backend_UnitTest,
            Backend_IntegrationTest,
            Backend_APITest
        );

    Target Backend_UnitTest => _ => _
        .DependsOn(Backend_Compile)
        .Executes(() =>
        {
            DotNetTest(s => s
                .SetProjectFile(Backend_Unit_Tests_Directory)
                .SetConfiguration(Configuration)
                .EnableNoRestore()
                .EnableNoBuild());
        });

    Target Backend_IntegrationTest => _ => _
        .DependsOn(Backend_Compile)
        .Executes(() =>
        {
            DotNetTest(s => s
                .SetProjectFile(Backend_Integration_Tests_Directory)
                .SetConfiguration(Configuration)
                .EnableNoRestore()
                .EnableNoBuild());
        });

    Target Backend_APITest => _ => _
        .DependsOn(Backend_Compile)
        .Executes(() =>
        {
            DotNetTest(s => s
                .SetProjectFile(Backend_API_Tests_Directory)
                .SetConfiguration(Configuration)
                .EnableNoRestore()
                .EnableNoBuild());
        });
}

These are front end targets

The frontend needs to be restored (packages installed) and compiled in a few steps, as you can see in the code below.

.build/Build.Frontend.cs


partial class Build : NukeBuild
{
    //---------------
    // Enviroment
    //---------------

    AbsolutePath FrontendDirectory = RootDirectory / "Src" / "BFF" / "API" / "ClientApp";

    //---------------
    // Build process
    //---------------

    Target Frontend_Clean => _ => _
        .Before(Frontend_Restore)
        .Executes(() =>
        {

            SourceDirectory.GlobDirectories("**/BFF/API/ClientApp/node_modules")
                .ForEach(DeleteDirectory);
        });

    Target Frontend_Restore => _ => _
        .Executes(() =>
        {
            // Ecample of intsall packages based on package.json
            NpmTasks.NpmInstall(settings =>
                settings
                    .EnableProcessLogOutput()
                    .SetProcessWorkingDirectory(FrontendDirectory));
        });

    Target Frontend_AddTailwind => _ => _
        .DependsOn(Frontend_Restore)
        .Executes(() =>
        {
            // Example of dirrcty npm install command
            NpmTasks.Npm(
                "install -D tailwindcss@npm:@tailwindcss/postcss7-compat postcss@^7 autoprefixer@^9",
                FrontendDirectory);
        });

    Target Frontend_RelayCompile => _ => _
        .After(Frontend_Restore)
        .Executes(() =>
        {
            // Compile relay
            NpmTasks.NpmRun(s => s
                .SetCommand("relaycompile_presisted_js")
                .SetProcessWorkingDirectory(FrontendDirectory)
            );
        });

    Target Frontend_TryBuild => _ => _
        .DependsOn(Frontend_AddTailwind, Frontend_RelayCompile)
        .Executes(() =>
        {
            NpmTasks.NpmLogger = CustomLogger;

            NpmTasks.NpmRun(settings =>
                settings
                    .SetCommand("build")
                    .SetProcessWorkingDirectory(FrontendDirectory)
            );
        });

    // Helper
    public static void CustomLogger(OutputType type, string output)
    {
        switch (type)
        {
             case OutputType.Std:
                Logger.Normal(output);
                break;
            case OutputType.Err:
                {
                    if (
                        output.Contains(
                            "npmWARN",
                            StringComparison.OrdinalIgnoreCase
                        ) ||

                        output.Contains(
                            "npm WARN",
                            StringComparison.OrdinalIgnoreCase
                        ))

                        Logger.Warn(output);

                    else

                        Logger.Error(output);
                    break;
                }
        }
    }

}

What's next?

Next time I will show you packing and notification process.

Repository

You can find the full source code under .build folder:

https://github.com/damikun/trouble-training

Web App Security, Understanding the Meaning of the BFF Pattern

Dalibor Kundrat — Tue, 19 Oct 2021 22:21:51 +0000

Identity and security overview

When it comes to security. There is a lot of guidance and also advice and commentary on why you should or should not use each approach. It's hard to understand all the different options and security issues they have.

You also have different clients (mobile/web/app) and each has different security requirements for managing, storing and processing data in the form of tokens, cookies or sessions.

At the end, your app may scale and not all options can be deployed as multiple instance nodes without additional changes to the app and infrastructure.

This means that you really need to make decisions at the beginning and ask questions like: What are your customers? and What are your requirements? and What can be added to the system in the future? and How is it possible to deal with the chosen approach? etc. etc..

This article is part of opensource training.

Authentication vs Authorization

Authentication - process of verifying who a user is (identity).
Authorization - process of checking what a user has access to (checking permissions to access a particular resource).

Tokens vs Cookie and Sessions

Tokens, cookies and sessions can be interpreted as resources or tools used by various protocols/standards/patterns (OAuth, OpenID, BFF) to perform identity-related tasks.

It is important to understand the basics as they end up being used in combination.

In the end, you can wrap the JWT token in a cookie and use it with session data for 'authorization' and 'authentication'.

Cookies

Cookies can be understood as small blocks of data created by the server that are written once in the response and automatically resent with each subsequent request from the web browser until their lifetime expires.

They can be used for authentication/authorization or tracking and marketing purposes. And much more...

IndentityServer cookies example:

There are 3 main properties of cookies:

HttpOnly - An http-only cookie cannot be accessed by client-side APIs, such as JavaScript. The browser will not allow you to access this cookie from the front-end code.
Secure A secure cookie can only be transferred over an encrypted HTTPS connection.
SameSite 3 option values Strict, Lax or None - this tells the browser which domain and address the cookie can be sent to.

What is origin?

The origin is usually ip and port or domain name and subdomain name.

// This are different origins since subdomain are different
https://developer.mozilla.org
https://mozilla.org

// This are also different origins since port number is different
https://localhost:5001
https://localhost:7001

Another cookie definitions:

Session cookies - Created only for the browser session (in memory) and deleted/lost after closing.
Third-party cookies - Usually the domain attribute of a cookies matches the domain displayed in the address bar of the web browser. as first-party cookies. The third-party cookies does not match the current domain and is used as tracking cookies to track user activity.

Sessions

Session is used to temporarily store information on the server for use across multiple pages of the site. It is usually associated with a cookie that is used to identify the session stored on the server, but does not contain any data.

Tokens

Tokens are data elements that allow application systems to perform the authorization and authentication process. They are usually encoded as base64 strings.

There are several types of tokens:

access token - Includes user claims and signs them with a secret. It uses JWT tokens.
refresh token - Used to "refresh " and get a new 'access token' after its lifetime expires.
id token - JSON encoded data about user profile information
etc, etc...

JWT Tokens

JSON Web Token is an open standard that defines how information can be securely transferred between parties as a JSON object.

They are used for authorization and information exchange as they provide a security proof that the information wrapped in them is valid and written by a trusted source.

You can easily write arbitrary data to tokens, sign that data, and then have clients use it to access server resources. The server can verify that the token was signed and is still valid.

Basic JWT token flow example:

JWT content:

JWT consist of 3 parts:

Header - Contains information such as the type of token (JWT) and the signing algorithm used, e.g. HMAC SHA256 or RSA.
```
{
"alg": "HS256",
"typ": "JWT"
}
```

Payload - Securly signed data (claims)

{
"sub": "1234567890",
"name": "John Doe",
"admin": true
}

Signature - Encrypted header, the encrypted payload, a secret and signed by an algorithm specified in the header.
```
HMACSHA256(
base64UrlEncode(header) + "." +
base64UrlEncode(payload),
secret
```
More additional info about JWT tokens can be found at: official documentation.

The most commonly used token for authorizing access to APIs is the Bearer token.

Identity protocols

There are several protocols/specifications available to manage your identity or authorization process.

This was necessary to standardize authentication and authorization between services and clients. So we can use different global identity/authentication providers like Facebook, Google (external/internal) and also standardize the way the process is implemented.

This demo focuses on the most commonly used protocols OAuth and OpenID Connect.

Both protocols use a JWT token by default to encrypt and sign sensitive data, or to verify that the request was sent from a trusted source. It is also possible to use cookies on the front end and let the back end do the session and token authorization for you.

You can also watch and learn from various talks:

OAuth

Primarily used to authorize an app's access to a specific resource. This is done without having to share your password with external sites.

If you have ever signed in to a new app and agreed to access your contacts, calendar, etc., you have used OAuth 2.0. This protocol does not provide any information about the user's endpoint, just a token to access certain resources. You can read more about OAuth at this document.

OAuth generally provides clients with "secure delegated access " to certain resources. Imagine you are a Google user and an app wants to access your calendar data. This can be an example of a flow:

OAuth flow example:

In the above example, an application like Slack, Jira, etc. only gets permission to access a specific resource (e.g. the calendar), but not the user itself, so profile data like username and email are not transferred and remain protected.

If you want to learn more about OAuth, you can watch the following presentations:

Introduction to OAuth 2.0 Flow - [Philippe De Ryck]

Token exchange flows

There are several ways in which grant can be substituted. The choice depends on what kind of client is requesting access and how much that client is trusted.

Authorization code flow
Flow of authorization code with PKCE
Implicit Flow
Client credentials flow

Image from Okta

OpenID Connect

OpenID is a protocol for decentralized authentication

A login used by multiple internal/external applications. If you used your Google or Facebook etc. to log in to an external web or app, then you used OpenID Connect.

OpenID Connect is based on OAuth 2.0. (OAuth is the underlying protocol and OpenId is the identity layer built on top of it) and also uses a JWT token called id_token which encapsulates identity claims in JSON format. For more information about OpenId, see under this specification.

id_token example:

{
"iss": "http://server.example.com",
"sub": "248289761001",
"aud": "s6BhdRkqt3",
"nonce": "n-0S6_WzA2Mj",
"exp": 1311281970,
"iat": 1311280970,
"name": "Dalibor Kundrat",
"given_name": "Dalibor",
"family_name": "Kundrat",
"gender": "male",
"birthdate": "0000-10-31",
"email": "d.kundrat@example.com",
"picture": "http://example.com/somepicture_of_dalibor.jpg"
}

OpenId flow example:

There are several flows that can be used with OpenId. You can read more about them under this article.

Each OpenId server by specification provides multiple endpoints to interact with.

The URLs of all endpoints can be explored using the global discovery endpoint. Often referred to as disco. It is available under the path: /.well-known/openid-configuration and returns JSON OpenID Connect metadata related to the specified authorization server.

Example disco response:

  {
   "issuer":"https://localhost:5001",
   "jwks_uri":"https://localhost:5001/.well-known/openid-configuration/jwks",
   "authorization_endpoint":"https://localhost:5001/connect/authorize",
   "token_endpoint":"https://localhost:5001/connect/token",
   "userinfo_endpoint":"https://localhost:5001/connect/userinfo",
   "end_session_endpoint":"https://localhost:5001/connect/endsession",
   "check_session_iframe":"https://localhost:5001/connect/checksession",
   "revocation_endpoint":"https://localhost:5001/connect/revocation",
   "introspection_endpoint":"https://localhost:5001/connect/introspect",
   "device_authorization_endpoint":"https://localhost:5001/connect/deviceauthorization",
   "frontchannel_logout_supported":true,
   "frontchannel_logout_session_supported":true,
   "backchannel_logout_supported":true,
   "backchannel_logout_session_supported":true,
   "scopes_supported":["profile","openid","email","role", "offline_access" //etc..],
   "claims_supported":["name","family_name","profile","email", etc..],
   "grant_types_supported":["authorization_code","client_credentials", "refresh_token", //etc..],
   "response_types_supported":["code","token", "id_token","id_token token" //etc..],
   "response_modes_supported":["form_post","query","fragment"],
   "token_endpoint_auth_methods_supported":["client_secret_basic","client_secret_post" ],
   "id_token_signing_alg_values_supported":["RS256"],
   "subject_types_supported":["public" ],
   "code_challenge_methods_supported":["plain","S256"],
   "request_parameter_supported":true,
   "request_object_signing_alg_values_supported":["RS256","RS384" //etc..],
   "authorization_response_iss_parameter_supported":true
}

Main OpenId endpoints:

/authorization_endpoint - interacting with the resource owner and obtaining an authorization permission.
/token_endpoint - Obtain an access and/or ID token by presenting an authorization permission (code) or refresh token
/revocation_endpoint - Revoke an access or refresh token.
/end_session_endpoint - Ends the session associated with the specified ID token.
/userinfo_endpoint - Provides information about the authenticated end user.

⠀
NOTE: All of these values in the discovery endpoint refer to the current server configuration. You can adjust or enable/disable certain options in your code during idnetityserver configuration.
⠀

Backend for Frontend pattern (BFF)

BBF is a backend used by a particular front-end application.

Since endpoint APIs may have multiple clients with different requests, BFF can provide a client-specific backend mediator and act as a proxy that forwards and merges multiple requests to different service APIs.

⠀
Ok, we have cookies, tokens, and sessions. We use them for various authentication/authorization protocols (OpenId, OAuth, etc.) and what the hack BFF is good for.?

Answer is:

Security reasons

Architecture reasons ⠀

Security reasons

In recent years, it was common to implement OpenID Connect for SPAs in Javascript (React, Angular, Vue...), and this is no longer recommended:

Using access tokens in the browser has more security risks than using secure cookies.
A SPA is a public client and cannot keep a secret, as such a secret would be part of the JavaScript and could be accessible to anyone inspecting the source code.
Recent browser changes to prevent tracking may result in 'third-party cookies' being dropped.
It is not possible to store something securely in the browser for a long period of time, as it can be stolen by various attacks.

Due to the above issues, the best security recommendation for SPA is to avoid storing tokens in the browser and create a lightweight backend to help with this process, called Backend for Frontend pattern (BFF).

This way, you can still use acces_tokens to authorize access to all your APIs, but clients will use session cookies or tokens in case of mobile devices and BFF will proxy this sides.

BFF can be:

statefull - stores tokens in memory and uses a session to manage them.
stateless - stores the tokens in encrypted HTTP-only, same-page cookies.

Architectural reasons

When you design your application, you have several options on how to access APIs from clients (web/mobile/external).

1) A single API gateway that provides a single API for all clients
2) A single API gateway that provides an API for each type of client
3) A per-client API gateway that provides an API to each client.

BFF vs API gateway

While an API Gateway is a single entry point into to the system for all clients, a BFF is only responsible for a single type of client.

BBF cookies termination and token isolation

As mentioned in the text, the most important thing:

Avoid storing tokens in the browser. (No tokens in the browser policy).
Store tokens on the server side and use encrypted/signed HTTP-only cookies.

Recommended BFF pattern to secure SPA frontends:

Using this, all communication from the SPA frontend to the authorization server now passes through the BFF and tokens do not reach the SPA.
The BFF now issues session cookies. These are part of the request to APIs and are exchanged for an access token at the proxy level.
Client-side cookies are terminated by the BFF proxy.

What's next?

Next time I will show you concrete implementation of BFF.

Repository

You can find the full source code of the app, including identity, distributed logging, tracing and monitoring, in the open source Github repo:

https://github.com/damikun/trouble-training

Integrate WebHooks under ASP.NET Core (c#) backend

Dalibor Kundrat — Tue, 19 Oct 2021 05:08:41 +0000

WebHooks

A webhook is a way for a application to provide callbacks for 3-part applications.

When an event occurs, the source application typically triggers an http POST call to a pre-configured external URL and wraps the data from the triggered event in the request's payload.

This approach allows external applications to respond to events via the standard WebAPI interface.

This pattern is used by most large companies and I will use Github (listen to new pull request or issue was created) as a template and integrate this functionality into the NetCore (c#) backend application.

Demo requirements

Be able to create/update and delete webhooks.
Each WebHook can listen for one or more events.
WebHook processing must be asynchronous to the user request, including some basic retry policies in case delivery fails.
You can view the history of WebHooks that have been triggered.
You can view the system history (audit logs) when the webhook was created/deleted or the parameters were updated.

⠀
Note: You can find the full source code of the app, including identitiy, distributed logging, tracing and monitoring, in the open source github repo:

https://github.com/damikun/trouble-training
⠀

ED diagram

The ED diagram (Entity Relationship Model) shows the basic WebHook database tables and the relationships between them. This is what we need to build in our database

On the left you see the System Event Model, which is used to store information such as WebHook Created/Updated or Deleted. This represents some system audit (tracking) logs for monitoring webhook history in the application.

On the right side you can see the WebHook database model, which is very simple. Each webhook has multiple headers and multiple trigger history.

WebHook structure:

ID - primary key of the database
WebHookUrl - the url that will be triggered when someone is listening.
Secret - this is the API key or authorization token used by 3-part applications to protect the API and only accept requests that have a valid secreat. The secret is transmitted with each request as the HTTP header ''X-Trouble-Secret'', which you can later rename as you wish.
ContentType - represents the ContentType of the HTTP message.
IsActive - is a flag to enable/disable the hook to send events.
HookEvents - collection of events that the hook should listen for.
Headers - list of additional headers you want to send with the webhook.
Records - history of the webhook triggering log.
LastTrigger - last time when the webhook was triggered.

WebHookHistory Structure:

ID - Primary key of the database
WebHookID - Associated webhook primary key
Guid - alternate global unique key
HookType - type of event that triggers the hook
Result - result object of the webhook run
StatusCode - result status code
ResponseBody - body rcv from triggered endpoint url
RequestBody - requested body
RequestHeaders - requested headers sent with the message
Exception - information about any exceptions
Timestamp - date and time of the triggering event

WebHookHeader structure:

ID - primary key of the database
WebHookID - associated webhook primary key
Name - name of the header
value - value of the header

To convert this diagram into real database tables, I use the 'code-first-approach' with EFCore ORM (Object-relational mapper) and some design packages that help us convert C# code into an automatically generated DB migration.

Stack

This is the most important part of this demo:

Netcore as the backend framework.
EF-Core as ORM to map our C# objects to database entities and EF-Tools to create database migrations.
MediatR as in-memory messaging to process commands like CreateWebHook/ProcessWebHook etc.
Hangfire for processing hooks in a queue and for basic retry policies.
PostgreSQL as a database. (You can also use in-memory for testing).

NOTE: The final code sources are more complex and this article focuses only on WebHook integration!

Domain level

The domain layer is a collection of entity objects and associated business logic that we call domain models.

WebHook Domain Model

Since we are using the code-first approach to create database tables, we should define our WebHook domain:

/Src/APIServer/Domain/Hooks/WebHook.cs


namespace APIServer.Domain.Core.Models.WebHooks {
    public class WebHook {

        public WebHook() {
            this.Headers = new HashSet<WebHookHeader>();
            this.HookEvents = new HookEventType[0];
            this.Records = new List<WebHookRecord>();
        }

        /// <summary>
        /// Hook DB Id
        /// </summary>
        public long ID { get; set; }

        /// <summary>

        /// <summary>
        /// Webhook endpoint
        /// </summary>
        public string WebHookUrl { get; set; }

        /// <summary>
        /// Webhook secret
        /// </summary>
        #nullable enable
        public string? Secret { get; set; }
        #nullable disable

        /// <summary>
        /// Content Type
        /// </summary>
        public string ContentType { get; set; }

        /// <summary>
        /// Is active / NotActiv
        /// </summary>
        public bool IsActive { get; set; }

        /// <summary>
        /// Hook Events context
        /// </summary>
        public HookEventType[] HookEvents { get; set; }

        /// <summary>
        /// Additional HTTP headers. Will be sent with hook.
        /// </summary>
        virtual public HashSet<WebHookHeader> Headers { get; set; }

        /// <summary>
        /// Hook call records history
        /// </summary>
        virtual public ICollection<WebHookRecord> Records { get; set; }

        /// <summary>
        /// Timestamp of last hook trigger
        /// </summary>
        /// <value></value>
        public DateTime? LastTrigger { get; set; }
    }
}

Each hook contains multiple WebHookRecord.

/Src/APIServer/Domain/Hooks/WebHookRecord.cs

namespace APIServer.Domain.Core.Models.WebHooks {
    public class WebHookRecord {

        public WebHookRecord() { }

        /// <summary>
        /// Hook record DB Id
        /// </summary>
        public long ID { get; set; }

        /// <summary>
        /// Linked Webhook Id
        /// </summary>
        public long WebHookID { get; set; }

        /// <summary>
        /// Linked Webhook
        /// </summary>
        public WebHook WebHook { get; set; }

        /// <summary>
        /// Unique GUID
        /// </summary>
        public string Guid { get; set; }

        /// <summary>
        /// WebHookType
        /// </summary>
        public HookEventType HookType { get; set; }

        /// <summary>
        /// Hook result enum
        /// </summary>
        public RecordResult Result { get; set; }

        /// <summary>
        /// Response
        /// </summary>
        public int StatusCode { get; set; }

        /// <summary>
        /// Response json
        /// </summary>
        public string ResponseBody { get; set; }

        /// <summary>
        /// Request json
        /// </summary>
        public string RequestBody { get; set; }

        /// <summary>
        /// Request Headers json
        /// </summary>
        public string RequestHeaders { get; set; }

        /// <summary>
        /// Exception
        /// </summary>
        public string Exception { get; set; }

        /// <summary>
        /// Hook Call Timestamp
        /// </summary>
        public DateTime Timestamp { get; set; }
    }


    public enum RecordResult {
        undefined = 0,
        ok,
        parameter_error,
        http_error,
        dataQueryError
    }
}

Each hook contains multiple WebHookHeader

/Src/APIServer/Domain/Hooks/WebHookHeader.cs

namespace APIServer.Domain.Core.Models.WebHooks {
    public class WebHookHeader {

        /// <summary>
        /// Hook header ID
        /// </summary>
        public long ID { get; set; }

        /// <summary>
        /// Linked Webhook Id
        /// </summary>
        public long WebHookID { get; set; }

        /// <summary>
        /// Linked Webhook
        /// </summary>
        public WebHook WebHook { get; set; }

        /// <summary>
        /// Header Name
        /// </summary>
        public string Name { get; set; }

        /// <summary>
        /// Header content
        /// </summary>
        public string Value { get; set; }

        /// <summary>
        /// Header created time
        /// </summary>
        public DateTime CreatedTimestamp { get; set; }
    }
}

Each webhook can be triggered by multiple events:

/Src/APIServer/Domain/Hooks/WebHookEvents.cs

namespace APIServer.Domain.Core.Models.WebHooks {

    public enum HookEventType {

        // Take this as an example, you can implement any event source you like.
        hook, //(Hook created, Hook deleted ...)

        file, // (Some file uploaded, file deleted)

        note, // (Note posted, note updated)

        project, // (Some project created, project dissabled)

        milestone // (Milestone created, milestone is done etc...)

        //etc etc.. You can define your custom events types....
    }
}

Each event type groups different related actions:

/Src/APIServer/Domain/Hooks/HookActions.cs

namespace APIServer.Domain.Core.Models.WebHooks {

    // Actions of HookEventType
    public enum HookResourceAction {
        undefined,
        hook_created,
        hook_removed,
        hook_updated,
        // etc...
    }

    // Actions of ProjectEventType
    public enum projectAction {
        undefined,
        project_created,
        project_renamed,
        project_archived,
        //etc...
    }

}

Events Domain model

/Src/APIServer/Domain/Events/HookActions.cs

namespace APIServer.Domain.Core.Models.Events {

    public class DomainEvent {
        public long ID { get; set; }
        #nullable enable
        public Guid? ActorID { get; set; }
        #nullable disable
        public DateTime TimeStamp { get; set; }

        public EventType EventType { get; set; }
    }
}

where EventType is:

/Src/APIServer/Domain/Events/EventType.cs

namespace APIServer.Domain.Core.Models.Events {

    public enum EventType {
        WebHook,
        System,
        Project,
    }
}

And then we can define our system events as follows:

/Src/APIServer/Domain/Events/EventType.cs


    /// <summary>
    /// WebHookCreated
    /// </summary>
    public class WebHookCreated : DomainEvent {

        public WebHookCreated() { }

        public long WebHookId {get;set;}

        // Add any custom props hire...
    }

    // Equal for

Persistence layer

This layer provides helpers to maintain application state at DB. In general, it defines DBContext, database configuration files and migrations.

Make sure you have installed the required packages in your APIServer.Persistence.csproj.

<PackageReference Include="Microsoft.EntityFrameworkCore" Version="5.0.9" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="5.0.9">

Based on our domain model, the DBContext must contain these collections:

WebHooks - Stored WebHooks.
WebHooksHistory - WebHook history logs
Events - WebHook system logs

Database DbContext:

/Src/APIServer/Persistence/ApiDbContext.cs

namespace APIServer.Persistence {

    public class ApiDbContext : DbContext {

        public DbSet<WebHook> WebHooks { get; set; }

        public DbSet<WebHookRecord> WebHooksHistory { get; set; }

        public DbSet<DomainEvent> Events { get; set; }

        public ApiDbContext(
            DbContextOptions<ApiDbContext> options)
            : base(options) { }

        protected override void OnModelCreating(ModelBuilder modelBuilder) {

            modelBuilder.ApplyConfigurationsFromAssembly(typeof(ApiDbContext).Assembly);

            modelBuilder.Entity<WebHookCreated>().ToTable("WebHookCreatedEvent");

            modelBuilder.Entity<WebHookRemoved>().ToTable("WebHookRemovedEvent");

            modelBuilder.Entity<WebHookUpdated>().ToTable("WebHookUpdatedEvent");

            modelBuilder.Entity<WebHook>().HasData(
                new WebHook() { 
                    ID = 1,
                    WebHookUrl = "https://localhost:5015/hookloopback",
                    IsActive = true,
                    ContentType = "application/json",
                    HookEvents=new HookEventType[]{HookEventType.hook}
                });

            base.OnModelCreating(modelBuilder);
        }
    }
}

By overriding the OnModelCreating function, we can configure our database model.

ApplyConfigurationsFromAssembly - Specifies that all configuration files are loaded. These are defined manually in the /Src/APIServer/Persistence/Configuration folder as separate configuration classes.
Define associations of units to tables (TPT)
Initial database data

Entity configuration: (loaded with 'ApplyConfigurationsFromAssembly')

/Src/APIServer/Persistence/Configuration/WebHook.cs

namespace APIServer.Presistence {
    public class WebHookonfiguration : IEntityTypeConfiguration<WebHook> {
        public void Configure(EntityTypeBuilder<WebHook> builder) {

            builder.HasKey(e => e.ID);

            builder.HasMany(e => e.Headers)
            .WithOne(e => e.WebHook)
            .HasForeignKey(e => e.WebHookID);

            builder.HasMany(e => e.Records)
            .WithOne(e => e.WebHook)
            .HasForeignKey(e => e.WebHookID);
        }
    }
}

To generate DB migrations from our DBContext definition and DB configuration files an EFCore Tools is used.

You need to install this with cli by running:

dotnet tool install --global dotnet-ef

You can verify the installation by running dotnet ef in the terminal and you should get the following:

                     _/\__       
               ---==/    \\      
         ___  ___   |.    \|\    
        | __|| __|  |  )   \\\   
        | _| | _|   \_/ |  //|\\ 
        |___||_|       /   \\\/\\

Entity Framework Core .NET Command-line Tools 5.0.9

To generate migration files you can run this command from Src/APIServer/Persistence

dotnet ef migrations add Init

This will create the Migrations folder with the appropriate migration files. To apply these migrations to your database, you can run the following:

dotnet ef database update

⠀
NOTE: Please make sure that the connection string defined in your configuration MigrationConfig.cs is valid. The migration tool needs it to connect to your DB and perform table creation.
⠀

Since we want to keep the start of our main project clean, we also define a service collection extension. This helps us register our DBContext with a single command services.AddApiDbContext(...); and keep all the configuration out of the Startup.cs.

/Src/APIServer/Persistence/Extensions/AddApiDbContext.cs

namespace APIServer.Persistence.Extensions {

    public static partial class ServiceExtension {

        public static IServiceCollection AddApiDbContext(
            this IServiceCollection serviceCollection,
            IConfiguration Configuration, IWebHostEnvironment Environment) {

            serviceCollection.AddPooledDbContextFactory<ApiDbContext>(
                (s, o) => o
                    .UseNpgsql(Configuration["ConnectionStrings:ApiDbContext"], option => {
                    option.EnableRetryOnFailure();

                    if (Environment.IsDevelopment()) {
                        o.EnableDetailedErrors();
                        o.EnableSensitiveDataLogging();
                    }

                    }).UseLoggerFactory(s.GetRequiredService<ILoggerFactory>()));

            return serviceCollection;
        }
    }
}

The ConnectionStrings:ApiDbContext is defined in Src/APIServer/API/appsettings.json as:

{
  "ConnectionStrings": {
    "ApiDbContext": "Host=localhost;Port=5432;Database=APIServer;Username=postgres;Password=postgres",
  },
}

As you can see from the definition, the AddPooledDbContextFactory is used. This is required because GraphQL resolves load data in parallel, so multiple queries are made to the database at the same time. This is not possible with the traditional Scoped db context, as it is not thread-safe and cannot perform parallel processing!

Application layer

This layer contains MediatR (CQRS) commands and queries to accomplish a specific application task.

It also contains a GraphQL layer on top of the MediatR application logic. Unfortunately, since GraphQL and MediatR provide similar concepts, the DDD pattern sometimes does not follow 100% of the known rules and is split between this two parts. Ideally, there would be complete separation, but since advanced error handling or cursor pagination is used, some of the DDD rules are ignored.

This is the distributed trace of the CreateWebHook command. You can see each part that we will implement.

Theoretical flow:

Real trace measurement:

WebHook commands and queries

Lets have a look on example CreateWebHook command:

Src/APIServer/Aplication/Public/Hooks/Create_WebHook.cs

namespace APIServer.Aplication.Commands.WebHooks {

    /// <summary>
    /// Command for creating webhook
    /// </summary>
    [Authorize] // <-- Activate Auth check for command
    // [Authorize(FieldPolicy = true)] <-- Uncommend to activate Field Auth check for command
    public class CreateWebHook : IRequest<CreateWebHookPayload> {

        public CreateWebHook() {
            this.HookEvents = new HashSet<HookEventType>();
        }

        /// <summary> Url </summary>
        public string WebHookUrl { get; set; }

        /// <summary> Secret </summary>
        public string? Secret { get; set; }

        /// <summary> IsActive </summary>
        public bool IsActive { get; set; }

        /// <summary> HookEvents </summary>
        public HashSet<HookEventType> HookEvents { get; set; }
    }

    /// <summary>
    /// CreateWebHook Validator
    /// </summary>
    public class CreateWebHookValidator : AbstractValidator<CreateWebHook> {

        private readonly IDbContextFactory<ApiDbContext> _factory;

        public CreateWebHookValidator(IDbContextFactory<ApiDbContext> factory){
            _factory = factory;

            RuleFor(e => e.WebHookUrl)
            .NotEmpty()
            .NotNull();

            RuleFor(e => e.WebHookUrl)
            .Matches(Common.URI_REGEX)
            .WithMessage("Does not match URI expression");

            RuleFor(e => e.WebHookUrl)
            .MustAsync(BeUniqueByURL)
            .WithMessage("Hook endpoint allready exist");

            RuleFor(e => e.WebHookUrl)
            .MustAsync(CheckMaxAllowedHooksCount)
            .WithMessage("Max allowed hooks count detected");

            RuleFor(e => e.HookEvents)
            .NotNull();
        }

        public async Task<bool>  BeUniqueByURL(string url, CancellationToken cancellationToken) {

            await using ApiDbContext dbContext = 
                _factory.CreateDbContext();

            return await dbContext.WebHooks.AllAsync(e => e.WebHookUrl != url);
        }

        public async Task<bool> CheckMaxAllowedHooksCount(string url, CancellationToken cancellationToken) {

            await using ApiDbContext dbContext = 
                _factory.CreateDbContext();

            const long  MAX_HOOK_COUNT = 3;

            return (await dbContext.WebHooks.CountAsync()) <= MAX_HOOK_COUNT;
        }
    }

    /// <summary>
    /// Authorization validators for CreateWebHook
    /// </summary>
    public class CreateWebHookAuthorizationValidator : AuthorizationValidator<CreateWebHook> {

        private readonly IDbContextFactory<ApiDbContext> _factory;
        public CreateWebHookAuthorizationValidator(IDbContextFactory<ApiDbContext> factory) {

            _factory = factory;

            // Add Field authorization cehcks.. (use  [Authorize(FieldPolicy = true)] to activate)
        }
    }


    /// <summary>
    /// ICreateWebHookError
    /// </summary>
    public interface ICreateWebHookError { }

    /// <summary>
    /// CreateWebHookPayload
    /// </summary>
    public class CreateWebHookPayload : BasePayload<CreateWebHookPayload, ICreateWebHookError> {

        /// <summary>
        /// Created WebHook
        /// </summary>
        public WebHook hook { get; set; }
    }

    /// <summary>Handler for <c>CreateWebHook</c> command </summary>
    public class CreateWebHookHandler : IRequestHandler<CreateWebHook, CreateWebHookPayload> {

        /// <summary>
        /// Injected <c>IDbContextFactory<ApiDbContext></c>
        /// </summary>
        private readonly IDbContextFactory<ApiDbContext> _factory;

        /// <summary>
        /// Injected <c>IPublisher</c>
        /// </summary>
        private readonly APIServer.Extensions.IPublisher _publisher;

        /// <summary>
        /// Injected <c>ICurrentUser</c>
        /// </summary>
        private readonly ICurrentUser _current;

        /// <summary>
        /// Main constructor
        /// </summary>
        public CreateWebHookHandler(
            IDbContextFactory<ApiDbContext> factory,
            APIServer.Extensions.IPublisher publisher,
            ICurrentUser currentuser) {

            _factory = factory;

            _publisher = publisher;

            _current = currentuser;
        }

        /// <summary>
        /// Command handler for <c>CreateWebHook</c>
        /// </summary>
        public async Task<CreateWebHookPayload> Handle(CreateWebHook request, CancellationToken cancellationToken) {

            await using ApiDbContext dbContext = 
                _factory.CreateDbContext();

                WebHook hook = new WebHook {
                    WebHookUrl = request.WebHookUrl,
                    Secret = request.Secret,
                    ContentType = "application/json",
                    IsActive = request.IsActive,
                    HookEvents = request.HookEvents != null ? request.HookEvents.Distinct().ToArray() : new HookEventType[0]
                };

                dbContext.WebHooks.Add(hook);

                await dbContext.SaveChangesAsync(cancellationToken);

                try {

                    await _publisher.Publish(new WebHookCreatedNotifi() {
                        ActivityId = Activity.Current.Id
                    }, PublishStrategy.ParallelNoWait, default(CancellationToken));

                } catch { }

                var response = CreateWebHookPayload.Success();

                response.hook = hook;

                return response;
        }
    }
}

As you can see, the logic of the commands (as underling under GraphQL) already prepares the request/response objects to fit well in the GraphQL layer.

This means:

Validation and Athorization checks are automatically transferred into validation and authorization errors under the MediatR behaviour.
The handler returns a payload based object that is later consumed by GraphQL.
Each command also defines a custom marker interface, such as ICreateWebHookError.

This is explained in more detail in one of the last parts GraphQL Mutation Errors aka 6a, but it is important to remember that all handlers return a payload object!

On the GraphQL layer side, we need to define WebHookMutations as:

Src/APIServer/Aplication/Graphql/Mutations/WebHook.cs

namespace APIServer.Aplication.GraphQL.Mutation {

    /// <summary>
    /// WebHooks Mutations
    /// </summary>
    [ExtendObjectType(OperationTypeNames.Mutation)]
    public class WebHookMutations {

        /// <summary>
        /// Crate new  webhook
        /// </summary>
        public class CreateWebHookInput {

            /// <summary> Url </summary>
            public string WebHookUrl { get; set; }

            /// <summary> Secret </summary>
            public string? Secret { get; set; }

            /// <summary> IsActive </summary>
            public bool IsActive { get; set; }

            /// <summary> HookEvents </summary>
            public HookEventType[] HookEvents { get; set; }
        }

        /// <summary>
        /// Create new  webhook
        /// </summary>
        /// <returns></returns>
        public async Task<CreateWebHookPayload> CreateWebHook(
            CreateWebHookInput request,
            [Service] IMediator _mediator) {

            return await _mediator.Send(new CreateWebHook() {
                WebHookUrl = request.WebHookUrl,
                Secret = request.Secret,
                IsActive = request.IsActive,
                HookEvents = request.HookEvents != null ? new HashSet<HookEventType>(request.HookEvents) : new HashSet<HookEventType>(new HookEventType[0]),
            });
        }
    }
}

As you can see, mutation == command and the GraphQL layer simply passes the input object to MediatR and takes back the payload response. This is a similar approach to calling MediatR via the API controller, except GraphQL can do some extra magic (sugar) before and after processing.

When it comes to querying, it's a little different. If we want to use the built-in pagination and filtering, we need to pass DBContex to our handler or keep the logic in the GraphQL layer. To keep the demo simple, we will query the data directly from the GraphQL layer:

[UseApiDbContextAttribute]
[UsePaging(typeof(WebHookType))]
[UseFiltering]
public IQueryable<GQL_WebHook> Webhooks(
[Service] ICurrentUser current,
[ScopedService] ApiDbContext context) {

    if (!current.Exist) {
        return null;
    }

    return context.WebHooks
        .AsNoTracking()
        .Select(e=>  new GQL_WebHook {
            ID = e.ID,
            WebHookUrl = e.WebHookUrl,
            ContentType = e.ContentType,
            IsActive = e.IsActive,
            LastTrigger = e.LastTrigger,
            ListeningEvents = e.HookEvents
        });
}

Let us take a quick look at how the mutation type is defined under the GraphQL layer:

namespace APIServer.Aplication.GraphQL.Types {
    public class CreateWebHookPayloadType : ObjectType<CreateWebHookPayload> {
        protected override void Configure(IObjectTypeDescriptor<CreateWebHookPayload> descriptor) {
            descriptor.Field(e => e.hook).Type<WebHookType>().Resolve(context => {
                WebHook e = context.Parent<CreateWebHookPayload>().hook;

                if (e == null) {
                    return null;
                }

                return new GQL_WebHook {
                    ID = e.ID,
                    WebHookUrl = e.WebHookUrl,
                    ContentType = e.ContentType,
                    IsActive = e.IsActive,
                    LastTrigger = e.LastTrigger,
                    ListeningEvents = e.HookEvents
                };
            });
        }
    }

    public class CreateWebHookErrorUnion : UnionType<ICreateWebHookError> {
        protected override void Configure(IUnionTypeDescriptor descriptor) {
            descriptor.Type<ValidationErrorType>();
            descriptor.Type<UnAuthorisedType>();
            descriptor.Type<InternalServerErrorType>();
        }
    }
}

From the definition, we can see that this mutation of MediatR returns CreateWebHookPayload with the field hook. We can extend or override any value with GraphQL and this is the right place to change the response or define how additional data should be resolved.

Another important point is CreateWebHookErrorUnion which defines that the command can return 3 different errors as unions. They are available under payload Error[]. This is also explained in more detail in the part GraphQL mutation errors aka 6a.

It's a similar story for GraphQL queries. Let us take a look at WebHookType:

namespace APIServer.Aplication.GraphQL.Types {

    /// <summary> Graphql WebHookType </summary>
    public class WebHookType : ObjectType<GQL_WebHook> {

        protected override void Configure(IObjectTypeDescriptor<GQL_WebHook> descriptor) {

            descriptor.AsNode().IdField(t => t.ID).NodeResolver((ctx, id) =>
                ctx.DataLoader<WebHookByIdDataLoader>().LoadAsync(id, ctx.RequestAborted));

            descriptor.Field(t => t.ID).Type<NonNullType<IdType>>();

            descriptor.Field("systemid").Type<NonNullType<LongType>>().Resolve((IResolverContext context) => {
                return context.Parent<GQL_WebHook>().ID;
            });

            descriptor.Field(t => t.Records)
            .UseDbContext<ApiDbContext>()
            .Resolve(async  ctx => {

                ApiDbContext _context = ctx.Service<ApiDbContext>();

                ICurrentUser _current = ctx.Service<ICurrentUser>();

                long hook_id = ctx.Parent<GQL_WebHook>().ID;

                if (!_current.Exist) {
                    return null;
                }

                if (hook_id <= 0) {
                    return new List<GQL_WebHookRecord>().AsQueryable();
                }

                return _context.WebHooksHistory
                .AsNoTracking()
                .Where(e => e.WebHookID == hook_id)
                .Select(e => new GQL_WebHookRecord() {
                    ID = e.ID,
                    WebHookID = e.WebHookID,
                    WebHookSystemID = e.WebHookID,
                    StatusCode = e.StatusCode,
                    ResponseBody = e.ResponseBody,
                    RequestBody = e.RequestBody,
                    TriggerType = e.HookType,
                    Result = e.Result,
                    Guid = e.Guid,
                    RequestHeaders = e.RequestHeaders,
                    Exception = e.Exception,
                    Timestamp = e.Timestamp,
                }!).OrderByDescending(e => e.Timestamp);

            })
            .UsePaging<WebHookRecordType>()
            .UseFiltering();
        }
    }
}

This ObjectType uses a DTO object and defines how each field should be resolved. Some fields are resolved by Parent (our top query) and other fields contain custom resolution logic like the Records field.

Since Hotchocolate GraphQL Server follows the Relay specification, the Relay Global Id and Node interfaces are implemented. The corresponding definition can be found under WebHookType.

This allows us to query any object that implements this using the node query.

descriptor.AsNode().IdField(t => t.ID).NodeResolver((ctx, id) =>
    ctx.DataLoader<WebHookByIdDataLoader>().LoadAsync(id, ctx.RequestAborted));

Where DataLoader<WebHookByIdDataLoader> is :

Src/APIServer/Aplication/Graphql/Dataloaders/WebHooks/WebHookById_DataLoader.cs

namespace APIServer.Aplication.GraphQL.DataLoaders {

    public class WebHookByIdDataLoader : BatchDataLoader<long, GQL_WebHook> {
        /// <summary> Injected <c>ApiDbContext</c> </summary>
        private readonly IDbContextFactory<ApiDbContext> _factory;

        /// <summary> Injected <c>ICurrentUser</c> </summary>
        private readonly ICurrentUser _current;

        private readonly SemaphoreSlim _semaphoregate = new SemaphoreSlim(1);

        public WebHookByIdDataLoader(
            IBatchScheduler scheduler,
            IDbContextFactory<ApiDbContext> factory,
            ICurrentUser current) : base(scheduler) {
            _current = current;
            _factory = factory;
        }

        protected override async Task<IReadOnlyDictionary<long, GQL_WebHook>> LoadBatchAsync(
            IReadOnlyList<long> keys,
            CancellationToken cancellationToken) {

            if (!_current.Exist) {
                return new List<GQL_WebHook>().ToDictionary(e => e.ID, null);
            }

            await using ApiDbContext dbContext = 
                _factory.CreateDbContext();

            await _semaphoregate.WaitAsync();

            try {
                return await dbContext.WebHooks
                .AsNoTracking()
                .Where(s => keys.Contains(s.ID))
                .Select(e => new GQL_WebHook {
                    ID = e.ID,
                    WebHookUrl = e.WebHookUrl,
                    ContentType = e.ContentType,
                    IsActive = e.IsActive,
                    LastTrigger = e.LastTrigger,
                    ListeningEvents = e.HookEvents
                })
                .ToDictionaryAsync(t => t.ID, cancellationToken);
            } finally {
                _semaphoregate.Release();
            }
        }
    }
}

Dataloaders are optimization techniques. They are not GraphQL specific and can be part of an application/business logic outside of GraphQl. We use them to solve the N+1 problem, i.e., resolving similar data with different IDs in the same time.

The advantages and disadvantages + benchmarks for dataloaders compared to the traditional method are explained in a separate part, where you can go in depth and understand how to design your scheme. This part contains only a basic introduction to backend logic, so do not worry if you do not understand the topic well.

WebHook notification and processing

We have already seen how mutation is transfered to the MediatR command and how the result is returned, and now we can quickly and easily set up how webhooks are processed. This is internal functionality and this command should not be publicly available.

You can find all internal commands at: Src/APIServer/Apliaction/Commands/Internal.

To trigger internal notifications, the MediatR Publisher is used:

await _publisher.Publish(new WebHookCreatedNotifi() {
    ActivityId = Activity.Current.Id
}, PublishStrategy.ParallelNoWait, default(CancellationToken));

These are internal notifications that are handled at the user's request. It is good to keep them fast to collect only context data and pass the heavy processing to the scheduler (background worker) with enque.

This is the MediatR extension:

public string Enqueue(IRequest request, string parentJobId, JobContinuationOptions continuationOption, string description = null) {
    var mediatorSerializedObject = SerializeObject(request, description);

    return BackgroundJob.ContinueJobWith(parentJobId, () => _commandsExecutor.ExecuteCommand(mediatorSerializedObject), continuationOption);
}

This is an internal handler to enque all webhook listeners:


namespace APIServer.Aplication.Commands.Internall.Hooks  {

    /// <summary>
    /// Command for processing WebHook event
    /// </summary>
    public class EnqueueRelatedWebHooks : CommandBase {
        public HookEventType EventType { get; set; }
        public object Event { get; set; }
    }

    /// <summary>
    /// Command handler for <c>EnqueueRelatedWebHooks</c>
    /// </summary>
    public class EnqueueRelatedWebHooksHandler : IRequestHandler<EnqueueRelatedWebHooks, Unit> {

        /// <summary> Injected <c>ApiDbContext</c> </summary>
        private readonly IDbContextFactory<ApiDbContext> _factory;

        /// <summary> Injected <c>IMediator</c> </summary>
        private readonly IMediator _mediator;

        /// <summary> Injected <c>IHttpClientFactory</c> </summary>
        private readonly IHttpClientFactory _clientFactory;

        /// <summary> Main Constructor </summary>
        public EnqueueRelatedWebHooksHandler(
            IDbContextFactory<ApiDbContext> factory,
            IMediator mediator,
            IHttpClientFactory httpClient) {
            _factory = factory;
            _mediator = mediator;
            _clientFactory = httpClient;
        }

        /// <summary> Command handler for  <c>EnqueueRelatedWebHooks</c> </summary>
        public async Task<Unit> Handle(EnqueueRelatedWebHooks request, CancellationToken cancellationToken) {

            if (request == null || request.Event == null) {
                throw new ArgumentNullException();
            }

            await using ApiDbContext dbContext = 
                _factory.CreateDbContext();

            List<WebHook> hooks = await dbContext.WebHooks
            .AsNoTracking()
            .Where(e => e.HookEvents.Contains(HookEventType.hook))
            .ToListAsync(cancellationToken);

            if (hooks != null) {
                foreach (var hook_item in hooks) {
                    if (hook_item.IsActive && hook_item.ID > 0) {
                        try {
                            _mediator.Enqueue(new ProcessWebHook() {
                                HookId = hook_item.ID,
                                Event = request.Event,
                                EventType = request.EventType
                            });
                        } catch { }
                    }
                }
            }

            return Unit.Value;
        }
    }
}

And the webhook processing may look like this:

namespace APIServer.Aplication.Commands.Internall.Hooks {

    /// <summary> Command for processing WebHook event </summary>
    public class ProcessWebHook : CommandBase {
        public long HookId { get; set; }
        public dynamic Event { get; set; }
        public HookEventType EventType { get; set; }
    }

    /// <summary> Command handler for <c>ProcessWebHook</c> </summary>
    public class ProcessWebHookHandler : IRequestHandler<ProcessWebHook, Unit> {

        /// <summary> Injected <c>ApiDbContext</c> </summary>
        private readonly IDbContextFactory<ApiDbContext> _factory;

        /// <summary> Injected <c>IMediator</c> </summary>
        private readonly IMediator _mediator;

        /// <summary> Injected <c>IHttpClientFactory</c> </summary>
        private readonly IHttpClientFactory _clientFactory;

        /// <summary> Main Constructor </summary>
        public ProcessWebHookHandler(
            IDbContextFactory<ApiDbContext> factory,
            IMediator mediator,
            IHttpClientFactory httpClient) {
             _factory = factory;
            _mediator = mediator;
            _clientFactory = httpClient;
        }

        /// <summary> Command handler for  <c>ProcessWebHook</c> </summary>
        public async Task<Unit> Handle(ProcessWebHook request, CancellationToken cancellationToken) {

            WebHookRecord record = new WebHookRecord() {
                WebHookID = request.HookId,
                Guid = Guid.NewGuid().ToString(),
                HookType = request.EventType,
                Timestamp = DateTime.Now
            };

            if (request == null || request.HookId <= 0) {
                record.Result = RecordResult.parameter_error;
            }

            await using ApiDbContext dbContext = 
                _factory.CreateDbContext();

            try {

                WebHook hook = null;

                try {
                    hook = await dbContext.WebHooks
                        .Where(e => e.ID == request.HookId)
                        .FirstOrDefaultAsync(cancellationToken);

                } catch (Exception ex) {
                    record.Result = RecordResult.dataQueryError;
                    record.Exception = ex.ToString();

                    return Unit.Value;
                }

                if (hook != null) {

                    var options = new JsonSerializerOptions {
                        WriteIndented = true,
                        IncludeFields = true,
                    };

                    var serialised_request_body = new StringContent(
                          JsonSerializer.Serialize<dynamic>(request.Event, options),
                          Encoding.UTF8,
                          "application/json");


                    var httpClient = _clientFactory.CreateClient();

                    /* Set Headers */
                    httpClient.DefaultRequestHeaders.Add("X-Trouble-Delivery", record.Guid);

                    if (!string.IsNullOrWhiteSpace(hook.Secret)) {
                        httpClient.DefaultRequestHeaders.Add("X-Trouble-Secret", hook.Secret);
                    }

                    httpClient.DefaultRequestHeaders.Add("X-Trouble-Event", request.EventType.ToString().ToLowerInvariant());

                    record.RequestBody = await serialised_request_body.ReadAsStringAsync(cancellationToken);

                    var serialized_headers = new StringContent(
                                     JsonSerializer.Serialize(httpClient.DefaultRequestHeaders.ToList(), options),
                                     Encoding.UTF8,
                                     "application/json");

                    record.RequestHeaders = await serialized_headers.ReadAsStringAsync(cancellationToken);

                    if (!string.IsNullOrWhiteSpace(hook.WebHookUrl)) {
                        try {
                            using var httpResponse = await httpClient.PostAsync(hook.WebHookUrl, serialised_request_body);

                            if (httpResponse != null) {
                                record.StatusCode = (int)httpResponse.StatusCode;

                                if (httpResponse.Content != null) {
                                    record.ResponseBody = await httpResponse.Content.ReadAsStringAsync(cancellationToken);
                                }
                            }

                            record.Result = RecordResult.ok;
                        } catch (Exception ex) {
                            record.Result = RecordResult.http_error;
                            record.Exception = ex.ToString();
                        }
                    } else {
                        record.Result = RecordResult.parameter_error;
                    }
                } else {
                    record.Result = RecordResult.parameter_error;
                }

            } finally {

                try {
                    dbContext.WebHooksHistory.Add(record);
                    await dbContext.SaveChangesAsync(cancellationToken);
                } catch { }

            }

            return Unit.Value;
        }
    }
}

API Layer

This is where the API microservice is configured. This depends on the application and persistence layer.

The Startup.cs class configures the host application using DI setup and middelware pipeline. This is a demo configuration:

Src/APIServer/API/Startup.cs

namespace APIServer
{
    public class Startup
    {
        public Startup(IConfiguration configuration, IWebHostEnvironment enviroment)
        {
            Configuration = configuration;

            Environment = enviroment;
        }

        public IWebHostEnvironment Environment { get; }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddControllers();

            services.AddSwaggerGen(c =>{
                c.SwaggerDoc("v1", new OpenApiInfo { Title = "Api", Version = "v1" });
            });

            services.AddAuth(Configuration);

            services.AddDbContext(Configuration,Environment);

            services.AddHttpClient();

            services.AddHealthChecks();

            services.AddGraphql(Environment);

            services.AddHttpContextAccessor();

            services.AddMemoryCache();

            services.AddTelemerty(Configuration,Environment);

            services.AddScoped<ICurrentUser, CurrentUser>();

            services.AddMediatR();

            services.AddScheduler(Configuration);

            services.AddSingleton(Serilog.Log.Logger);
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(
            IApplicationBuilder app,
             IWebHostEnvironment env,
             IServiceProvider serviceProvider,
             IServiceScopeFactory scopeFactory)
        {
            app.UseForwardedHeaders(new ForwardedHeadersOptions{
                ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto | ForwardedHeaders.XForwardedHost,
            });

            app.UseEnsureApiContextCreated(serviceProvider,scopeFactory);

            if (env.IsDevelopment()) {
                app.UseDeveloperExceptionPage();
                app.UseSwagger();
                app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "Api v1"));
            }

            app.UseHealthChecks("/health");

            app.UseHttpsRedirection();

            app.UseRouting();

            app.UseAuthentication();

            app.UseAuthorization();

            app.UseHangfireServer();

            if(env.IsDevelopment()){
                app.UseHangfireDashboard("/scheduler");
            }

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers()
                    .RequireAuthorization("ApiCaller");

                endpoints.MapGraphQL()
                .WithOptions(new GraphQLServerOptions {
                    EnableSchemaRequests = env.IsDevelopment(),
                    Tool = { Enable = env.IsDevelopment() },
                });

                endpoints.MapControllerRoute(
                    name: "default",
                    pattern: "{controller}/{action=Index}/{id?}");
            });
        }
    }
}

Since the whole configuration is explained in separate parts for logging, tracing, monitoring and gql errors, I will not go into it here on a daily basis and just provide the most important parts. To see all configurations, please follow the sources at the end of this article!

The configuration of the GraphQL part consists of:

namespace APIServer.Configuration {
    public static partial class ServiceExtension {
        public static IServiceCollection AddGraphql(
            this IServiceCollection serviceCollection, IWebHostEnvironment env) {

            serviceCollection.AddGraphQLServer()
                    .SetPagingOptions(
                        new PagingOptions { IncludeTotalCount = true, MaxPageSize = 200 })
                    .ModifyRequestOptions(requestExecutorOptions => {
                        if (env.IsDevelopment() ||
                            System.Diagnostics.Debugger.IsAttached) {
                            requestExecutorOptions.ExecutionTimeout = TimeSpan.FromMinutes(1);
                        }

                         requestExecutorOptions.IncludeExceptionDetails = !env.IsProduction();
                    })

                    .AddGlobalObjectIdentification()
                    .AddQueryFieldToMutationPayloads()

                    .AddFiltering()
                    .AddSorting()

                    .AddQueryType<Query>()
                        .AddTypeExtension<WebHookQueries>()
                        .AddTypeExtension<UserQueries>()
                        .AddTypeExtension<SystemQueries>()
                    .AddMutationType<Mutation>()
                        .AddTypeExtension<WebHookMutations>()

                    .BindRuntimeType<DateTime, DateTimeType>()
                    .BindRuntimeType<int, IntType>()

                    .AddType<BadRequestType>()
                    .AddType<InternalServerErrorType>()
                    .AddType<UnAuthorisedType>()
                    .AddType<ValidationErrorType>()
                    .AddType<BaseErrorType>()
                    .AddType<UserDeactivatedType>()
                    .AddType<BaseErrorInterfaceType>()
                    .AddType<WebHookNotFoundType>()
                    .AddType<WebHookRecordType>()
                    .AddType<WebHookType>()
                    .AddType<UserType>()
                    .AddType<UpdateWebHookUriPayloadType>()
                    .AddType<UpdateWebHookUriErrorUnion>()
                    .AddType<UpdateWebHookTriggerEventsPayloadType>()
                    .AddType<UpdateWebHookTriggerEventsErrorUnion>()
                    .AddType<UpdateWebHookSecretPayloadType>()
                    .AddType<UpdateWebHookSecretErrorUnion>()
                    .AddType<UpdateWebHookPayloadPayloadType>()
                    .AddType<UpdateWebHookPayloadErrorUnion>()
                    .AddType<UpdateWebHookActivStatePayloadType>()
                    .AddType<UpdateWebHookActivStateErrorUnion>()
                    .AddType<RemoveWebHookPayloadType>()
                    .AddType<RemoveWebHookErrorUnion>()
                    .AddType<CreateWebHookPayloadType>()
                    .AddType<CreateWebHookErrorUnion>()

                    .AddDataLoader<UserByIdDataLoader>()
                    .AddDataLoader<WebHookByIdDataLoader>()
                    .AddDataLoader<WebHookRecordByIdDataLoader>()

                    .UsePersistedQueryPipeline()
                    .UseReadPersistedQuery()
                    .AddReadOnlyFileSystemQueryStorage("./persisted_queries");

            return serviceCollection;
        }
    }
}

MediatR configuration:

namespace APIServer.Configuration {
    public static partial class ServiceExtension {
        public static IServiceCollection AddMediatR(this IServiceCollection services) {

            // Command executor
            services.AddMediatR(cfg => cfg.Using<AppMediator>(), typeof(CreateWebHook).GetTypeInfo().Assembly);

            services.AddTransient<APIServer.Extensions.IPublisher, APIServer.Extensions.Publisher>();

            services.AddValidatorsFromAssembly(typeof(CreateWebHookValidator).GetTypeInfo().Assembly);

            services.AddTransient<IRequestHandler<EnqueSaveEvent<WebHookCreated>, Unit>, EnqueSaveEventHandler<WebHookCreated>>();

            services.AddTransient<IRequestHandler<EnqueSaveEvent<WebHookUpdated>, Unit>, EnqueSaveEventHandler<WebHookUpdated>>();

            services.AddTransient<IRequestHandler<EnqueSaveEvent<WebHookRemoved>, Unit>, EnqueSaveEventHandler<WebHookRemoved>>();

            services.AddMediatRSchedulerIntegration();

            services.AddTransient(typeof(IPipelineBehavior<,>), typeof(TracingBehaviour<,>));

            services.AddTransient(typeof(IPipelineBehavior<,>), typeof(CommandPerformanceBehaviour<,>));

            services.AddTransient(typeof(IPipelineBehavior<,>), typeof(ValidationBehaviour<,>));

            services.AddTransient(typeof(IPipelineBehavior<,>), typeof(AuthorizationBehaviour<,>));

            services.AddTransient(typeof(IPipelineBehavior<,>), typeof(UnhandledExBehaviour<,>));

            return services;
        }
    }
}

Where AppMediator is:


namespace APIServer.Extensions {

    public class AppMediator : Mediator {

        private Func<IEnumerable<Func<INotification, CancellationToken, Task>>, INotification, CancellationToken, Task> _publishStrategy;

        public AppMediator(
            ServiceFactory serviceFactory
            ) : base(serviceFactory) {
        }

        public AppMediator(
            ServiceFactory serviceFactory,
            Func<IEnumerable<Func<INotification, CancellationToken, Task>>, INotification, CancellationToken, Task>? publishStrategy
            ) : base(serviceFactory) {

            _publishStrategy = publishStrategy != null ? publishStrategy : SyncStopOnException;
        }

        public Task<TResponse> Send<TResponse>(ICommandBase<TResponse> request, CancellationToken cancellationToken = default) {
            return base.Send<TResponse>(request, cancellationToken);
        }

        public Task<object?> Send(ICommandBase request, CancellationToken cancellationToken = default) {
            return base.Send(request as object, cancellationToken);
        }

        private static async Task SyncStopOnException(IEnumerable<Func<INotification, CancellationToken, Task>> handlers, INotification notification, CancellationToken cancellationToken) {
            foreach (var handler in handlers) {
                await handler(notification, cancellationToken).ConfigureAwait(false);
            }
        }

        protected override Task PublishCore(IEnumerable<Func<INotification, CancellationToken, Task>> allHandlers, INotification notification, CancellationToken cancellationToken = default) {

            Activity activity = null;

            if (notification is INotificationBase) {

                INotificationBase I_base_notify = notification as INotificationBase;

                // If any activity is in context will be set as default in case value of ActivityId == null
                if (I_base_notify.ActivityId == null
                    && Activity.Current != null
                    && Activity.Current.Id != null) {
                    I_base_notify.ActivityId = Activity.Current.Id;
                }

                activity = Sources.DemoSource.StartActivity(
                    String.Format("PublishCore: Notification<{0}>", notification.GetType().FullName), ActivityKind.Producer);

                // This chane activity parrent / children relation..
                if (I_base_notify.ActivityId != null
                     && Activity.Current != null
                     && Activity.Current.ParentId == null) {
                    Activity.Current.SetParentId(I_base_notify.ActivityId);
                }

                if (Activity.Current != null
                    && Activity.Current.ParentId != null) {
                    Activity.Current.AddTag("Parrent Id", Activity.Current.ParentId);
                }

            } else {
                activity = Sources.DemoSource.StartActivity(
                    String.Format("PublishCore: Notification<{0}>", notification.GetType().FullName), ActivityKind.Producer);
            }

            try {
                Activity.Current.AddTag("Activity Id", Activity.Current.Id);

                activity.Start();

                return _publishStrategy != null ? _publishStrategy(allHandlers, notification, cancellationToken) : base.PublishCore(allHandlers, notification, cancellationToken);

            } finally {
                activity.Stop();
                activity.Dispose();
            }
        }
    }
}

UI interface

The detailed integration of React UI is explained in separate materials. This is the current demo showcase:

Repository

You can find the full source code of the app, including identity, distributed logging, tracing and monitoring, in the open source Github repo:

https://github.com/damikun/trouble-training

GraphQL mutation errors (union errors with interface aka 6a) with Hotchocolate GraphQL Server

Dalibor Kundrat — Wed, 06 Oct 2021 04:26:22 +0000

Intro

Dealing with errors in Graphql is a topic on which there are numerous articles and recommendations, but there are not many real-world examples of how to implement a concrete approach under a given language and GraphQL server.

This article tries to show the concrete implementation of stage 6a mutation error handling from Marc-Andre Giroux Guide to graphql errors under Hotchocolate (.Net) GraphQL Server including some base integration with MediatR.

These are the advantages of the 6a approach:
✅Pro: Expressive and Discoverable Schema
✅Pro: Support for Multiple Errors
✅Pro: Easier Evolution

Before we begin, let me show you an example from the playground of 6a error. The pattern allows you to choose whether to use one concrete type as the union or group them all together using the common IBaseError interface:

⠀
NOTE: Please make sure you have already read his article, because my article does not explain the advantages or disadvantages of the different approaches. I only take the most recommended approach (6a) and show the implementation under Hotchocolate GraphQL Server.
⠀

Content

GraphQL Errors
Mutation Errors
Stage 6a Integration
- Abstraction
  - Validation errors
  - Authorization errors
- Hotchocolate integration
Demo UI
Repository

⠀
The full demo sources with frontend and backend can be found at the end of this article!
⠀

GraphQL Errors

Before I explain the specific mutation error handling and schema design, let us first understand what kinds of errors we may face in Fronted. To explain this, think of the errors as different groups with different purposes. I have classified them as follows:

Network Errors - An error probably occurred during the transfer and the endpoint was not reached, or a serious problem occurred before execution began. This error may be returned, for example, if the service is down. The correct error code is returned via the http status code. This does not have much to do with graphql and is more related to the transport or services (proxies) before the API. In this scenario, you should probably display the error page with the option to retry the operation at UI or automatically retrigger it, as it could be a temporary connection problem.
GraphQL Top-Level Errors - (syntax, validation, and execution errors). The Graphql server response status code is OK200 because the request was processed and the specific error is returned with Graphql Error[] in the Graphql response. Depending on the error type, you should react differently. These errors are usually developer errors, which you generally do not want to show to users as they are not very helpful as they provide developer-related context to the problem. They are important if you have public APIs or want to test your Graphql server interactively, as they provide information about missing fields, wrong types or errors in execution.
- Syntax - incorrect query syntax.
- Validation - query includes field that does not exist
- Execution errors - exceptions, timeouts, etc.
Mutation Errors - User-defined errors for concrete mutations (mutation-specific errors). It is best to group this erros in Error[] field in the payload of each mutation. This can be done using interfaces or unions. The network status code is always OK200.
Query Error - A query usually asks for several pieces of data, and some parts may possibly fail. There are the following options on how to handle these failed parts (errors).
- Errors can be grouped in the default graphql response Error[] or in a separate custom field for them. The network status code is always OK200.
- Using unios and interfaces so that a concrete field can return not only the response type, but also multiple errors as separate types. This must be handled at the Qurey level in your client applications. The network status code is always OK200.

Mutation Errors

Each mutation can return multiple errors. For example, the most common errors you can get back are:

ValidationError - if some fields in the InputObjects type are invalid.
AuthorizationError - if you do not have permission to trigger a specific side-effect or access a resource defined in the InputObject.
InternalServerError - something bad happened during the mutation process.

You can also specify any other contextual error, such as UserDeactivated or PostIsNotPublished, depending on the state of the object in your system. This way, you can control the user experience in UI and respond differently depending on the error type returned. And each error type can contain different fields.

⠀
NOTE: To learn more about the different approaches to dealing with mutation errors, please read the article by Marc-Andre Giroux Guide to graphql errors before reading on. This should help you understand the advantages and disadvantages of each option he presents.
⠀

These are the theoretical concepts and in this article I will show you the basic implementation of the most recommended pattern 6a using unions and interfaces.

Stage 6a Integration

There are some packages you need to use with my demo. The main ones are:

Hotchocolate GraphQL server. This is the most advanced server for a .Net environment. This article does not go into the basics and assumes some framework knowledge.
MediatR as a handler for the application logic, leaving the mutations a thin layer on top. This can be helpful if you share your business logic with more than just GraphQL.
FluentValidator for validating and authorising certain fields.

Let me go back to Marc's example of the 6a schema from the article I mentioned:

type CreateUserPayload {
  user: User
  Errors: [CreateUserError!]!
}

union CreateUserError = UserNameTaken | UnAuthorised | MustBeOver18

type UserNameTaken implements IBaseError {
  message: String!
  path: String!
  suggestion: String!
}

interface IBaseError {
  message: String!
  path: String!
}

These are 3 important points from schema definition

Mutation returns an object called a payload (CreateUserPayload).
Each payload contains the Errors[] - (union error array).
Each error type implements the common interface IBaseError.

Abstraction

Since MediatR is used to fire and process the commands, you can create a generic abstraction that can help you with authorization and validation and provide a PayloadBase abstraction.

A few lines below is an example of application command CreateUserand these are parts of that file:

Command definition - Command itself
Command Authorization Rules - rules for authorization before the command is processed.
Command validation rules - rules for validating fields before processing the command
Command handler - logic of the command handler.

As you can see, everything related to a command is in a file. This helps with orientation if your system has multiple commands to keep the context in one place.

Src/Aplication/Commands/CreateUser.cs

namespace ErrorHandling.Aplication.Commands {

    // [Authorize] // <-- Uncommend to activate Auth check for command
    // [Authorize(FieldPolicy = true)] <-- Uncommend to activate Field Auth check for command
    public class CreateUser : IRequest<CreateUserPayload> {

        public string NickName {get; set;}

        public int Age {get; set;}
    }

    /// <summary>  CreateUser field Authorization </summary>
    public class CreateUserAuthorizationValidator : AuthorizationValidator<CreateUser> {

        private readonly IDbContextFactory<AppDbContext> _factory;
        public CreateUserAuthorizationValidator(IDbContextFactory<AppDbContext> factory) {

            _factory = factory;

            // Add Field authorization cehcks.. (use  [Authorize(FieldPolicy = true)] to activate)
        }
    }

    /// <summary> CreateUser Validator </summary>
    public class CreateUserValidator : AbstractValidator<CreateUser> {

        private readonly IDbContextFactory<AppDbContext> _factory;

        public CreateUserValidator(
            IDbContextFactory<AppDbContext> factory){

            _factory = factory;

            RuleFor(e => e.NickName)
            .NotEmpty()
            .NotNull();

            RuleFor(e => e.Age)
            .GreaterThan(18)
            .LessThan(100)
            .WithMessage("The agemust be between 18-100"); // Oh sorry Grandma :)

            RuleFor(e=>e.NickName)
            .MustAsync(HasUniqueName)
            .WithMessage("Nickname must be unique");
        }

        public async Task<bool> HasUniqueName(string name, CancellationToken cancellationToken) {

            await using AppDbContext dbContext = 
                _factory.CreateDbContext();

            return await dbContext.Users.AllAsync(e => e.NickName != name);
        }
    }

     /// <summary> ICreateUserError </summary>
    public interface ICreateUserError { }

    // CreateUserPayload
    public class CreateUserPayload : BasePayload<CreateUserPayload, ICreateUserError> {
        public GQL_User user {get;set;}

        // And any other return field ...
    }

    /// <summary>Handler for <c>CreateUser</c> command </summary>
    public class CreateUserHandler : IRequestHandler<CreateUser, CreateUserPayload> {

        /// <summary>  Injected <c>IDbContextFactory</c> </summary>
        private readonly IDbContextFactory<AppDbContext> _factory;

        /// <summary> Main constructor </summary>
        public CreateUserHandler(
             IDbContextFactory<AppDbContext> factory) {

            _factory = factory;
        }

        /// <summary> Command handler for <c>CreateUser</c>  </summary>
        public async Task<CreateUserPayload> Handle(CreateUser request, CancellationToken cancellationToken) {

            await using AppDbContext dbContext = 
                _factory.CreateDbContext();

            User new_user = new User(){
                    NickName = request.NickName,
                    Age = request.Age
            };

            dbContext.Users.Add(new_user);

            await dbContext.SaveChangesAsync(cancellationToken);

            if(!string.IsNullOrWhiteSpace(new_user.Guid)){
                var payload = CreateUserPayload.Success();

                payload.user = new GQL_User(){
                    Guid = new_user.Guid,
                    NickName = new_user.NickName,
                    Age = new_user.Age
                };

                return payload;
            }else{
                return CreateUserPayload.Error(
                    new InternalServerError("Failed to create new user"));
            }
        }
    }
}

Notes:

As you can see in the example, the command contains the payload object CreateUserPayload : BasePayload<CreateUserPayload, ICreateUserError>.
This payload object is derived from BasePayload class. This base class provides support for the Union Error Array, which is required for the Stage 6a error pattern.
The command itself can additionally contain any other fields.

The BasePayload<U, T> implementation:

/Src/Aplication/Core/PayloadBase.cs

namespace ErrorHandling.Aplication.Payload  {

    public abstract class BasePayload<U, T> : IBasePayload where U : BasePayload<U, T>, new() {

        public BasePayload() {
            this.errors = new List<T>();
        }

        ///List of possible union errors
        public List<T> errors { get; set; }

        // Add errors collection and return itself
        [GraphQLIgnore]
        public U PushError(params T[] errors) {
            this.errors.AddRange(errors);

            return (U)this;
        }

        // Return new instance with errors
        /// <param name="errors"></param>
        [GraphQLIgnore]
        public static U Error(params T[] errors) {
            U u = new U();
            u.errors.AddRange(errors);
            return u;
        }

        // Returns new instance
        [GraphQLIgnore]
        public static U Success() {
            return new U();
        }

        [GraphQLIgnore]
        public void AddError(object o) {

            if (o is T) {
                T tmp = (T)o;
                this.errors.Add(tmp);
            } else {
                throw new NotSupportedException("Error type does not match base payload supported types");
            }
        }
    }

}

The BasePayload contains two generic parameters.:

T - The type of command to which the payload belongs.
U - The union marker interface, which contains no methods or constants. It provides type information about objects at runtime. This is used to interpret union in typed languages like C#, since there is no native support for unions.

The BasePayload implements IBasePayload. This ensures that all commands support the most important features that help you add errors!

Payload base interface:

/Src/Aplication/Core/PayloadBase.cs

  public interface IBasePayload { // <-- Base payload interface
      void AddError(object o);
  }

All payloads allow you to add some errors to it. The errors are defined in a grouped file. This is an example of an error definition. (For large projects, you can put it in a subclass depending on the context).

/Src/Aplication/Graphql/Errors/GlobalErrors.cs


public class UnAuthorised : BaseError, ICreateUserError, IRemoveUserError {
    public UnAuthorised() {
        this.message = "Unauthorised to process or access resource";
    }

    public UnAuthorised(string s) {

        this.message = s;
    }

    public UnAuthorised(object content, string message) {

        this.message = message;
    }
}

public class InternalServerError : BaseError, ICreateUserError, IRemoveUserError {

    public InternalServerError() {
        this.message = "Internal server error";
    }

    public InternalServerError(string s) {
        this.message = s;
    }
}

public class ValidationError : BaseError, ICreateUserError, IRemoveUserError {
    public ValidationError() {
        this.message = "Some parameter/s (fields) are invalid";
    }

    public ValidationError(string s) {
        this.message = s;
    }

    public ValidationError(string propName, string message) {
        this.message = message;
        this.FieldName = propName;
    }
    #nullable enable
    public string? FieldName { get; set; }
    #nullable disable
}

As you can see each error is derived from the BaseError class

/Src/Aplication/Core/Errors/BaseError.cs

namespace ErrorHandling.Aplication.GraphQL.Errors {
    public class BaseError : IBaseError {
            public string message { get; set; }
    }
}

And this class implements the common error interface IBaseError, which allows you to simply search for common fields like message instead of specifying a concrete type.

/Src/Aplication/Core/Errors/IBaseError.cs

namespace ErrorHandling.Aplication.GraphQL.Interfaces {

    public interface IBaseError {
        string message { get; set; }
    }
}

MediatR supports the integration of custom pipelines. This allows you to create custom validation and authorization handlers for each command triggered by the mediator. This helps us to create a small abstraction and reuse the authorization and validation logic for all commands.

This also allows the authentication and validation errors to be handled automatically by the pipeline.

Validation errors

The validation behavior selects all validators available for the specific command and performs the validation process.

If an error occurs during validation and the request (in this case the command) is derived from BasePayload, it automatically adds new ValidationError("message") to the Error[] of the command payload.
If the request is not a command but a query, it throws a ValidationException which is handled by the Graphql error filter.

You can customize this logic to your needs!

/Src/Aplication/Core/Behaviours/ValidationBehaviour.cs

namespace ErrorHandling.Aplication.Shared.Behaviours {

    /// Validation behaviour for MediatR pipeline
    public class ValidationBehaviour<TRequest, TResponse> : IPipelineBehavior<TRequest, TResponse> {
        private readonly IEnumerable<IValidator<TRequest>> _validators;
        private readonly ILogger _logger;

        public ValidationBehaviour(
            ICurrentUser currentUserService,
            IEnumerable<IValidator<TRequest>> validators,
             ILogger logger) {
            _validators = validators;
            _logger = logger;
        }

        public async Task<TResponse> Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate<TResponse> next) {

            if (_validators.Any()) {
                var activity = Sources.DemoSource.StartActivity(
                        String.Format("ValidationBehaviour: Request<{0}>", request.GetType().FullName), ActivityKind.Server);

                try {

                    activity.Start();
                    var context = new ValidationContext<TRequest>(request);

                    var validationResults = await Task.WhenAll(
                        _validators.Where(v => !(v is IAuthorizationValidator))
                        .Select(v => v.ValidateAsync(context, cancellationToken)));

                    var failures = validationResults
                    .SelectMany(r => r.Errors)
                    .Where(f => f != null)
                    .ToList();

                    if (failures.Count != 0) {
                        return HandleValidationErrors(failures);
                    }

                } catch (Exception ex) {

                    SharedCore.Aplication.Shared.Common.CheckAndSetOtelExceptionError(ex,_logger);

                    // In case it is Mutation Response Payload = handled as payload error union
                    if (SharedCore.Aplication.Shared.Common.IsSubclassOfRawGeneric(typeof(BasePayload<,>), typeof(TResponse))) {
                        return Common.HandleBaseCommandException<TResponse>(ex);
                    } else {
                        throw;
                    }

                } finally {
                    activity.Stop();
                    activity.Dispose();
                }
            }

            return await next();
        }

        private static TResponse HandleValidationErrors(List<ValidationFailure> error_obj) {

            // In case it is Mutation Response Payload = handled as payload error union
            if (SharedCore.Aplication.Shared.Common.IsSubclassOfRawGeneric(typeof(BasePayload<,>), typeof(TResponse))) {
                IBasePayload payload = ((IBasePayload)Activator.CreateInstance<TResponse>());

                foreach (var item in error_obj) {

                    payload.AddError(new ValidationError(item.PropertyName, item.ErrorMessage));
                }

                return (TResponse)payload;
            } else {

                if (error_obj != null) {

                    var first_item = error_obj.First();
                    if (first_item != null) {
                        throw new SharedCore.Aplication.Shared.Exceptions.ValidationException(string.Format("Field: {0} - {1}", first_item.PropertyName, first_item.ErrorMessage));
                    }

                }
                throw new SharedCore.Aplication.Shared.Exceptions.ValidationException("Validation error appear");

            }
        }
    }
}

Authorization errors

The authorization bahaviour iterates over all available [Authorize] attributes marked on the command using reflection and selects all validators available for it, and runs the authorization process.

If an error occurs and the request (in this case the command) is derived from BasePayload, it automatically adds new UnAuthorised("message") to the Error[] association of the command payload.
If the request is not a command but a query, it throws an UnAuthorisedException which is handled by the Graphql error filter.

You can customize this logic to your needs!

/Src/Aplication/Core/Behaviours/AuthorizationBehaviour.cs

namespace ErrorHandling.Aplication.Shared.Behaviours {

    // Authorization marker interface for Fluent validation
    public interface IAuthorizationValidator { }

    // Authorization validator wrapper
    /// <typeparam name="TRequest"></typeparam>
    public class AuthorizationValidator<TRequest> : AbstractValidator<TRequest>, IAuthorizationValidator {

    }

    // Authorization behaviour for MediatR pipeline
    public class AuthorizationBehaviour<TRequest, TResponse> : IPipelineBehavior<TRequest, TResponse> {
        private readonly ICurrentUser _currentUserService;
        private readonly IEnumerable<IValidator<TRequest>> _validators;
        private readonly ILogger _logger;

        public AuthorizationBehaviour(
            ICurrentUser currentUserService,
            IEnumerable<IValidator<TRequest>> validators,
             ILogger logger) {
            _currentUserService = currentUserService;
            _validators = validators;
            _logger = logger;
        }

        public async Task<TResponse> Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate<TResponse> next) {

            var authorizeAttributes = request.GetType().GetCustomAttributes<AuthorizeAttribute>();

            if (authorizeAttributes.Any()) {

                var activity = Sources.DemoSource.StartActivity(
                String.Format("AuthorizationBehaviour: Request<{0}>", request.GetType().FullName), ActivityKind.Server);

                try {
                    activity.Start();

                    // Must be authenticated user
                    if (!_currentUserService.Exist) {
                        return HandleUnAuthorised(null);
                    }

                    // Role-based authorization
                    var authorizeAttributesWithRoles = authorizeAttributes.Where(a => !string.IsNullOrWhiteSpace(a.Roles));
                    if (authorizeAttributesWithRoles.Any()) {
                        foreach (var roles in authorizeAttributesWithRoles.Select(a => a.Roles.Split(','))) {
                            var authorized = false;

                            foreach (var role in roles) {

                                if (_currentUserService.HasRole(role.Trim())) {
                                    authorized = true;
                                    break;
                                }
                            }

                            // Must be a member of at least one role in roles
                            if (!authorized) {
                                return HandleUnAuthorised("Role authorization failure");
                            }
                        }
                    }

                    // Policy-based authorization
                    var authorizeAttributesWithPolicies = authorizeAttributes.Where(a => !string.IsNullOrWhiteSpace(a.Policy));
                    if (authorizeAttributesWithPolicies.Any()) {
                        foreach (var policy in authorizeAttributesWithPolicies.Select(a => a.Policy)) {
                            if (!_currentUserService.HasRole(policy.Trim())) {
                                return HandleUnAuthorised($"Policy: {policy} authorization failure");
                            }
                        }
                    }

                    // Inner command validator autorization checks
                    var authorizeAttributesWithInnerPolicies = authorizeAttributes.Where(a => a.InnerPolicy == true);
                    if (authorizeAttributesWithInnerPolicies.Any()) {
                        IValidator<TRequest>[] authValidators = _validators.Where(v => v is IAuthorizationValidator).ToArray();
                        ValidationFailure[] authorization_validator_failures = await CommandAuthValidationFailuresAsync(request, authValidators);

                        if (authorization_validator_failures.Any()) {
                            return HandleUnAuthorised(authorization_validator_failures);
                        }
                    }
                } catch (Exception ex) {
                    SharedCore.Aplication.Shared.Common.CheckAndSetOtelExceptionError(ex,_logger);

                    // In case it is Mutation Response Payload = handled as payload error union
                    if (SharedCore.Aplication.Shared.Common.IsSubclassOfRawGeneric(typeof(BasePayload<,>), typeof(TResponse))) {
                        return Common.HandleBaseCommandException<TResponse>(ex);
                    } else {
                        throw ex;
                    }
                } finally {
                    activity.Stop();
                    activity.Dispose();
                }
            }

            // Continue in pipe
            return await next();
        }

        private static TResponse HandleUnAuthorised(object error_obj) {

            // In case it is Mutation Response Payload = handled as payload error union
            if (SharedCore.Aplication.Shared.Common.IsSubclassOfRawGeneric(typeof(BasePayload<,>), typeof(TResponse))) {
                IBasePayload payload = ((IBasePayload)Activator.CreateInstance<TResponse>());

                if (error_obj is ValidationFailure[]) {
                    foreach (var item in error_obj as ValidationFailure[]) {
                        payload.AddError(new UnAuthorised(item.CustomState, item.ErrorMessage));
                    }
                } else if (error_obj is string) {
                    payload.AddError(new UnAuthorised(error_obj as string));
                } else {
                    payload.AddError(new UnAuthorised());
                }

                return (TResponse)payload;
            } else {
                // In case it is query response = handled by global filter
                if (error_obj is ValidationFailure[]) {
                    throw new AuthorizationException(error_obj as ValidationFailure[]);
                } else if (error_obj is string) {
                    throw new AuthorizationException(error_obj as string);
                } else {
                    throw new AuthorizationException();
                }
            }
        }

        private static async Task<ValidationFailure[]> CommandAuthValidationFailuresAsync(TRequest request, IEnumerable<IValidator<TRequest>> authValidators) {
            var validateTasks = authValidators
                .Select(v => v.ValidateAsync(request));
            var validateResults = await Task.WhenAll(validateTasks);
            var validationFailures = validateResults
                .SelectMany(r => r.Errors)
                .Where(f => f != null)
                .ToArray();

            if (validationFailures == null) {
                return new ValidationFailure[0];
            } else {
                return validationFailures;
            }

        }
    }
}

Hotchocolate integration

Until now all was +- related to the application logic, which prepared everything that was important to integrate proper error handling into the GraphQL layer.

For schema configuration, I will use the pure code approach. I like this the best because everything is tightly defined in your code. The downside is that multiple objects need to be defined.

Let us first create the root types query and mutation:

/Src/Aplication/Graphql/Queries/Query.cs

namespace ErrorHandling.Aplication.GraphQL.Queries {

    public class Query { }
}

/Src/Aplication/Graphql/Mutations/Mutation.cs

namespace ErrorHandling.Aplication.GraphQL.Mutation {

    public class Mutation { }

}

We need to create the extension UserMutations of the base class Mutation and add some mutations to it:

/Src/Aplication/Graphql/Mutations/UserMutations.cs

namespace ErrorHandling.Aplication.GraphQL.Mutation {

    // User Mutation extension
    [ExtendObjectType(OperationTypeNames.Mutation)]
    public class UserMutations {

         /// <summary> Create user input object  </summary>
        public class CreateUserInput {

            public string NickName {get; set;}

            public int Age {get; set;}
        }

        /// <summary> Create user mutation </summary>
        public async Task<CreateUserPayload> CreateUser(
            CreateUserInput request,
            [Service] IMediator _mediator) {

            return await _mediator.Send(new CreateUser(){
                NickName = request.NickName,
                Age = request.Age
            });
        }
    }
}

As you can see from the code, the mutation CreateUser takes an input object and transfers it to the command MediatR. This command returns CreateUserPayload, which is returned as the response type.

Now we need to define concrete GraphQl ObjectTypes:

/Src/Aplication/Graphql/Types/ObjectTypes/UserType.cs

namespace ErrorHandling.Aplication.GraphQL.Types {

    /// Graphql UserType
    public class UserType : ObjectType<GQL_User> {

        protected override void Configure(IObjectTypeDescriptor<GQL_User> descriptor) {

            descriptor.ImplementsNode().IdField(t => t.Guid)
            .ResolveNode((ctx, id) =>
                 ctx.DataLoader<UserByIdDataLoader>()
                 .LoadAsync(id, ctx.RequestAborted));

            descriptor.Field("systemid").Type<NonNullType<StringType>>()
            .Resolve((IResolverContext context) => {
                return context.Parent<GQL_User>().Guid.ToString();
            });

        }

        private class UserResolvers {

        }
    }
}

And mutation typesa are:

namespace ErrorHandling.Aplication.GraphQL.Types {
    public class CreateUserPayloadType : ObjectType<CreateUserPayload> {
        protected override void Configure(IObjectTypeDescriptor<CreateUserPayload> descriptor) {

            // Add any graphql descriptor cfg for payload obj. in this palce
        }
    }

    public class CreateUserErrorUnion : UnionType<ICreateUserError> {
        protected override void Configure(IUnionTypeDescriptor descriptor) {
            descriptor.Type<ValidationErrorType>();
            descriptor.Type<UnAuthorisedType>();
            descriptor.Type<InternalServerErrorType>();
        }
    }
}

Define the schema BaseError ObjectType and define the common interface IBaseError.

namespace ErrorHandling.Aplication.GraphQL.Types {

    public class BaseErrorType : ObjectType<BaseError> {
        protected override void Configure(IObjectTypeDescriptor<BaseError> descriptor) {

            descriptor.Implements<BaseErrorInterfaceType>();
        }
    }

    public class BaseErrorInterfaceType : InterfaceType<IBaseError> {
        protected override void Configure(IInterfaceTypeDescriptor<IBaseError> descriptor) {
            descriptor.Field(e => e.message).Type<StringType>();
        }
    }
}

Define ObjectTypes for each Erros. In general, they can contain custom resolvers and fields. To keep this demo simple, implement only the IBaseError interface.

namespace ErrorHandling.Aplication.GraphQL.Types {

    public class InternalServerErrorType : ObjectType<InternalServerError> {
        protected override void Configure(IObjectTypeDescriptor<InternalServerError> descriptor) {
            descriptor.Implements<BaseErrorInterfaceType>();
        }
    }
}

namespace ErrorHandling.Aplication.GraphQL.Types {

    public class UnAuthorisedType : ObjectType<UnAuthorised> {
        protected override void Configure(IObjectTypeDescriptor<UnAuthorised> descriptor) {
            descriptor.Implements<BaseErrorInterfaceType>();
        }
    }
}

namespace ErrorHandling.Aplication.GraphQL.Types {

    public class ValidationErrorType : ObjectType<ValidationError> {
        protected override void Configure(IObjectTypeDescriptor<ValidationError> descriptor) {
            descriptor.Implements<BaseErrorInterfaceType>();
        }
    }
}

Finally, set up all the created ObjectTypes, Unios and Interfaces in your GraphQL server configuration:

src/API/Program.cs

namespace Device.Configuration {
    public static partial class ServiceExtension {
        public static IServiceCollection AddGraphql(
            this IServiceCollection serviceCollection, IWebHostEnvironment env) {

            serviceCollection.AddGraphQLServer()
                    .SetPagingOptions(
                        new PagingOptions { IncludeTotalCount = true, MaxPageSize = 100 })
                    .ModifyRequestOptions(requestExecutorOptions => {
                        if (env.IsDevelopment() ||
                            System.Diagnostics.Debugger.IsAttached) {
                            requestExecutorOptions.ExecutionTimeout = TimeSpan.FromMinutes(1);
                        }

                         requestExecutorOptions.IncludeExceptionDetails = !env.IsProduction();

                    })

                    .AddGlobalObjectIdentification()
                    .AddQueryFieldToMutationPayloads()

                    .AddFiltering()
                    .AddSorting()

                    .AddQueryType<Query>()
                        .AddTypeExtension<SystemQueries>()
                        .AddTypeExtension<UserQueries>()
                    .AddMutationType<Mutation>()
                        .AddTypeExtension<UserMutations>()

                    .BindRuntimeType<DateTime, DateTimeType>()
                    .BindRuntimeType<int, IntType>()

                    .AddType<InternalServerErrorType>()
                    .AddType<UnAuthorisedType>()
                    .AddType<ValidationErrorType>()
                    .AddType<BaseErrorType>()
                    .AddType<UserType>()
                    .AddType<BaseErrorInterfaceType>()

                    .AddDataLoader<UserByIdDataLoader>()

                    .AddType<CreateUserPayloadType>()
                    .AddType<CreateUserErrorUnion>()
                    .AddType<RemoveUserPayloadType>()
                    .AddType<RemoveUserErrorUnion>();


            return serviceCollection;
        }
    }
}

Demo UI

Sources also includes a small React SPA frontend served by the NetCore Static Files middelware. The frontend itself only displays the basic mutations and uses the Relay GraphQL Clinet to communicate withthe backend server.

This is an example where the mutation "Create user " was triggered. (This has nothing to do with user management, but is just a dummy example):

This is an example of CreateNewUserMutation, which already uses the shared IBaseError to get the message field and publish this error with the toast component.


const [
commit_create,
isInFlight,
] = useMutation<CreateNewUserMutation>(graphql`
mutation CreateNewUserMutation(
    $request: CreateUserInput
    $connections: [ID!]!
    ) {
    createUser(request: $request) {

        user @prependNode(
            connections: $connections
            edgeTypeName: "GQL_User"
        ){
            id
            nickName
            age
        }
        ... on CreateUserPayload {
            errors {
                ... on IBaseError {
                    message
                }
            }
        }
    }
}
`);

This is an example of a validation error returned via the IBaseError interface (in case of error):

{
  "data": {
    "createUser": {
      "user": null,
      "errors": [
        {
          "message": "Nickname must be unique"
        }
      ]
    }
  }
}

This will give you a proper typed error object with Typescript to respond to this response.

These are generated types when only IBaseError is used:

export type CreateUserInput = {
    nickName?: string | null;
    age: number;
};
export type CreateNewUserMutationVariables = {
    request?: CreateUserInput | null;
    connections: Array<string>;
};
export type CreateNewUserMutationResponse = { // <-- Look on response object
    readonly createUser: {
        readonly user: {
            readonly id: string;
            readonly nickName: string | null;
            readonly age: number;
        } | null;
        readonly errors: ReadonlyArray<{
            readonly message?: string | null;
        } | null> | null;
    } | null;
};
export type CreateNewUserMutation = {
    readonly response: CreateNewUserMutationResponse;
    readonly variables: CreateNewUserMutationVariables;
};

You can also rewrite your mutation to query for specific error types if they contain some error-specific fields:

mutation{
  createUser(request:{nickName:"SomeName",age:55}){
    ... on CreateUserPayload{
      user{
        id
        nickName
        age
      }
      errors{
        __typename
        ...on ValidationError{
            message
            fieldName
        }
        ... on InternalServerError{
            message
        }
        ... on UnAuthorised{
            message
        }
      }
    }
  }
}

This is a json response and fieldName is a special field that is only present with type ValidationError!


{
  "data": {
    "createUser": {
      "user": null,
      "errors": [
        {
          "message": "Nickname must be unique",
          "fieldName": "NickName"
        }
      ]
    }
  }
}

And these are the types generated by RelayCompiler

export type CreateUserInput = {
    nickName?: string | null;
    age: number;
};
export type CreateNewUserMutationVariables = {
    request?: CreateUserInput | null;
    connections: Array<string>;
};
export type CreateNewUserMutationResponse = {
    readonly createUser: {
        readonly user: {
            readonly id: string;
            readonly nickName: string | null;
            readonly age: number;
        } | null;
        readonly errors: ReadonlyArray<({
            readonly __typename: "ValidationError";
            readonly fieldName: string | null;
            readonly message: string | null;
        } | {
            readonly __typename: "InternalServerError";
            readonly message: string | null;
        } | {
            readonly __typename: "UnAuthorised";
            readonly message: string | null;
        } | {
            /*This will never be '%other', but we need some
            value in case none of the concrete values match.*/
            readonly __typename: "%other";
        }) | null> | null;
    } | null;
};
export type CreateNewUserMutation = {
    readonly response: CreateNewUserMutationResponse;
    readonly variables: CreateNewUserMutationVariables;
};

Quick start

1) Make sure you have Net 5.0 SDK installed. You can cheque the SDK version by running: dotnet --list-sdks in your terminal.

#Example output on Windows
PS C:\Users\dakupc> dotnet --list-sdks
5.0.100 [C:\Program Files\dotnet\sdk]
5.0.201 [C:\Program Files\dotnet\sdk]

2) Clone the repo

 git clone https://github.com/damikun/hotchocolate-mutation-errors.git

3) cd to Src/API/ and run dotnet restore

4) cd to Src/API/ClientApp and run yarn install or npm install

NOTE: This may take some time since all node_modules of Frontend SPA needs to be installed on first startup.

5) After succesfull restoring cd to Src/API/ and run dotnet run.

6) Demo runs on: https://localhost:5001

In case you have error with certificates. Install Dev. Certificates:

dotnet dev-certs https --trust

Repository

The source project of the article is available in this repo:

https://github.com/damikun/hotchocolate-mutation-errors

I also have an free advanced example of a fullstack app (workshop):

https://github.com/damikun/trouble-training

This second link is an example of an open source fullstack app that also covers this topic, but is more complex and difficult to understand as it consists of different microservices and shows identity and distributed tracing and logging configuration.

Fullstack app (workshop) with distributed tracing, logging and monitoring

Dalibor Kundrat — Fri, 01 Oct 2021 06:41:48 +0000

Intro

I would like to announce the OpenSource Fullstack App Example (Workshop) that shows a specific stack implementation from frontend to backend, including identity and distributed tracing, logging and monitoring.

https://github.com/damikun/trouble-training

What is the idea behind this?

There are many courses, trainings (paid/opensource) online that show you specific things, but not many that show you a fullstack implementation.

This is not perfectly done! Take this as my notes. Some parts were exported from my custom CCMS that I use for factory automation field, and I remember when I was just starting out and did not have snippets to help me solve certain problems.

What it's about.

Showing simple frontend integration for a webhooks management app with login/logout via backend bff using React, Relay, Typescript.
Shows exporting telemetry and logs to distributed monitoring endpoints with Opentelemetry and ElasticStack
Shows a full backend implementation for webhooks, including self-hosted identityServer and protected Graphql API.

For who it is

You are a full-stack developer (an attempt is made to explain advanced topics).
You are a .Net developer and use modern technologies.
Or you are implementing a modern React and Relay frontend based on a .Net backend.
You want to understand the advanced concept of security patterns as BFF.
You want to know how to trace and monitor a distributed application.

Topics

This release contains 7+ main topics that you can read about in detail:

(Some topics are language specific, others relate to a specific platform).

Repository:

https://github.com/damikun/trouble-training

Export request traces from React SPA to backend Opentelemetry collector

Dalibor Kundrat — Thu, 23 Sep 2021 16:00:20 +0000

Distributed tracing

Tracing in distributed systems is nothing new. There are many solutions on the web that give you full frontend tracing and monitoring analysis, and they do a good job.

What all these solutions have in common is that they are not globally standardized, so you can not just switch to or integrate with another solution. In most cases, they do not integrate with your backend.

This is changing now, because there is Open Telemetry as a new standard, and in 2021 OpenTelemetry has reached an important milestone: the OpenTelemetry Tracing Specification version 1.0.

What is Opentelemetry

Is a collection of tools, APIs and SDKs. Used to collect telemetry data from distributed systems to troubleshoot, debug, and understand software performance and behavior.

Many modern applications are based on microservices. These are essentially an interconnected network of services, so understanding system performance from multiple sources is a major challenge. A single call in an application can trigger dozens of events.

How can developers and engineers isolate a problem when something goes wrong or a request runs slowly?

Opentelemetry standardized the way and also offers SDKs that allow you to collect data from different systems and in different programming languages to debug your stack at a high level.

All relevant information on the Opentelemtry specification can be found at its official documentation.

Opentelemetry components

APIs and SDKs per programming language for generating and emitting traces (SDK for Java,.Net,C++,Golang,Python,Javascript,PHP,Ruby etc...)
Collectors - provides a vendor independent implementation for receiving, processing and exporting telemetry data.
The OTLP Protocol specification describes the encoding, transport and transmission mechanism of telemetry data. You can read more.

Nowadays, some languages natively support passing trace contexts (trace context propagation), such as .NetCore, and many cloud providers allow importing or exporting traces from or to the cloud via the otel protocol.

And that's a good thing, because you can easily reuse an analytics platform and integrate your applications there, or take all the metrics and pass them to your platform.

This is an example of distributed tracing from frontend to backend

You can see all the operations over time, every detail and the logs for each record (span). The entire request flow between Frontend > Backend > Post-Request Async processing.

This article will not show you how to integrate a fullstack tracing solution. I have a free open source workshop for that, including full working application to handle WebHooks.

This article is exclusively about exporting request traces from your frontend React to the backend Opentelemetry Collector.

Frontend instrumentation

For frontend JavaScript clients, opentelemtry provides the main SDK opentelemetry-js. There are also several additional packages needed for instrumentation and trace export.

NOTE: If the language is not natively supported, use the Opentelemetry SDK. If it is, use a native!

Packages

In most cases, you do not need a full SDK and tracing this request requires the following imports in package.json:



"dependencies": {
    "@opentelemetry/api": "1.0.4",
    "@opentelemetry/context-zone": "1.0.1",
    "@opentelemetry/exporter-trace-otlp-http": "0.27.0",
    "@opentelemetry/instrumentation-document-load": "0.27.0",
    "@opentelemetry/instrumentation-fetch": "0.27.0",
    "@opentelemetry/sdk-trace-base": "1.0.1",
    "@opentelemetry/sdk-trace-web": "1.0.1",
    "@opentelemetry/resources": "1.0.1",
}

There are other tools you can use to measure document load time or navigation between pages, etc., but that's not the use case for full request tracing in this article! That has more to do with metrics and performance analysis.

Front-end transactions are often thought of as "loading the entire page, navigation, adding items to cart", etc. This article is about requests and looks at transactions as isolated backend commands like CreateUser or SubmitForm that have a single responsibility.

Frontend integration

On the frontend, I mostly use the ideology of provider components. This is a set of components that wrap around each other on root to provide a specific functionality, such as UserProvider or EnviromentProvider or in our case TraceProvider.

** Please check last sources for latest integration. The API of opentelemetry-js changes in time since it is in dev.**



// Providers.tsx
  <EnviromentContext.Provider value={providerInit}>
    <EnviromentContext.Consumer>
      {(state) =>
        state && (
          <RelayEnvironmentProvider environment={state?.env}>
            <Suspense fallback={fallback ? fallback : null}>
              <TraceProvider>
                <UserProvider>
                    <ToastProvider>{children}</ToastProvider>
                </UserProvider>
              </TraceProvider>
            </Suspense>
          </RelayEnvironmentProvider>
        )
      }
    </EnviromentContext.Consumer>
  </EnviromentContext.Provider>

where <TraceProvider> is implemented as this:



import React from "react";
import { WebTracerProvider  } from '@opentelemetry/sdk-trace-web';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { ZoneContextManager } from '@opentelemetry/context-zone';
import { FetchInstrumentation } from '@opentelemetry/instrumentation-fetch';
import { registerInstrumentations } from '@opentelemetry/instrumentation';
import { Resource } from '@opentelemetry/resources';
import { OTLPTraceExporter  } from "@opentelemetry/exporter-trace-otlp-http"
import {TRACES_ENDPOINT} from "../constants"

const collectorOptions = {
    url: TRACES_ENDPOINT, 
    headers: {
    "Content-Type": "application/json", 
    'Access-Control-Allow-Headers': '*',
    'X-CSRF': '1',
  },
    concurrencyLimit: 10,
  };

// Trace provider (Main aplication trace)
const provider = new WebTracerProvider({
  resource: new Resource({
    "service.name": "Frontend",
  }
  )});

// Exporter (opentelemetry collector hidden behind bff proxy)
const exporter = new OTLPTraceExporter (collectorOptions);

// Instrumentation configurations for frontend
const fetchInstrumentation = new FetchInstrumentation({
  ignoreUrls : ["https://some-ignored-url.com"]
});

fetchInstrumentation.setTracerProvider(provider);

provider.addSpanProcessor(new SimpleSpanProcessor(exporter));

provider.register({
  contextManager: new ZoneContextManager(),
});

// Registering instrumentations
registerInstrumentations({
    instrumentations: [new FetchInstrumentation()],
});

export type TraceProviderProps = {
  children?: React.ReactNode;
};

export default function TraceProvider({ children }: TraceProviderProps) {

  return (
    <>
      {children}
    </>
  );
}

There are a few important points to remember:

Setting the application name "service.name": "Frontend" is important for monitoring the user interface.
When configuring the exporter, send the correct headers like 'X-CSRF': '1' etc. based on the backend configuration
Specify ignoreUrls - you do not want to track additional system requests or 3-part requests that you are not interested in.
Specify the URL of the export endpoint. Example: https://localhost:5015/traces

Trace results

This is an example of a trace sent from the frontend to the collector



{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "Frontend"
            }
          },
          {
            "key": "telemetry.sdk.language",
            "value": {
              "stringValue": "webjs"
            }
          },
          {
            "key": "telemetry.sdk.name",
            "value": {
              "stringValue": "opentelemetry"
            }
          },
          {
            "key": "telemetry.sdk.version",
            "value": {
              "stringValue": "1.0.1"
            }
          }
        ],
        "droppedAttributesCount": 0
      },
      "instrumentationLibrarySpans": [
        {
          "spans": [
            {
              "traceId": "d6d75718930b3558e4fe0808877f8e80",
              "spanId": "3b7f9b452a7b5ddf",
              "name": "HTTP POST",
              "kind": 3,
              "startTimeUnixNano": 1644389713311600000,
              "endTimeUnixNano": 1644389713673100000,
              "attributes": [
                {
                  "key": "component",
                  "value": {
                    "stringValue": "fetch"
                  }
                },
                {
                  "key": "http.method",
                  "value": {
                    "stringValue": "POST"
                  }
                },
                {
                  "key": "http.url",
                  "value": {
                    "stringValue": "/graphql"
                  }
                },
                {
                  "key": "http.status_code",
                  "value": {
                    "intValue": 200
                  }
                },
                {
                  "key": "http.status_text",
                  "value": {
                    "stringValue": ""
                  }
                },
                {
                  "key": "http.host",
                  "value": {
                    "stringValue": "localhost:5015"
                  }
                },
                {
                  "key": "http.scheme",
                  "value": {
                    "stringValue": "https"
                  }
                },
                {
                  "key": "http.user_agent",
                  "value": {
                    "stringValue": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.82 Safari/537.36"
                  }
                },
                {
                  "key": "http.response_content_length",
                  "value": {
                    "intValue": 168
                  }
                }
              ],
              "droppedAttributesCount": 0,
              "events": [
                {
                  "timeUnixNano": 1644389713312300000,
                  "name": "fetchStart",
                  "attributes": [],
                  "droppedAttributesCount": 0
                },
                {
                  "timeUnixNano": 1644389713312300000,
                  "name": "domainLookupStart",
                  "attributes": [],
                  "droppedAttributesCount": 0
                },
                {
                  "timeUnixNano": 1644389713312300000,
                  "name": "domainLookupEnd",
                  "attributes": [],
                  "droppedAttributesCount": 0
                },
                {
                  "timeUnixNano": 1644389713312300000,
                  "name": "connectStart",
                  "attributes": [],
                  "droppedAttributesCount": 0
                },
                {
                  "timeUnixNano": 1644389713312300000,
                  "name": "secureConnectionStart",
                  "attributes": [],
                  "droppedAttributesCount": 0
                },
                {
                  "timeUnixNano": 1644389713312300000,
                  "name": "connectEnd",
                  "attributes": [],
                  "droppedAttributesCount": 0
                },
                {
                  "timeUnixNano": 1644389713314500000,
                  "name": "requestStart",
                  "attributes": [],
                  "droppedAttributesCount": 0
                },
                {
                  "timeUnixNano": 1644389713670100000,
                  "name": "responseStart",
                  "attributes": [],
                  "droppedAttributesCount": 0
                },
                {
                  "timeUnixNano": 1644389713670800100,
                  "name": "responseEnd",
                  "attributes": [],
                  "droppedAttributesCount": 0
                }
              ],
              "droppedEventsCount": 0,
              "status": {
                "code": 0
              },
              "links": [],
              "droppedLinksCount": 0
            }
          ],
          "instrumentationLibrary": {
            "name": "@opentelemetry/instrumentation-fetch",
            "version": "0.27.0"
          }
        }
      ]
    }
  ]
}

Opentelemetry collector

To run the collector in the backend you can use the attached file docker-compose.yml to set up and configure the simple collector. Take this as an example. You still need to export the data from the collector to the Trace Analytics software. I can recommend you:

Elastic Stack - High performance self and cloud hosted solution
Jaeger tracing - Self hosted, Easy to start

This article does not explain how to set up a full collector with analytics service and storage. If you want to see real example, you can read and try my free opensource workshop on github. There are also advanced concepts such as BFF patterns and hiding Collector and API behind proxy.

This sample collector receives data from the source via grpc or http and exports it back to a storage or analysis service via the grpc using otel protocol

Collector compose file:



version: '3'

services:
  opentelemetry-collector:
    container_name: opentelemetry-collector
    hostname: opentelemetry-collector
    image: otel/opentelemetry-collector:0.43.0
    command: [ "--config=/etc/otel-collector-config.yml" ]
    volumes:
      - ./otel-collector-config.yml:/etc/otel-collector-config.yml
    ports:
      - "14250:14250"
      - "55680:55680"
      - "55690:55690"
    networks:
      - tracing

networks:
  tracing:
    driver: bridge

Collector config file:



receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:55680
      http:
        endpoint: "0.0.0.0:55690"

processors:
  batch:

exporters:
  otlp/2:
    endpoint: apm-server:8200
    tls:
      insecure: true
  logging:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, otlp/2]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, otlp/2]

Jaeger docker-compose.yaml



version: '3'
services:
  jaeger:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686"
      - "14268"
      - "14250"
    networks:
      - jaeger-example

networks:
  jaeger-example:

Forem: Dalibor Kundrat

From .Net Metrics to Graphql Sub.

.Net Metric != Opentelemetry Metric.

Counter as example

What happens if you call counter.Add(Value) multiple times?

Then why is it called a counter?

How to get real counter behaviour?

Value representation

Unusal custom reader+exporter

API Testing with Postman and Newman from CI/CD

Postman as GraphQL IDE?

Import GraphQL SDL

Pre-requests scripts

Is there a initial script?

How to sign-in with OAuth?

Request access token

Validation of the GraphQL response

Export test collection

Newman headless runner

Integration in Build and CI/CD

Github-Action run:

Repository

Why GraphQL IDEs on the market are not what you want

Todays GraphQL IDEs

The requirements

Result

Cross-platform Build Automation with Nuke

Intro

Build automation is not just about building the code

GithubActions, AzurePipelines, TeamCity and what else?

Nuke build

Quick setup

More complex Demo example

What's next?

Repository

Web App Security, Understanding the Meaning of the BFF Pattern

Identity and security overview

Authentication vs Authorization

Tokens vs Cookie and Sessions

Cookies

What is origin?

Another cookie definitions:

Sessions

Tokens

JWT Tokens

Identity protocols

OAuth

Token exchange flows

OpenID Connect

Backend for Frontend pattern (BFF)

Security reasons

Architectural reasons

BFF vs API gateway

BBF cookies termination and token isolation

What's next?

Repository

Integrate WebHooks under ASP.NET Core (c#) backend

WebHooks

Demo requirements

ED diagram

Stack

Domain level

WebHook Domain Model

Events Domain model

Persistence layer

Application layer

WebHook commands and queries

WebHook notification and processing

API Layer

UI interface

Repository

GraphQL mutation errors (union errors with interface aka 6a) with Hotchocolate GraphQL Server

Intro

Content

GraphQL Errors

Mutation Errors

Stage 6a Integration

Abstraction

Validation errors

Authorization errors

What happens if you call `counter.Add(Value)` multiple times?