Forem: Peter Davis

Protect Azure Functions with API Keys using Azure API Management - Part 3

Peter Davis — Wed, 06 Sep 2023 16:49:56 +0000

In Part 1 and Part 2 of this series we created two Azure Functions protected by function keys, published those to Azure and then wrapped those using Azure API Management so we could control access using products and subscription keys.

In this final part we're going to update our System function to let it create new subscriptions directly in API Management which can then be used to call our user functions.

Enable Management API

Open the Management API tab from the left hand menu of the API Management instance in Azure and then set Enable Management REST API to Yes. In addition click on the Generate button next to Access Token, this will allow us to test our calls to make sure everything is working.

The API Management service exposes a management API that we can use to create subscriptions, along with many other things, and you can get more information here.

As you can see from the screenshot above ours is exposed at https://devtoapiman.management.azure-api.net and using the Access Token we generated above we can make calls to this API. Let's give it a go and get the details for the DevToUserFunctions product we created earlier.

We will need to make a call to:

https://devtoapiman.management.azure-api.net/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ApiManagement/service/{serviceName}/products/{productId}?api-version=2022-08-01

{subscriptionId} can be found on the Overview tab.
{resourceGroupName} can be found on the Overview tab.
{serviceName} is the name you gave to your API Management service (devtoapiman in this example)
{productId} is the product we want to get the details of (DevToUserFunctions in this example)

We will also need to add a header to the call. This will be called Authorization and the value should be set to the Access Token you generated above.

If we provide these details we should get information regarding our product returned, confirming we have access to the management API.

We will need to use the Subscription end point of this API to programmatically create new subscriptions. Details for this end point can be reviewed here.

Update our System API

Head back to Visual Studio and our devto.apiman.system project and the first thing we want to do is add RestSharp as a dependency, for this example add the following nuget packages:

RestSharp
RestSharp.Serializers.SystemTextJson

Now add a new class called APIManagementSubscription to our project.

namespace devto.apiman.system
{
    public class APIManagementSubscription
    {
        public APIManagementSubscription()
        {
            id = string.Empty;
            type = string.Empty;
            name = string.Empty;
            properties = new Properties();
        }

        public string id { get; set; }
        public string type { get; set; }
        public string name { get; set; }
        public Properties properties { get; set; }
    }

    public class Properties
    {
        public Properties()
        {
            scope = string.Empty;
            displayName = string.Empty;
            primaryKey = string.Empty;
            secondaryKey = string.Empty;
        }

        public string scope { get; set; }
        public string displayName { get; set; }
        public bool allowTracing { get; set; }
        public string primaryKey { get; set; }
        public string secondaryKey { get; set; }
    }

    public class APIManagementSubscriptionCreate
    {
        public APIManagementSubscriptionCreate()
        {
            properties = new Properties();
        }

        public Properties properties { get; set; }
    }
}

This will be used to generate the JSON to create our new subscriptions.

We can let the management API auto generate our subscription primary and secondary key, but we're going to provide our own. In this case we'll just generate a random string, but in reality we may use guids generated in a database or another value that comes from somewhere else. Our random string generator is:

private string GenerateAPIKey(int randomLength = 50)
{
    char[] chars = "_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890-=".ToCharArray();

    var randomString = new StringBuilder();

    for (var i = 0; i < randomLength; i++)
    {
        randomString.Append(chars[Random.Shared.Next(chars.Length)]);
    }

    return randomString.ToString();
}

Next we're going to generate our payload, based on the documentation here.

private APIManagementSubscriptionCreate GenerateSubscriptionCreation(string userId)
{
    var apiManagementSubscriptionCreate = new APIManagementSubscriptionCreate();

    var subscriptionId = "XXXXXXXXXXXXXX";
    var resourceGroup = "DevTo-APIManagement";
    var serviceName = "devtoapiman";
    var productName = "devtouserfunctions";

    apiManagementSubscriptionCreate.properties.displayName = userId;
    apiManagementSubscriptionCreate.properties.scope = $"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.ApiManagement/service/{serviceName}/products/{productName}";
    apiManagementSubscriptionCreate.properties.primaryKey = GenerateAPIKey();
    apiManagementSubscriptionCreate.properties.secondaryKey = GenerateAPIKey();
    apiManagementSubscriptionCreate.properties.allowTracing = true;

    return apiManagementSubscriptionCreate;
}

Note that we're hardcoding our values like subscription ID and resource group in this example (and you should change them to be appropriate for your setup), but in a proper application we'd read those in from something like environment variables or an Azure Key Vault.

We need to generate an Access Token to be able to call the Management API and for that we can use the following code.

private string GenerateToken()
{
    var id = "integration";
    var key = "XXXXXXXXXXXXXXXXXXXXXX";
    var expiry = DateTime.UtcNow.AddDays(10);
    using (var encoder = new HMACSHA512(Encoding.UTF8.GetBytes(key)))
    {
        var dataToSign = id + "\n" + expiry.ToString("O", CultureInfo.InvariantCulture);
        var hash = encoder.ComputeHash(Encoding.UTF8.GetBytes(dataToSign));
        var signature = Convert.ToBase64String(hash);
        var encodedToken = string.Format("SharedAccessSignature uid={0}&ex={1:o}&sn={2}", id, expiry, signature);
        return encodedToken;
    }
}

The 'id' above is the Identifier field from the Management API tab we looked at in Azure earlier. The 'key' is either the Primary key or Secondary key from the same screen.

Our final step before we can make our call is generating the API endpoint to use.

private string GetManagerEndPoint(string userId)
{
    var subscriptionId = "XXXXXXXXXXXXXX";
    var resourceGroup = "DevTo-APIManagement";
    var serviceName = "devtoapiman";

    return $"subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.ApiManagement/service/{serviceName}/subscriptions/{userId}?api-version=2022-08-01";
}

Now we can update our function to call the management API to create a subscription and return the result.

[Function("GenerateUserAPIKey")]
public async Task<HttpResponseData> Run([HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req)
{
    var restOptions = new RestClientOptions("https://devtoapiman.management.azure-api.net");

    var restClient = new RestClient(restOptions);

    var managerToken = GenerateToken();

    restClient.AddDefaultHeader("Authorization", managerToken);

    var apiManagementSubscriptionCreate = GenerateSubscriptionCreation("generatedUserSubscription");

    var managerEndpoint = GetManagerEndPoint("generatedUserSubscription");

    var subscriptionResult = await restClient.PutJsonAsync<APIManagementSubscriptionCreate, APIManagementSubscription>(managerEndpoint, apiManagementSubscriptionCreate);

    if(subscriptionResult == null)
    {
        var errorResponse = req.CreateResponse(HttpStatusCode.InternalServerError);
        errorResponse.Headers.Add("Content-Type", "text/plain; charset=utf-8");
        errorResponse.WriteString("Error Generating user subscription");
    }

    var validResponse = req.CreateResponse(HttpStatusCode.OK);
    await validResponse.WriteAsJsonAsync(subscriptionResult);
    return validResponse;
}

The only other change we're going to make is to our user function, to display the API Key that was passed in so we can demonstrate retrieving that from the request so we could then use it to look up user data.

[Function("GetUserData")]
public HttpResponseData Run([HttpTrigger(AuthorizationLevel.Function, "get")] HttpRequestData req)
{
    _logger.LogInformation("C# HTTP trigger function processed a request.");

    var response = req.CreateResponse(HttpStatusCode.OK);
    response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

    response.WriteString($"Found the API Key: {GetAPIKey(req)}");

    return response;
}

public string GetAPIKey(HttpRequestData req)
{
    var foundAPIKey = req.Headers.Where(req => req.Key == "devto-key").FirstOrDefault();

    if (foundAPIKey.Value == null)
        return string.Empty;

    return foundAPIKey.Value.FirstOrDefault(string.Empty);
}

You can now re-publish these functions from Visual Studio, but before you test the System call to generate a new subscription you'll need to manually create a subscription to call that API.

Call the system 'GenerateUserAPIKey' function and check you get a valid response.

Assuming it completed successfully you can check that in the subscriptions to see the new value

And if we call the user function with the created API key we should now be able to retrieve that key from the request.

Final Thoughts

Hopefully this guide has provided you with all the basic building blocks you need to create new Azure Functions and protect them with API Keys provided via Azure API Management. You have also seen how you can use the management API to generate those keys so that this can be an automated process in your application on user sign up.

It's important to note that although we have been using Azure Functions in this example, you could easily use the same functionality in API Management to protect REST APIs created and hosted via different means.

I hope you found this useful.

Pete

Protect Azure Functions with API Keys using Azure API Management - Part 2

Peter Davis — Wed, 06 Sep 2023 16:49:46 +0000

In Part 1 of this series we created two Azure Functions protected by function keys and published those to Azure. In this part we're going to use Azure API Management to wrap those functions and provide access via API Keys.

Create an API Management service

Head back into Azure create a new resource choosing API Management

Provide an appropriate resource name for your instance and then choose the Developer (no SLA) pricing tier. As mentioned in part 1, there are some restrictions placed on the consumption tier so for a real world application you would probably want to use one of the other tiers, and for this example we need access to the management API that you don't get with the consumption tier.

All other settings can be left at their default values for this example. The creation process may take a little bit of time.

Import Functions

Once your API Management resource has created open it in Azure and choose APIs from the left menu and then Function App

In the import wizard that opens you should be able to browse to your Function and Select the calls you want to import.

Your import for the System function should look similar to this.

In my example I've used the 'Full' option and adjusted the 'API URL suffix' and included a 'Version identifier' to make the URL for this function a bit cleaner.

Do the same for the User function.

If we open the v1 User API and click on the Test tab you should be able to see the complete Request URL.

Lets head back to Postman and try calling this API, but this time don't add the x-functions-key header value.

Notice that we received a slightly different error message this time. If we head back to the Function and look at the App keys you'll see that API Management has automatically created it's own Host key, apim-DevToAPIMan, and automatically uses this to call the function.

The API Management URI is now protected by a subscription key that can be generated in the API Management interface and we're going to use this alongside Products to provide us with a lot of control when it comes to providing access to our APIs.

Add two new products via the left hand menu option in API Management, one for the system function and one for the user function. Give them appropriate names and then also choose the APIs you want this product to give access to, ticking the 'Published' checkbox along the way.

Being able to specify which APIs are included in a product allows us to create separate products for different user types, with access to only the APIs those types require if we want.

Once we've created the products head over to the Subscriptions left hand menu entry and add a new subscription.

You'll notice in the create subscription screen we can set the scope Product and then select the product that this subscription has access to.

When the subscription is created two keys are generated for us, a primary and a secondary. This could allow us to, for example, use the primary key internally in our application to call services as a user, but then make the secondary key visible to the user so that they can call the APIs themselves, this way we can differentiate between an internal application call and an external user call at runtime, and also revoke and regenerate one or both of the keys as required.

Before we go ahead and use these keys, head back to our APIs and in the settings change the header and query parameter names to something easier, in this example 'devto-key'.

With 'Subscription required' ticked if we now provide one of the primary or secondary keys via a header or query parameter called 'devto-key' we should gain access to our API call.

Let's give it a test.

Next Steps

We've now got Azure Functions which we've published that can be called directly if we include a host key, but we won't use that method for access, instead we've created an instance of Azure API Management, imported our functions and then used products and subscriptions to control access to the APIs.

In the final step we're going to bring this all together by making changes to our System Azure Function so that it can programmatically create new user subscriptions in our API Management instance.

Head on over to Part 3 to continue this.

Pete

Protect Azure Functions with API Keys using Azure API Management - Part 1

Peter Davis — Wed, 06 Sep 2023 16:49:34 +0000

Secure your Azure Functions with API Keys

While building REST APIs using .NET I've generally handled all the authentication tasks within the code I'm writing, however as I've transitioned across to using Isolated Azure Functions I've started to take advantage of the built in authentication provided by Azure, this allows me to remove authentication code from my projects and have that handled externally, providing me with the flexibility to change how I authenticate without the need to update my code.

Because my projects have previously used Azure B2C authentication I'd setup my Azure Functions to use that as an identity provider, which meant I needed to provide a valid token to authenticate.

I wanted to change this to allow for authentication with an API Key instead and by combining Azure Functions with Azure API Management I've been able to setup a simple way to achieve this programmatically.

In this article I'll cover:

Creating Azure Functions and publishing them to Azure
Importing Azure Functions into Azure API Management
Setting up Products and Subscriptions in Azure API Management
Generating API Keys and creating subscriptions Programmatically

Costs

One thing to note before we dive in is that the use of Azure Functions and API Management does come with a cost. In this example we'll use the consumption based tier for our functions so that cost will be minimal but for API management, there are limitations on the number of subscriptions you can create in the consumption based plan and more importantly we need access to the Management API which isn't provided at that level so we need to use the developer tier.

Create our Azure Functions

We're going to create two very simple Azure functions for this example. A System Function, that will handle the creation of new API Keys and a User Function that will be protected by those created API keys.

We will protect our System Function with an API Key that only we know and in a real world scenario this would be used by our application to create new API keys for users, likely at the point of user sign-up, without us needing to do it manually. Those generated keys can then be used to call our other function, and if we store those keys somewhere that will also allow us to identify the user making the call.

Within Azure choose Create a resource and then select Function App:

Create our first System function using a new resource group and name, choosing to deploy code using the .NET stack, 7 Isolated for the version and a region applicable to you. The operating system will be Linux and we'll use the Consumption hosting option.

Choose Review + create as all other settings can be left as their defaults.

Once this is done create a second User function with the same options but a different name

After this you should have two functions created in Azure, mine are called devto-apiman-system and devto-apiman-user.

Create Visual Studio Projects

You can use whatever development tool you want to create your code, in this example I'm going to use Visual Studio, the community edition is free to use, but you can also use Visual Studio Code or something like Rider from Jet Brains.

In Visual Studio choose to create a new Azure Function and choose a name, I'm going to use devto.apiman.system with a solution called DevTo APIManagement

On the next screen make sure you've choose .NET 7.0 Isolated for the 'Functions worker' and Http trigger for the 'Function'. Make sure that the 'Authorization level' is set to Function, this means that a key will be required to call the function once it is deployed to Azure. If we set this to 'Anonymous' we could still wrap it with an API Key when calling the function from Azure API Management, but if the user found the direct URI of the function they would be able to call it directly without a key.

Once created, add another function project to the solution with the same settings but for the user function. Once done you should have a solution that looks a little like this.

We're going to keep this simple for the moment, so change the Function1.cs in your system function so that the function name is something better ("GenerateUserAPIKey" in my case), set the Trigger to be a "post" request only, and tweak the message returned.



using System.Net;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;

namespace devto.apiman.system
{
    public class Function1
    {
        private readonly ILogger _logger;

        public Function1(ILoggerFactory loggerFactory)
        {
            _logger = loggerFactory.CreateLogger<Function1>();
        }

        [Function("GenerateUserAPIKey")]
        public HttpResponseData Run([HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req)
        {
            _logger.LogInformation("C# HTTP trigger function processed a request.");

            var response = req.CreateResponse(HttpStatusCode.OK);
            response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

            response.WriteString("Generated a user API Key!");

            return response;
        }
    }
}

Edit Function1.cs in the user function in a similar way but make it a "get" request.



using System.Net;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;

namespace devto.apiman.user
{
    public class Function1
    {
        private readonly ILogger _logger;

        public Function1(ILoggerFactory loggerFactory)
        {
            _logger = loggerFactory.CreateLogger<Function1>();
        }

        [Function("GetUserData")]
        public HttpResponseData Run([HttpTrigger(AuthorizationLevel.Function, "get")] HttpRequestData req)
        {
            _logger.LogInformation("C# HTTP trigger function processed a request.");

            var response = req.CreateResponse(HttpStatusCode.OK);
            response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

            response.WriteString("We got some user data!");

            return response;
        }
    }
}

Update you solution properties so that both projects get started and then just run the solution to test it out. You should get the two functions starting, with their URIs.

If we use something like Postman to call these APIs you should get an 'OK' and the appropriate message back.

Notice that although we've specified AuthorizationLevel.Function for these APIs, we don't need to provide a key to call them. this is because when run locally the function authorization is ignored, it will only come into effect once we upload to Azure. We could still get the passed in function key from the request header for testing purposes, but it won't be validated during the call.

Lets push these functions up to Azure and test them there.

Deploy to Azure

Right-Click on the system function project and choose publish. Work through the wizard, logging into your Azure account, choosing 'Azure Function App (Linux)' and then your system function.

This will generate you a publishing profile, once this has completed choose Publish and after a minute or so you should receive a 'Publish succeeded' message.

Go ahead and publish our user function in the same way.

If you happen to receive errors during this publish stage quite often if you head over to Azure and stop the function and then retry the publish that will clear the issue.

Once published head over to our User function in Azure and you should see our GetUserData function listed at the bottom of the page (Sometimes you might need to restart and refresh the app a few times to see this) and the URI in the top right.

Lets go ahead and call the GetUserData function like we did locally.

This time notice that we get a 401 Unauthorized response, that's because the AuthorizationLevel.Function has now activated and we now need a function key to call the API.

Head over to the App keys tab in azure and notice that we have some host keys

If we copy the default key and then add a x-functions-key header, which includes that default key value, to our request in Postman then our request should now complete successfully.

Next Steps

Mission complete right, we have now published our functions to Azure and protected them with a key....

....the trouble is, we want to provide our users with their own API Keys, rather than a global key that everyone has, so that we can add and withdraw access as required.

While we could manually create new host keys for the functions, we really want a way to manage that easily, programmatically generate our keys, and potentially provide access to specific calls for different users.

For that, we'll need a better way to manage access and we'll use Azure API Management.

Head on over to Part 2 to continue this.

Pete

Converting ASP.NET Core user secrets to environment variables at runtime

Peter Davis — Fri, 11 Aug 2023 16:06:15 +0000

I'm currently in the process of building Panache Sports with a front end coded as a server side Blazor web application backed by serverless Azure Functions.

For the front end I'm using the secret manager to prevent me from checking configuration details into GitHub.

This works well and is simple to use. In my project directory I run the following in the .NET CLI:

dotnet user-secrets init

This creates a 'secrets.json' specific to the project on my local machine in this location:

%APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json

I can then add new secrets using the command:

dotnet user-secrets set "Mysecrets:ApiKey" "12345"

Or, I can use the 'Manage User Secrets' option when right-clicking on my project in Visual Studio:

Which allows me to directly edit the json file.

In my code I can then simply refer to these via the IConfiguration interface as if they had been placed in 'appsettings.json' within the project, so for example in the 'Program.cs' file of my project I could do the following to get the secret I created above:

var mySecret = builder.Configuration.GetValue<string>("Mysecrets:ApiKey");

So I can create and access configuration details when running the code locally, knowing that my API Keys and other data won't be checked into GitHub for others to see.

When I push this project from my local development environment up to an Azure Web App Service the configuration setup will be stored as environment variables, but Azure will link everything up so that my 'builder.Configuration.GetValue' call still works.

Azure Functions

When developing the Azure Functions locally that I use for the backend data access I can use environment variables to get the same type of details.

The Azure Function project in Visual Studio includes a 'local.settings.json' file, which Git ignores by default, where I can store my configuration details.

I can then access those values via the 'System.Environment' call:

var mySecret = Environment.GetEnvironmentVariable("My_Secret_Api_Key");

And again, this works locally and when I publish my Function to Azure.

The shared code issue

Now comes the problem I faced. I have some code that I'm using in my Azure Function that I would like to re-use in my Web application, and in the Azure Function that's using a 'Environment.GetEnvironmentVariable' call.

No problem right, that call will work quite happily when I push the code to Azure, so the following lookup:

var mySecret = Environment.GetEnvironmentVariable("My_Secret_Api_Key");

will return the exact same value as getting it from the configuration via.

var mySecret = builder.Configuration.GetValue<string>("My_Secret_Api_Key");

So I can reuse the code in both projects without any changes and all will work fine in Azure.

But what about when running locally? That's where I now have an issue.

var mySecret = Environment.GetEnvironmentVariable("My_Secret_Api_Key");

looks in a different place to 'appsettings.json' or the secret manager. It looks at the environment variables stored in 'launchSettings.json', which are also available via the launch profiles screen:

Although I can avoid checking these into GitHub as well the main issue is that I'm now storing some configuration in the secret manager while the rest needs to be stored in the build configuration.

That's not a major problem, but it is a pain to maintain two areas. It would be much nicer if I could put everything in the secret manager and not have to change any of my calls.

Well it turns out I can.

Setting environment variables via code

Not only can I read environment variables at runtime, but I can also set them at runtime with the following call:

Environment.SetEnvironmentVariable("My_Secret_Api_Key", "12345");

Now I can read my secret from the secret manager and set it as an environment variable at runtime:

var mySecret = builder.Configuration.GetValue<string>("My_Secret_Api_Key");

Environment.SetEnvironmentVariable("My_Secret_Api_Key", mySecret);

The only issue with this is that it is unnecessary when I'm running in Azure, so I can limit it to only setting the environment variable when running in my (local) development environment.

You can determine the environment in different ways, but in my case I simply do the following in the 'Program.cs' of my web application:

var mySecret = builder.Configuration.GetValue<string>("My_Secret_Api_Key");

var app = builder.Build();

if (app.Environment.IsDevelopment()) 
{
Environment.SetEnvironmentVariable("My_Secret_Api_Key", mySecret);
}

Now when I'm running locally I store all my configuration in the secret manager and convert them at runtime to environment variables if required.

When I push the code to Azure everything gets set using the configuration tab of the Azure web app service and the code works without any changes.

Pete

Switching from SQL Server to Azure CosmosDB

Peter Davis — Fri, 04 Aug 2023 11:11:48 +0000

Switching from SQL Server to Cosmos DB

For the past 20 years or so I’ve been using and developing solutions using relational databases, primarily SQL Server, but I’ve also used Oracle and MySQL.

As I began to plan out development for Panache Sports my initial thoughts were to do what I’ve always done and use a relational database to store all my data. I’d normally look at using MySQL to keep costs down, but with the resources provided by Microsoft Startups the option to use a managed Azure SQL database was also available.

This would have been the easiest route to go for, but as I looked at the data I’d be storing for Panache Sports, and how I was building it using serverless Azure functions, I wondered if there was another way to do things.

Enter Azure Cosmos DB.

With Panache Sports being designed as a serverless SaaS (Software as a Service) solution, being able to take advantage of a high availability, scalable database like Cosmos DB was very appealing. But there was a big downside, throwing away 20 years of relational database modelling to move to a NoSQL, unstructured database was going to have a significant learning curve.

So, if you’ve never looked at Cosmos DB, or have wondered if it might be applicable to your use case, here’s my introduction, including some of the decisions around why I choose to go down this route.

Data Decisions

Lets first look at how I would have structured my data using a relational database. This will be a very high level simplified example, but it should cover some of the key concepts.

Panache Sports is designed to store data about sports organisations and the staff and players of those organisations. So let’s think of a simple example structure for storing that information.

We’ve approached things in the normal relational DB way. We’ve normalised our data so that we have tables storing specific chunks of information. We’re using header and detail tables to split the data we store for users and organisations, and tables like Address are shared between users, organisations and Players so we keep that data in a single consistent location.

For organisations, we’re storing the type of sport (football, Baseball, Formula One etc.) but we link to a separate table via an Id so that we can add new sports types and change names and descriptions without changing anything on our organisation, knowing we’ll always get the latest data.

All looks fine and normal, you’ probably make changes, but this is just a simple example.

So how would we move this to Cosmos DB.

Moving our relational structure to Cosmos DB

Cosmos DB, in its NoSQL mode, which is what I’ll be discussing here, stores unstructured data in JSON format. So rather than creating rigid table structures, you can simply store any JSON document in a Cosmos DB container that you want, even completely different structured JSON documents in the same container.

At a very high level Cosmos DB consists of Databases, Containers and Items.

Databases contain a group of Containers and then each container stores items.

Containers can have Triggers, Stored Procedures and Functions which allow you to maintain the items they store. These triggers and functions can be built as fully functional .NET Azure functions.

Items within a container are JSON documents and individual documents within the same container can have completely different structures, however you must define a partition key for the container and all items in a container must use the same partition key, even if the rest of the data is different.

Partition keys are simply a property on your JSON items and define how data is logically ordered within a container, and Cosmos DB will then distribute data across physical partitions automatically. The partition Keys are used to for writing and updating and this key can also be used in “where” clauses to provide efficient data retrieval.

Logical partitions can store 20GB of data, so making decisions on how you partition your data is important.

I’ll probably create a separate post that covers containers and partitions but for more background take a look at https://learn.microsoft.com/en-us/azure/cosmos-db/resource-model

Cosmos DB allows for Geo-Replication of data so you can make sure that data is always accessible in a physical location close to your users, however the cost increases significantly with this.

In Cosmos DB RUs are what matter. Rather than basing pricing tiers on storage space or compute Cosmos DB pricing is based on RUs, Request Units, per second.

An RU is 1 point read of 1kb of data in Cosmos DB. So if we need to read 20 kb of data, that’s going to be around 20 RUs.

You can get started with the free tier of Cosmos DB that provides 1000 RUs per second, and If you want a good introduction to RUs check out this Cosmos DB essentials episode https://www.youtube.com/watch?v=3naCwuXhLlk

Although we could build our Cosmos DB data structure in exactly the same way as we built our relational data structure, we start hitting issues with our RU charge increasing.

Microsoft provides a really good example of how you might model data here https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/model-partition-example and I suggest having a read through this.

But let’s take a look at our original data. If we want to get a complete Organisation record we need to query 4 separate tables and this begins to significantly increase our RUs.

If I split the same data up in Cosmos DB I’d have 4 items holding parts of the data I need so when looking up an organisation I’d need to lookup and return 4 objects and then reconstruct this data, this greatly increases the RUs required to look up a single Organisation record.

For Cosmos DB, I’m not so much worried about the size of database, as storage costs are increasingly becoming smaller and smaller, instead I want to limit the amount of lookups I’m performing.

So how about this for storing our data:

Now if we want to get an organisation record we only require one lookup, and because a JSON document is small the size isn’t really a factor.

If we have an API frontend to our system then the API call to get an organisation returns the data exactly as it is stored in Cosmos DB, I don’t need to use separate classes/objects for my database and my APIs.

What’s not to like right?

….Oh dear, I can hear a collective sigh being let out by all the DB Admins I’ve ever worked with.

For one, what about the sport type? If “Formula One” was to change it’s name to “Formula Super One” I’d have to update every record that was using it, instead of a single record.

If I want to find out all the addresses I have in the system I’ll need to look through every organisation, user and player record rather than on a single table.

Thinking differently

Because I’m trying to prevent additional lookups wherever possible I’m going to end up duplicating data across my records, and this is where in Cosmos DB we need to throw out everything we’ve previously learnt about normalising our data and instead make decisions on the frequency that data will be read and updated.

In my organisation record above, yes if I have 20 teams in the system and “Formula One” changes it’s name to “Formula Super One” I’m going to have to change 20 records, rather than just 1. But what is the frequency of that? It’s very unlikely that data like that will change so I will make the decision to take the hit on a large multi-record update at some point in the future knowing that I am saving on reads and speed of access on a day to day basis.

In Cosmos DB you want to de-normalise your data, storing duplicates of information in records to speed up reads if you know the likelihood of that data changing is small. We’ll probably be okay with storing duplicate addresses across multiple records because we know that the address of a teams stadium or head office isn’t likely to change all that often.

What about if I wanted to get a list of my organisations. One way of doing that would be to add a document type and then store a separate list of organisations, something like this:

Here I’ve added a type of “organisation” to my record and I can store another record with a completely different structure in the same Cosmos DB container with a type of “organisationlist”:

Now, if I want to display a list of organisations I look for a document with the type of “organisationlist” and return a single record with summary data, rather than querying for multiple organisations. If I then want to get the full organisation I’ve got the Id in this list so I can perform a direct lookup to get it.

Instead of doing a lookup that returns 20 organisation records we will return 1 list record, and if we need more information we can perform a second lookup as required.

Again, we’re duplicating data between the master record and the list record, and we have to keep these in sync, but we’re doing that with data we know isn’t going to change all that often so we’re prioritising the read operations for speed and cost.

This de-normalisation goes against everything we try to achieve in a relational database, and how I’ve learnt to structure things over the past 20 years, so it really takes some getting used to.

At a basic level, we’re structuring our data to prioritise how it’s going to be accessed. If we know it won’t change often but will be read frequently we’ll prioritise the reading of that data and duplicate information across records, where as if we know that the data will be updated frequently we may decide to normalise it as we would have done in a relational database and prioritise the writes.

In Cosmos DB the storage cost isn’t really the concern, the RUs are, and so how you structure your data will depend on how you think it will be accessed.

I’ve worked a lot with large ERP systems that store vast amounts of transactional data, like invoices, purchase orders and wages, with hundreds if not thousands of records moving through the system daily. For that data a relational database may make sense, where you avoid duplication of data and update small chunks of information rather than large records. But for Panache Sports we’re dealing with Organisations and player/staff data, this is unlikely to change on a day-to-day or even week-to-week basis, as such we can make use of Cosmos DB to efficiently store and retrieve our information frequently, and only need to worry about updates every now and then.

I’ll look to cover some further topics around Cosmos DB, like how you structure containers and their partition keys, which are a crucial component of making sure you can organise your data appropriately, but for now I hope this has provided a good introduction to how you need to think differently about structuring your data if you’re thinking about a move from a relational database to Cosmos DB.

Like everything, Cosmos DB isn’t appropriate for every situation, or all types of data, but it may work for you in specific areas.

Pete

Windows Terminal Themes

Peter Davis — Thu, 16 Jun 2022 12:21:07 +0000

A quick guide showing how to change your Windows Terminal from looking like this.

To this.

Install Fonts

Download the Cascadia Code (aka Caskaydia Cove Nerd Font) from Nerd Fonts.

Direct Link: Caskaydia Cove Nerd Font

Unzip the file and double click on the Caskaydia Cove Nerd Font Complete file and choose Install.

Install PowerShell Core (Optional)

If you want to use the new cross platform and open source version of PowerShell head over to GitHub and follow the install instructions applicable to you.

In this example we'll download the Windows (x64) MSI (Direct Link) and then run the MSI to install.

Once finished, re-open windows terminal and you should see a new PowerShell option.

You can also change your Windows Terminal start-up options to set PowerShell Core as your default profile.

The following steps will need to be performed in both PowerShell versions if you want to match your themes across both

Set default Font

Within Windows Terminal Settings > PowerShell > Appearance change the Font face to CaskaydiaCove Nerd Font.

Install Oh My Posh

Oh My Posh is the package that allows us to theme our terminal window. The installation process has changed several times since I started using it a couple of years ago but it's now pretty simple.

Make sure you have the latest version of App Installer from the Windows Store as this will provide you with winget, this should be installed by default on Windows 11 but may require an update.

App Installer

In PowerShell run the following command.

winget install JanDeDobbeleer.OhMyPosh

Create a new PowerShell profile if one doesn't exist with the following command.

if (!(Test-Path -Path $PROFILE )) { New-Item -Type File -Path $PROFILE -Force }

If you're using the original PoweShell run the following command as administrator to allow script execution.

Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope LocalMachine

This is not required for PowerShell Core.

To install new terminal icons (these change the folder/file icons in terminal) run the following command.

Install-Module -Name Terminal-Icons -Repository PSGallery

Edit your profile by using the following command.

notepad $PROFILE

Add the following lines, where {username} is replaced with your Windows username.

oh-my-posh init pwsh --config C:\Users\{username}\AppData\Local\Programs\oh-my-posh\themes\paradox.omp.json | Invoke-Expression
Import-Module -Name Terminal-Icons

Save and then restart Windows Terminal and you should see your new theme.

Other Themes

You can see the other themes available to you at Oh My Posh Themes, or by running the following command.

Get-PoshThemes

You can then change paradox.omp.json in your profile file to a different theme, i.e. jandedobbeleer.omp.json.

Pete

Making a local MicroK8s environment available externally (Part 5 - Reverse Tunnels)

Peter Davis — Wed, 15 Jun 2022 11:03:29 +0000

First a warning

If you've been following along with our previous steps you will have created your own local Linux VM running Microk8s, in this step we're going to expose this VM to the public internet.

Making a hole in yours, or anyone's network, to expose a machine, virtual or not, increases that networks risk of attack. You are responsible for your security and the risks created by following these steps.

...On with the show!

Reverse Tunnels

Because I'm running my MicroK8s Linux VM at home on my desktop it isn't exposed to the public internet without me needing to do some work.

I could pay extra for a static IP address, but depending on your ISP that may simply not be an option, or it could be prohibitively expensive.

As an alternative, we can use reverse tunnels to route network traffic to and from our local machine to the outside world, regardless of whether we have a static IP or not.

Previously I've used services like ngrok to perform this, and there is a list of other options here. But quite often when using a paid service I've quickly hit on limitations, like a limited number of addresses and not being able to route traffic across multiple ports. Because of this I looked to find another option, and like the rest of this series, I wanted to do it myself.

Enter SSH, which we can use to provide a tunnel from one server to another.

So far we've made use of free open source software....well, apart from Windows 11 pro, but I'm guessing that the majority of the audience already had that, but for this stage one of the things we will need is our own server already accessible to the outside world, which in general, is going to cost us something.

There are a huge amount of options here, varying in price and provider, but for this example I'm going to be creating and hosting a VM in Microsoft Azure. If you don't have an account with Azure, you can sign up for a free one and receive $200 credit for 30 days if you just want to test things out, alternatively you can use any other provider like digital ocean, AWS, Google Cloud, the choice is yours.

Now before we start to worry too much about cost, all we're going to be using this VM for is forwarding data to and from a couple of ports, so we're going to build all this on the cheapest VM we can get away with.

Last but not least, before we get started, I used the following guide from the brilliant Jeff Geerling when I was trying this out for the first time on a Raspberry Pi cluster I built (maybe that will be the basis for another guide!). I urge you to check out Jeff's page and YouTube channel, he makes great videos.

Create a Linux VM....again!

This time though we're creating a new VM using Azure. Now of course we could have created our first VM in Azure and then wouldn't have needed to go through this step, but the cost of hosting a full Linux VM running MicroK8s and our services could be costly, this way we're paying for a minimal VM.

With an azure account head into the portal and choose the option to 'Create Resource'.

And then choose the option to create a virtual machine.

The following are the settings I used for creating my VM but you may want to adjust these as you see fit. Pay close attention to the 'Size' as this determines the cost of your VM. I've chosen the minimum possible for my subscription, but if you want to use this VM for other things you may want to increase the performance here.

For 'Disks' you may want to change from premium storage to standard HDD which again reduces the cost.

My 'Network' settings are set as follows.

Make sure you have the 'Public IP' set to '(new)', and also make note of the inbound ports that are allowed (also shown on the first screen). This should default to allow connections via 'SSH' (port 22) and this will allow us to login to our VM once it's created.

Go ahead and create the machine, which should only take a few minutes, and once complete you should be able to see the resource in your dashboard.

Open up the terminal on your local machine and you should be able to SSH to this new VM using the IP address shown.

ssh {username}@{Azure VM IP}

Setup SSH

The first thing we need to do on our Azure VM is setup SSH correctly. We need to enable the AllowTCPForwarding option, which should be set to yes by default, but we can check it with the following command.

sudo sshd -T | grep -E 'allowtcpforwarding'

you should see the following.

allowtcpforwarding yes

We now need to enable GatewayPorts which you can do by editing the SSH Config file.

sudo nano /etc/ssh/sshd_config

Find the line that reads #GatewayPorts no and change this to GatewayPorts yes.

Exit out of nano CTRL+X, Y, ENTER and run the following to restart SSH.

sudo systemctl restart ssh

Now using the following command you can check that both options are enabled.

sudo sshd -T | grep -E 'gatewayports|allowtcpforwarding'

Back on our Local Linux VM we need to configure things to allow us to connect directly via SSH to our Azure VM. This involves us creating an SSH key pair.

Run the following command

ssh-keygen -t ed25519 -C "{Local VM HostName}"

And press Enter for all the prompts.

Get the contents of the created file using the following command.

cat /home/{username}/.ssh/id_ed25519.pub

Copy the string returned and then back on your Azure VM edit the
~/.ssh/authorized_keys file and paste the copied string in.

sudo nano ~/.ssh/authorized_keys

Exit out of nano CTRL+X, Y, ENTER and head back to your Local VM.

You should now be able to SSH to your Azure VM from your Local VM without the need for a password, simply use.

ssh {username}@{Azure VM IP}

answer yes when prompted and you should simply connect to the VM. you can type exit to return to your local VM.

Open Ports

As we saw when we created our Azure VM, apart from port 22 for SSH connections, most ports are disabled by default.

The Panache Legal services we have running on our local VM use ports 30000-30010 so we'll need to open those up. Choose the Networking menu item of our Azure VM and then click on Add inbound port rule and create a rule for our ports.

Setup AutoSSH

autossh will allow us to persist our connections. We could simply create the tunnels manually via the command line using something like.

ssh -nNTv -R 0.0.0.0:8080:localhost:80 {username}@{Azure VM IP}

This would map our webservers http port to our Azure VM, but you would need to run this after reboots, or when we log out of our VM, to maintain the connection. With autossh we can ensure that this is always running, but first, we need to install it.

On our Local VM run the following command.

sudo apt install autossh

Once the install completes create an autossh config file with the following command.

sudo nano /etc/default/autossh

Within this new file add the following lines.

AUTOSSH_POLL=60
AUTOSSH_FIRST_POLL=30
AUTOSSH_GATETIME=0
AUTOSSH_PORT=22000

Following those lines we need to add configuration for each of the ports. This requires a line beginning with SSH_OPTIONS=" -N then followed by -R 0.0.0.0:30001:localhost:30001 {Azure VM Username}@{Azure VM IP}, repeated for each of the ports. So for example.

SSH_OPTIONS="-N -R 0.0.0.0:30001:localhost:30001 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30002:localhost:30002 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30003:localhost:30003 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30004:localhost:30004 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30005:localhost:30005 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30006:localhost:30006 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30007:localhost:30007 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30008:localhost:30008 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30009:localhost:30009 {Azure VM Username}@{Azure VM IP} -R 0.0.0.0:30010:localhost:30010 {Azure VM Username}@{Azure VM IP}"

Exit out of nano CTRL+X, Y, ENTER.

Next we need to tell systemd about autossh.

Create a new file with.

sudo nano /lib/systemd/system/autossh.service

Add the following, making sure to set the {username} placeholder appropriately to the username you use on your local VM.

[Unit]
Description=autossh
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User={username}
EnvironmentFile=/etc/default/autossh
ExecStart=/usr/bin/autossh $SSH_OPTIONS
Restart=always
RestartSec=60

[Install]
WantedBy=multi-user.target

Exit out of nano CTRL+X, Y, ENTER, then add a symlink for systemd.

sudo ln -s /lib/systemd/system/autossh.service /etc/systemd/system/autossh.service

Finally run the following commands to tell systemd about autossh along with enabling it on startup.

sudo systemctl daemon-reload
sudo systemctl start autossh
sudo systemctl enable autossh

We have one final step, assuming you're using the Panache Legal containers, if you remember when we created our deployment files in step 4 some of the environment variables referred to our Local VM, this is where you replaced {server-IP} with the IP address of the local VM. You now need to change this to the IP address of your Azure VM, although you can leave {db-server-IP} as it is because the MySQL connection does not need to be routed across the internet.

Again, if you're using the Panache Legal containers you can run the DeleteService.sh script to remove all the pods, make your changes to the deployment files and then run the StartServices.sh script to recreate all the pods.

If all has gone according to plan you should be able to visit http://{Azure VM IP}:30001 in your browser and, fingers crossed, you should see the Panache Legal login page and then be able to login to the system.

All Finished :o)

That's it.

We've created a local Linux VM.

We've installed MicorK8s and MySQL on the local VM.

We've installed NGINX and phpMyAdmin.

We've spun up pods in MicroK8s using deployment files.

We've created an Azure Linux VM.

and finally we've configured Auto SSH and used a tunnel to access our local VM via the internet using our Azure VM.

Hopefully this has given you all the tools you need to build your own environments and expose them to the outside world if you need.

In Closing

Keep a look out for further tutorials and posts that I'll be putting out.

And please take a look at Panache Legal, it's a fully Open Source application built using .NET, with Blazor and identity server. It's still in active development so consider it pre-alpha, but take a look and why not get involved.

Pete

Making a local MicroK8s environment available externally (Part 4 - Running Docker Containers)

Peter Davis — Wed, 15 Jun 2022 11:03:21 +0000

As I mentioned previously, I'm currently building a .Net based Open Source LegalTech platform called Panache Legal. For development and testing I build docker images for the various microservices which are then hosted on Docker Hub and while I can use these with Docker Desktop, I'd prefer to get things up and running with Kubernetes so that I can learn more about that environment and also test things like scaling, which is why we're building this MicroK8s system.

In this example I'm going to be showing how I get Panache Legal up and running, but you may want to choose different containers more suited to your needs, either way, this should still provide the groundwork you need to get going.

Deployment files

To spin up our containers in MicroK8s we could simply jump straight into the command line and deploy our app that way using the following

microk8s kubectl create ......

But for Panache Legal we have multiple containers we need to create, all of which have various command line arguments, and this can become a little cumbersome.

Instead of the command line we'll use deployment files so that we have all the setup in an easy to maintain place.

For Panache Legal I maintain some basic deployment files in the GitHub repository here.

There are individual deployment files for each of the microservices, along with two other scripts DeleteServices.sh and StartServices.sh which provide an easy single script to run which will create or remove all of the services in one go.

Lets take a look inside these files and see how they are setup. For reference you can also check out the kubernetes documentation for deployments.

Let's have a look at the Client service, which provides a .NET Web API for storing and maintaining client information. This Microservice has its own database associated with it and is protected by JWT token authentication, handled by the Identity service.

Looking at the first part of the file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: panachesoftware-service-client
  labels:
    app: panachesoftware-service-client

We specify that this is a Deployment and we provide a name for the app panachesoftware-service-client This is the internal name MicroK8s will use for the app and can be set to whatever you need.

The spec part of the file.

spec:
  replicas: 1
  selector:
    matchLabels:
      app: panachesoftware-service-client
  template:
    metadata:
      labels:
        app: panachesoftware-service-client

Sets out what pods this deployment will manage, in this case we're keeping it simple so we just reference the name we provided in the metadata section.

In addition here we specify the number of replicas. For the moment I simply use 1 replica, but the microservice based approach to Panache Legal means that certain services, like Identity and the UI may be set to use multiple replicas so that more than one instance of the service will run, allowing our application to scale as demand increases.

The next section provides details of the container itself.

spec:
      containers:
      - env:
          - name: ASPNETCORE_ENVIRONMENT
            value: Development
          - name: ASPNETCORE_URLS
            value: http://+:55005
          - name: ConnectionStrings__MySQL
            value: server={db-server-IP};port=3306;database=PanacheSoftware.Client.K8S;user={db-user};password={db-password};GuidFormat=Char36
          - name: PanacheSoftware__CallMethod__APICallsSecure
            value: "False"
          - name: PanacheSoftware__CallMethod__UICallsSecure
            value: "False"
          - name: PanacheSoftware__CallMethod__UseAPIGateway
            value: "True"
          - name: PanacheSoftware__DBProvider
            value: MySQL
          - name: PanacheSoftware__Secret__ClientServiceSecret
            value: AA04416A-A87B-4D88-956B-27CBFFCC2802
          - name: PanacheSoftware__StartDomain
            value: panachesoftware.com
          - name: PanacheSoftware__Url__IdentityServerURL
            value: http://{server-IP}:30002
          - name: PanacheSoftware__Url__IdentityServerURLSecure
            value: https://{server-IP}:30002
        image: panachesoftware/panachesoftwareserviceclient:latest
        name: panachesoftware-service-client
        ports:
        - containerPort: 55005
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet

The name, value pairs shown here all correspond to environment variables expected by the service, so if you were to look at the appsettings.json file of the client service .NET project, you will see corresponding settings.

"ConnectionStrings": {
    "MSSQL": "Data Source=localhost;Database=PanacheSoftware.Client;User ID=sa;Password=Passw0rd123!;Connect Timeout=30;Encrypt=False;TrustServerCertificate=False;ApplicationIntent=ReadWrite;MultiSubnetFailover=False",
    "MySQL": "server=raspberrypi;port=3306;database=PanacheSoftware.Client;user=pi;password=Passw0rd123!;GuidFormat=Char36"
  },
  "PanacheSoftware": {
    "DBProvider": "MySQL",
    "StartDomain": "panachesoftware.com",
    "CallMethod": {
      "APICallsSecure": "false",
      "UICallsSecure": "false",
      "UseAPIGateway": "true"
    },
    "Url": {
      "IdentityServerURL": "http://localhost:55002",
      "IdentityServerURLSecure": "https://localhost:44302",
      "APIGatewayURL": "http://localhost:55003",
      "APIGatewayURLSecure": "https://localhost:44303"
    },
    "Secret": {
      "ClientServiceSecret": "1314EF18-40FA-4B16-83DF-B276FF0D92A9"
    }
  }

All I've changed in the example deployment files is to put placeholders, like {db-server-IP} and {db-user}, in the file where you will need to change these to be applicable in your system.

We setup MySQL in a previous part so you should have the relevant username and password for the services that require a database, and we're using the same machine to host MicroK8s and our MySQL database so {db-server-IP} and {server-IP} will be the same.

if you need to determine the IP address of your VM you can use the following command

hostname -I | awk '{print $1}'

Be aware that in general, a new IP address will likely be assigned each time you VM starts. One way to change this is to update the DHCP IP reservations within your router to make sure that the IP address assigned to your VM (based on its hostname) always remains the same.

Outside of the environment variables there are a couple of other important items.

image specifies the docker image that will be used in this deployment. This is the image name along with the appropriate tag, in this case the client service with the latest tag.

panachesoftware/panachesoftwareserviceclient:latest

Alongside this the containerPort specifies the port number that the service will be running on, in this case 55005.

When an app is deployed in MicroK8s it will be made available on the cluster IP, however we want to make the service available by going to the IP of the VM itself so we add an additional section to the deployment file.

---
apiVersion: v1
kind: Service
metadata:
  name: panachesoftware-service-client-service
spec:
  type: NodePort
  selector:
    app: panachesoftware-service-client
  ports:
    - protocol: TCP
      port: 55005
      targetPort: 55005
      nodePort: 30005

You could have included this service detail in a separate file, but for our setup it is easier to include it within the deployment file.

By choosing NodePort for our type it specifies that the app will be exposed via a port on the host, rather than on the cluster IP. If you don't add anything further it will be assigned a random port within the '30000-32767' range by default.

We need our services to talk to each other, as shown by the references to other other parts of the platform in the environment variables i.e. PanacheSoftware__Url__IdentityServerURL providing the address of the Identity Server, so we need to specify a specific port that will always be used, this is the final section.

ports:
    - protocol: TCP
      port: 55005
      targetPort: 55005
      nodePort: 30005

Where we tell MicroK8s to map the port '55005' of the service to be exposed on port '30005' of the host.

Performing the deployment

Now we've gone through the details of the deployment files go ahead and run through the Panache Legal examples adjusting the placeholders as appropriate.

If you edit the files on another machine, maybe the Windows Host rather than on the Linux VM we created then you can copy the files to your Linux VM using a tool like WinSCP.

If you want to use the provided scripts to run the deployment files for you then make sure to set those files as executable with the following commands.

chmod +x DeleteServices.sh
chmod +x StartServices.sh

One thing to note here is that if you have the Linux Firewall running, as discussed when we were setting up the MicroK8s dashboard, you may need to allow the ports that the services will use to be exposed. MicroK8s will actually edit iptables behind the scenes and allow access to the nodeport ports, 30001-30010 for our services, but if we go to these addresses in our browser it will fail, we need to enable the 55001-55010 port range in ufw, which is confusing, but works!

Allow these ports via the following command.

sudo ufw allow 55001:55010/tcp

Now, either run the StartServices.sh script with the following command.

./StartServices.sh

Or start-up your services individually with something like.

microk8s kubectl apply -f panachesoftware-client-service.yaml

If you're running the StartServices.sh script, once it completes you should be able to see the 10 pods it creates starting up by using the following command.

microk8s kubectl get pods -o wide

You can also watch the progress of the pod creation via the dashboard we configured in Step 2.

On first creation it will take a bit of time to complete as the various containers need to be downloaded from Docker Hub, but after a while you should see the status of the pods move from ContainerCreating to Running.

Once everything is running let's test it out, on your windows host you should be able to got to http://{VM-IP-Address}:30001 and see the Panache Legal Login screen.

Assuming you kept the PanacheSoftware__StartDomain environment variable as panachesoftware.com the system will have started up and created a default user with the details, Username: admin@panachesoftware.com, Password: Passw0rd123!. So hit Login and use those credentials to access the system.

In the left menu, if you go to Settings > API Access you should see a list of the individual services that make up the Panache Legal Platform, all of which are hosted as separate pods in MicroK8s. On this screen you can see the links to those services swagger definition, and you can note that they are all assigned to the ports specified in the deployment files.

If you head on over to phpMyAdmin, that we setup in step 2, you should be able to see that all of the databases associated with our services have been automatically created via the entity framework.

If you find that any pods didn't create, maybe they are stuck with a status of crash loop or some other error, you can check the logs for the service within the 'Pods' area of the dashboard, just click on the 3 dots on the right hand side and choose 'Logs'.

In general, the majority of errors often relate to the service not being able to connect to a database, so check usernames and passwords and all the access settings we configured in Step 3.

Next Steps

The main bulk of the work is now complete. So far we've.

Created a Linux VM
Setup MicroK8s
Setup MySQL
Setup NGINX
Setup phpMyAdmin
Spun up some services in MicroK8s

If all we need to do is host some docker containers locally then your work is done. But if you want a little bit more, lets head over to our final step where we're going to make our local environment available to the outside world by using SSH.

Pete

Making a local MicroK8s environment available externally (Part 3 - NGINX and phpMyAdmin)

Peter Davis — Wed, 15 Jun 2022 11:03:12 +0000

If you're following along with the steps to get our MicroK8s environment up and running this part isn't strictly necessary, but getting phpMyAdmin up and running can make the administration of our MySQL installation easier.

As part of this, we'll also look at installing the NGINX webserver and enable this through the built in firewall in Ubuntu.

If you haven't already SSH into our VM using windows terminal, in my case using the following command.

ssh pete@pl-k8s-vm-1

Install NGINX

To act as our webserver for hosting phpMyAdmin we're going to install NGINX.

This should be pretty quick and easy, simply run the following,

sudo apt update
sudo apt install nginx

Enable Firewall access

Now we've installed NGINX run the following

sudo ufw app list

This will provide us with a list of applications registered with the firewall that we can allow access to. In this instance we'll just enable Nginx HTTP as we don't require https access at this point.

...oh, and while we're here we'll also allow OpenSSH access as well.

Run the following commands.

sudo ufw allow 'Nginx HTTP'
sudo ufw allow OpenSSH

Once this is done we can check the status to make sure we're allowed access through the firewall with the following command.

sudo ufw status

This should return something like the following.

Once this is done we can check that the webserver is up and running by launching a web browser on our host machine and simply inputting our VMs hostname, in my case http://pl-k8s-vm-1, and hopefully you should see the welcome to nginx page.

Installing PHP

Before we can install phpMyAdmin we need to go ahead and install PHP and then setup our NGINX webserver to serve php pages.

Run the following command to install PHP.

sudo apt install php-fpm php-mysql

After this we need to tell NGINX to process our php pages so run the following to edit the config.

sudo nano /etc/nginx/sites-enabled/default

Within the editor find the line referencing the index files that are processed:

And add

index.php

to the list:

In the same config file find the reference to PHP scripts to FastCGI server:

And add the following.

location ~ \.php$ {
                include snippets/fastcgi-php.conf;
                fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
        }

Exit out of nano CTRL+X, Y, ENTER, and the get NGINX to reload its config with the following command.

sudo systemctl reload nginx

Once this is done lets quickly test if PHP is being correctly processed by NGINX.

Run the following command to create a demo php file in the route of the webserver.

sudo nano /var/www/html/index.php

And then add the following line of code.

<?php phpinfo(); ?>

Exit out of nano CTRL+X, Y, ENTER, reload the url we originally used to test our NGINX install, in my case http://pl-k8s-vm-1/, and hopefully you should see the PHP Info page:

Install phpMyAdmin

We're almost there, just phpMyAdmin left to setup.

Run the following command.

sudo apt install phpmyadmin

The installation will run and you'll be presented by a wizard with options to choose from. The first one will ask you to select which webserver to use, now NGINX won't be in the list, but we won't let that stop us, choose apache2 instead by pressing SPACE, TAB then ENTER.

Next we'll be asked to configure phpMyAdmin:

press ENTER on YES and in the next screen set a password of your choice for phpMyAdmin to use to connect to the database server.

Password validation

If during the MySQL installation you choose to enable a 'Validate Password plugin' you may receive password does not satisfy the current policy requirement errors when trying to create the phpmyadmin user. If this is the case the easiest way I found to fix it was to choose the option to ignore the error then log into MySQL with the following command.

mysql -u root -p

Run the following.

UNINSTALL COMPONENT "file://component_validate_password";

and exit out.

You can then run the following to remove and then re-install phpMyAdmin.

sudo apt remove phpmyadmin
sudo apt install phpmyadmin

This time your password should be accepted. If you then want to re-enable the password validation you can log back into MySQL and run the following.

INSTALL COMPONENT "file://component_validate_password";

Continuing the phpMyAdmin install

Once the install has completed we want to create a link from the phpmyadmin share to the root of our NGINX webserver by running the following command.

sudo ln -s /usr/share/phpmyadmin /var/www/html

After this we should be able to access phpMyAdmin by going to the following address http://pl-k8s-vm-1/phpmyadmin

We can then use the account we setup in Step 2, mine was pluser with a password of 5ecurePassw0rd!, to access our MySQL installation with phpMyAdmin.

Next Steps

I think we've got everything in place now so it's about time we got something up and running in our new Kubernetes environment. Lets head over to Part 4 to get our docker containers up and running.

Pete

Making a local MicroK8s environment available externally (Part 2 - Installing MicroK8s and MySQL)

Peter Davis — Wed, 15 Jun 2022 11:03:03 +0000

In the first part of this series we made sure Hyper-V was up and running and then created our Linux VM running Ubuntu. We also tweaked that VM so that the resolution was better suited to our system, and then went ahead and got SSH up and running....so that we wouldn't need to log into the desktop anyway. 😉

So lets go ahead and get on with the main part of this, setting up MicroK8s and MySQL.

Setting up MicroK8s

First up, there are several versions of Kubernetes available for us to use, K3s, k3d, Kind, minikube and MicroK8s.

They all use different methods, like VMs, docker or snap for hosting, and they all have their pro's and con's. I settled for MicroK8s because it offered all the functionality I needed and setup was quick and easy, but as you investigate these tools you may settle on using something else, but at least MicroK8s will offer a good introduction.

Installation is simple, and as we just need the terminal lets SSH into our VM via Windows Terminal with the following command.

ssh {username}@{VM name}

Once we're logged in simply run the following to install MicroK8s.

sudo snap install microk8s --classic

This will take a minute or two, depending on your network speed, but you should see an installed tick once it finishes.

To make sure the install completed okay and everything is up and running we can run the following command.

microk8s status --wait-ready

When we do this you'll notice that we don't have permission to access MicroK8s, but luckily you will be provided with the commands you need to fix this, in my case, because my username is pete I need to run the following.

sudo usermod -a -G microk8s pete
sudo chown -f -R pete ~/.kube
newgrp microk8s

After this we can issue our microk8s status --wait-ready command again and hopefully we should see that MicroK8s is running.

If everything is up and running you should see microk8s is running.

Next up we'll install a couple of add-ons, for this simple setup we'll just install DNS, which is often used by other addons so is almost always needed, and then the dashboard so we have a nice web based interface to see what's happening with the cluster.

Run the following command.

microk8s enable dashboard dns

Once that's finished we can check what services are running with the following command, and hopefully you can spot the dashboard and dns services in the list.

microk8s kubectl get all --all-namespaces

The last thing we'll do is check that the dashboard is running by using the following command.

microk8s dashboard-proxy

This will return us a token that we can use for login and also the port number that the dashboard is running on.

As you can see our dashboard is running on port 10443. Let's check this from our Windows host by opening a browser to https://{VM Name}:10443. You'll likely receive a message about the connection not being private but simply carry on to the page, choose to login with a token, paste in the token you were provided above, and hopefully you'll login to dashboard where you can see the status of your install.

Linux Firewall

In later parts we will be enabling connections via the Linux firewall. In preperation for that it will be useful to make sure that access to dashboard is possible when the firewall is running.

To check if the firewall is active run the following.

sudo ufw status

if the status does not come back as active you can enable the firewall with.

sudo ufw enable

You can then allow access to the port the dashboard uses with

sudo ufw allow 10443

Setting up MySQL

Panache Legal is designed to run against SQL Server or MySQL (or MariaDB if you're running on something like a Raspberry Pi) so I could just install the free developer version of SQL server on Windows, or even SQL server for Linux, but I'd prefer to keep things Open Source so lets go with MySQL.

First up, lets update all our packages so we're ready for the install.

sudo apt update
sudo apt upgrade

Once that's done, lets perform the install.

sudo apt install mysql-server

This won't take long to run and once it's finished run the following command to ensure it's all up and running.

sudo systemctl start mysql.service

It's not strictly necessary, but best practice is now to run the security script. By doing this you'll set a new root password as well as disabling certain pre-installed features and configuration that could be used to gain access to the server. In general you should answer Y to all the questions and accept the changes it wants to make.

Run the script with the following command.

sudo mysql_secure_installation

If you receive an error message when trying to change the root password exit it and run the following commands before running the above command again.

sudo mysql
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password by '{some password}';

The next time you run mysql_secure_installation you'll need to enter the password you supplied in {some password} above when the script starts.

Once this script is finished we'll create a new user that can be used by our microservices to login and create their databases.

mysql -u root -p

Next create a new user with a username of your choice replacing {username} and also a password of your choice replacing {password} by issuing the following command.

CREATE USER '{username}'@'%' IDENTIFIED BY '{password}';

You'll notice in the above that instead of a hostname we provided % after the username. This will allow us to connect to the MySQL database from an external machine if we want.

For example, in the above I'm using CREATE USER 'pluser'@'%' IDENTIFIED BY '5ecurePassw0rd!';, now don't tell anyone my password, that's between you and me!

The Panache Legal Microservices we'll be running need to be able to create their own database, as they use a code first approach in Entity Framework, so we need to grant appropriate privileges to this new user. In this instance we'll just grant all, but you may want to be more restrictive in your environment, especially if this is a production environment!

GRANT ALL PRIVILEGES ON *.* TO '{username}'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
exit

Even though we've configured our user to allow connections from external systems MySQL itself by default will only allow connections from localhost. To change this we need to edit the mysqld.cnf file using the following command.

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Look for a line that says.

bind-address            = 127.0.0.1

and change this to.

bind-address            = 0.0.0.0

Exit out of nano CTRL+X, Y, ENTER, and then restart MySQL with the following.

sudo systemctl restart mysql

Next Steps

We've got our Linux VM running, we've installed MicroK8s and also MySQL so now lets go ahead and setup phpMyAdmin, along with the NGINX webserver so that we can easily administer our MySQL installation.

This phpMyAdmin setup is optional, and isn't required to get everything else running so if you want to skip that part and head straight to getting the containers running simply skip forward to Part 4.

Pete

Making a local MicroK8s environment available externally (Part 1 - Building a Linux VM)

Peter Davis — Wed, 15 Jun 2022 11:02:50 +0000

I'm currently building an Open Source LegalTech platform called Panache Legal which is built using .NET and consists of a number of microservices which are all designed to run in Docker containers. This all works great locally, but one issue is how to host a version externally that can be used by colleagues for testing and demos.

Of course that's easy right, I've got docker images uploaded to docker hub, so just spin those up as web apps in Azure (or your cloud provider of choice) and it's problem solved.

But where is the fun in that. I don't really want to pay for too many hosted services, but most importantly, I want to learn more about Kubernetes and the whole DevOps pipeline. In short, I'd like to do it myself.

So here's what we're going to do instead.

Create a Linux VM locally in Hyper-V
Install MicroK8s and MySQL (including phpMyAdmin)
Get Panache Legal (or your docker images of choice) up and running on that VM

The above gives us a local Kubernetes environment using MicroK8s, but because we want that accessible externally, and because I don't have a static IP from my ISP, we'll have to jump through some additional hoops to expose this to the outside world. So we'll follow this up and...

Create a Linux VM in Azure
Setup SSH port forwarding to send traffic to and from that Azure VM to my local VMs

Now, there are many different ways we could have done this, like using WSL, using a local linux workstation, running it all in a VM in Azure, running MicroK8s on Windows. Plus there are security considerations around opening a route to our local network....but we've got to start somewhere, so lets go with the route outlined above and just get going, and hopefully, even if the whole setup isn't valid for you, parts of it might be.

Creating our Hyper-V Linux VMs

Before we start, go ahead and download a copy of the Ubuntu installation ISO from Ubuntu desktop. Other flavours of Linux are available if you prefer.

I'm creating this environment in Windows 11 and you'll need the Pro or Enterprise version to use Hyper-V. If you're looking for a free alternative you could likely use VirtualBox to achieve the same result.

Hyper-V won't be enabled by default so you'll need to open Control Panel > Programs and Features > Turn Windows features on or off and make sure Hyper-V is enabled.

Open up Hyper-V Manager and the first thing we'll do is use the Virtual Switch Manager... action to create an External Switch to allow our VM to see the outside world.

Once that's done use the New > Virtual Machine... action to create our VM. Give your VM a name and then you can choose either Generation 1 or Generation 2. Assign as much memory as you want and then make sure you choose your newly created External Switch for the networking.

When you get to Installation Options choose Install an operating system from a bootable image file and select the Ubuntu ISO you downloaded earlier and then finish.

I'm aware we could have used the Hyper-V Quick Create... option, but again, I like to do all of this myself so I'm familiar with the setup.

Before we go any further make sure you open the Settings... of your new VM. If you choose to use Generation 2 untick Enable Secure Boot so we don't get any issues with the Ubuntu installation. Also check on the Memory and Processor settings, you may want to adjust what's assigned here based on the spec of your machine.

Now simply use the Connect... option and start up your VM. You should boot into the install screen

Choose the Install Ubuntu option and then from their go with the defaults or options appropriate for you. When it comes to the Updates and other software screen I choose a minimal installation as I'm not going to be using the VMs for anything other than running MicroK8s and MySQL.

After the install has finished you'll boot into the desktop.

It's likely the auto software updater will run, so let it do its thing and once that's done we're almost ready to install MicroK8s and MySQL, there are just a couple of extra steps that might make our life easier.

SSH Setup

I generally use Windows Terminal (which you can download from the Microsoft Store) to interact with my Linux VMs but that needs us to have SSH running on our VMs, and Ubuntu does not come with that pre-installed, so let's get that up and running.

Open a new terminal window with Ctrl+Alt+T and use the following commands to install openssh-server.

sudo apt update

sudo apt install openssh-server

That should be all that's needed and we can confirm that we can login by opening windows terminal on our host and using

ssh {username}@{VM name}

Desktop Resolution

It's likely we'll do everything via windows terminal from this point onwards, but if you want to use the Linux desktop for anything you'll find that by default you can't change the resolution to anything better than the fairly disappointing 1024x768.

To fix this we need to make a couple of tweaks. Open a terminal with Ctrl+Alt+T and then use the nano editor (or vim if you prefer) to edit the grub file.

sudo nano /etc/default/grub

Find the line that begins with GRUB_CMDLINE_LINUX_DEFAULT and add video=hyperv_fb:1920x1080 to the end. You can use whatever resolution is suitable for your monitor.

In addition to this you also want to edit the line that begins with GRUB_CMDLINE_LINUX and make sure that is the same.

Things should look a little like this.

Save and exit the editor, Ctrl+x then Y then Enter in nano.

and then run the following command.

sudo update-grub

Reboot your VM and you should find it starts back up with your new resolution set.

Next steps

At this point we should our Linux VM up and running.

Lets head over to part two of this guide where we'll go through the installation of MicroK8s and MySQL.

Pete