Forem: Tobias Urban

Write-Up: TryHackMe Web Fundamentals - ZTH: Obscure Web Vulns

Tobias Urban — Wed, 05 Jan 2022 08:42:25 +0000

Write-Up: TryHackMe Web Fundamentals - ZTH: Obscure Web Vulns

This is a walkthrough through the TryHackMe course on Obscure Web Vulnerabilities and aims to provide help for learners who get stuck on certain parts of the course.

Agenda

Section 1: SSTI
Section 2: CSRF
Section 3: JWT Algorithm vulnerability
Section 3.5: JWT header vulnerability
Section 4: XXE
Bonus Section: JWT Brute-Forcing

Section 1: SSTI

This walkthrough uses the manual approach to receive the flag.

When you connect to the demo application, you can check that the site is vulnerable for SSTI attacks by typing in {{3+3}}. If the site returns 6 instead of the literal {{3+3}}, you know that the page just interpreted your statement.

Now that we know we can inject statements, let's use this to read out the hidden flag. As specified in the challenge, the flag is stored under \flag, so all we have to do is read out the file with the command provided:

{{ ''.__class__.__mro__[2].__subclasses__()[40]('/flag').read() }}

or via command line injection:

{{config.__class__.__init__.__globals__['os'].popen('cat /flag').read()}}

P.S.: If you want to look up other injections, read through the provided repository. This project is using the Jinja2 templating engine.

Section 2: CSRF

This section does not have an actual challenge, but it requires you to get familiar with the xsrfprobe library. You could either install the library to your device and use xsrfprobe --help to find the right argument for generating a POC, or use the official documentation from the web. Hint: Find out which command would let you craft an actual, malicious request.

For the practical "challenge", you could build a simple website which authenticates users via GET request and stores the session in a cookie. Then try from another page to GET your existing page and observe in the network traffic that your second page can make authenticated requests on behalf of your user.

Section 3: JWT Algorithm vulnerability

This is probably the toughest section in this whole room, so don't get frustrated if it might takes several tries to get this done!

To recap the exploit, the basic idea is to take a token which was signed with using RS-SHA (and therefore a secret, hidden private key) and transform it into a token signed with HS-SHA, which uses a shared secret (public key). If the receiving application doesn't differentiate between RS-SHA and HS-SHA and if the server creating the tokens uses the same keys, this means we can forge any token and trick the system into believing it is legit.

As every token issued in the lab has a lifetime of only two minutes, you will have to complete all steps necessary in that timeframe. I would recommend preparing the "static" parts and then getting a new token and running through all "token-specific" steps.

First of all you will need to get and prepare the public key. You can download the public key under /public, and once you have downloaded it you can convert the key into HEX using this command (in an UNIX/Bash shell):

cat ./public.pem | xxd -p | tr -d "\\n"

Copy and paste the resulting string as we'll need that in a bit!

Next up let's look at the JWT token itself. If you take the token the lab prepared for you and parse it (for example by pasting it into jwt.io), you will see a header which specifies that RS-SHA256 was used for the signature. What you want to do is create a new token header which specifies that we want to use HS-SHA256. You can do this right in jwt.io by changing "RS-SHA256" to "HS-SHA256". Note that this part is the same for all tokens we will generate. So you can copy it out and we have the first third of our token ready!

Now you should have a really long string resembling your public key in HEX as well as a base64 encoded string resembling your token header. All that is left now is re-generating the signature and crafting the final JWT token. To create the new hash, the room already gives us the right command:

echo -n "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.your-token-body" | openssl dgst -sha256 -mac HMAC -macopt hexkey:your-key-in-hex

your-key-in-hex is the same for every token we will generate, so the only token-specific info we need to inject is the body of our current token (you don't need to change anything here, but every token will have a slightly different body). This should give you another string which already resembles our final hash.

In order to fit into our JWT token, we need that hash in a Base64 format though. Make sure you have python installed on your device and run the following command:

python -c "exec(\"import base64, binascii\nprint base64.urlsafe_b64encode(binascii.a2b_hex('hash-from-last-step')).replace('=','')\")"

Now we just put everything back together and we have our finalised token. It should look something like this:
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.payload-from-original-JWT.Hash-from-last-step

Section 3.5: JWT header vulnerability

To get started, sign in with any account and find the JWT token securing your session in the cookies (the cookie is called token). Copy that JWT token and paste it into jwt.io. In your payload, you will see an attribute "role" where you should set the value to "admin".

Right now though, the token would be invalid as the signature doesn't match the token anymore. Therefore we also have to change the header and specify that the token shouldn't have a signature all together. As jwt.io won't let you change headers of a token, just replace the current header with eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0. Decode this base64 string to make sure you understand what we are doing here.

Now put your token back together, which should look something like this: eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.new-body-with-role:admin.
(Please note the trailing .. You will need this in order to create a valid JWT token)

Copy that new token and replace your current "token" cookie with that new JWT token. If you now re-load the page you should be signed in as an admin!

Section 4: XXE

Make sure you have Burp Suite, OWASP Zap or a similar tool installed for this challenge as we will have to change the request structure in order to achieve the challenge.

When you open the form, submit it with some random values. Give each field a unique value so that you can easily see which value will be echoed in the pages output. This should reveal that the value of the email field is posted in the page, so this is the value we'll look at closer. Do another request and intercept it in your tool of choice so we can modify the payload.

In the challenge, we are asked how many users exist on the target server and what user has the specific ID "1000". Remember that the "/etc/passwd" keeps track of every single user in the system and also references the unique user ID, so this file is all we really need.

First off we need to define a new XML variable that we can display in the email field. In this challenge we are interested in the "/etc/passwd" file of the server. Define a new variable like this:

<!DOCTYPE data [ <!ELEMENT data ANY> 
<!ENTITY xxe SYSTEM "file:///etc/passwd" >]>

and call it like this:

<email>&xxe;</email>

Your full payload should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE data [ <!ELEMENT data ANY> 
<!ENTITY xxe SYSTEM "file:///etc/passwd" >]>
<root>
<name>aa</name>
<tel>aa</tel>
<email>&xxe;</email>
<password>aa</password>
</root>

Sending the request with this modfied payload should reveal the "passwd" file. Copy this into a text editor for easier reading, count the entries (each line is an entry) to get the number of users and search for the user with the ID "1000".

Bonus Section: JWT Brute-Forcing

This section is really straight forward. Install the jwt-cracker library on your device and then run jwt-cracker eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.it4Lj1WEPkrhRo9a2-XHMGtYburgHbdS5s7Iuc1YKOE abcdefghijklmnopqrstuvwxyz 4 to crack the actual password.

We know from the answer pattern that we're looking for a string with 4 characters, so make sure to specify this number as otherwise the tool will try 26^12 combinations instead of 26^4 combinations, which is a lot more! (You do the math, but not specifying the secret length will result in 20 billion times more operations, or put differently you'll spend quite some time staring at your screen).

Wrap-Up

I hope this write-down could help you understand some of the more complicated parts of this lab. If you have any feedback, questions or suggestions please comment it under this post!

Azure Active Directory Application Creator

Tobias Urban — Mon, 24 Aug 2020 10:13:36 +0000

Azure Active Directory Application Creator

Submission Category: DIY Deployments

Project:

urmade / AAD_Service-Principal_Action

GitHub Action to create a new Azure Active Directory Service Principal within your workflow.

GitHub Action to create new Application registrations in Azure Active Directory

This action enables you to automize the creation of Azure Active Directory applications in order to test your graph-powered or Single Sign-on enabled application.

How to use

In order to generate new applications automatically, you need an existing application that the tenant administrator has granted the Application.ReadWrite.All scope.

Mandatory parameters:

adminApplicationId: Client ID of an existing application with the Application.ReadWrite.All scope
adminApplicationSecret: Client secret of the same existing application with the Application.ReadWrite.All scope
tenantId: ID of the tenant in which the new application should be created

Optional parameters:

applicationName: Any string, is set as the name of the application and displayed to users on sign-in
redirectUrl: A list or URLs that should be registered as redirect URLs (Format: "URL,URL,URL")
logoutUrl: A single URL that should be registered as the logout URL
allowImplicitIdToken: Boolean indicator if the ID token acquisition…

View on GitHub

Additional Resources / Info

This tool aims to automate the rollout process (currently only in testing environments) for Azure Active Directory Service Principals. This enables a smoother DevOps experiences when developing in the Microsoft Office ecosystem where one may want to test their rollout experience in a more automated way.

Seamless SSO login for Microsoft Teams Tabs

Tobias Urban — Sun, 17 Nov 2019 13:17:44 +0000

Maximize security AND user experience

Until now, it was only possible to build authentication in Microsoft Teams tabs by this flow: You spawn a popup, the user manually authenticates, and then (best case) you stored some sort of session token in the users locale storage so that you wouldn't have to re-authenticate the user everytime they want to use your tab. In the end, you had to take care of building the popup flow, managing sessions for the users and deciding on in which frequency you want to re-authenticate the user to make sure they are still in a safe context.

How to build a secure and seamless experience

In a nutshell, you only need three things to implement a simple Single Sign On mechanism in Teams: An Azure Active Directory App registration, a Teams manifest and a HTML page that hosts less than 10 lines of JavaScript. You can find a complete implementation of this in this GitHub repository. Let's start with Azure Active Directory.

Configuring an App registration

When acquiring a token for your user, Teams uses the so-called OAuth2 on-behalf flow. This means: Users use your app and give your app the consent that it can access their data. Your app then opens to a second app (that is managed by Microsoft, for simplicity it is referenced as Microsoft App throughout this tutorial) and allows that app to access everything that your app can access. The Microsoft app then acquires a new token (in your name), and returns you only an id_token that is then exposed to the Teams client.

What are reasons for this way of doing it? For one, you don't have to care about implementing this whole flow yourself. The Microsoft App runs on a server that handles token acquisition, whilst you can sit back and wait for a token to return. Another nice side effect of this is that Microsoft automatically filters the answer it gets back from Azure Active Directory when it acquires the user token, and only gives you an id_token. This token already contains information about the user, but it cannot be used to make requests against any other service. Therefore the token is worthless upon the information it contains itself, minimizing the attack surface on the users client.
Secondly it is a more secure flow whilst enabling great flexibility. Usually when you want to receive any user tokens directly in the browser, you have to provide a client_id and a client_secret to the client. With these two values, basically everyone else could acquire tokens in your name as well. With the on-behalf flow, Teams only knows the client_id (and even that only through the manifest), and the Microsoft App can identify the id and acquire a token with its own, hidden id and secret.

But how do we actually register an App to work with Teams Single Sign On?

The process consists of basically three steps: Telling the Azure Active Directory that we want to have an application, giving it the permissions to read the users profile, and opening this application to the Microsoft App so that this app can access the users profile on our behalf.

The process is well-documented in this article, and you can find a step-by-step tutorial right here:

Head to https://portal.azure.com and log in with your credentials.
Search for Azure Active Directory and select it.
On the AAD Dashboard, click on App registrations in the left-hand navigation.
Click on New registration in the top navigation bar.
1. Give it a name that people understand. It will be eventually shown to them whilst using your app.
2. In Supported Account types go for Accounts in any organizational directory.
3. Leave the Redirect URI blank.
4. Click on Register.
In your newly created app, click on Expose an API.
On Application ID URI, click on Set. The API must have the following structure: api://yourendpoint/client_id. For example: api://mydomain.com/00000000-0000-0000-0000-00000000000000.
Next, click on Add a scope. Give it the scope name access_as_user, make it consentable by users and admins and give it a display name and description that is understandable for your users. Make sure its State is enabled.
Click on Add a client application.
Give the following client_ids access to your API: 5e3ce6c0-2b1f-4285-8d4b-75ee78787346 and 1fec8e78-bce4-4aaf-ab1b-5451cc387264.
On the left-hand navigation, click on API permissions.
Click on Add a permission, then Microsoft Graph, Delegated permissions and select email,offline_access,openid and profile. Click on Add permissions.

Here you go, you have everything in place with Azure Active Directory to receive Single-Sign-On tokens!

Creating a Teams app manifest

Basically all you have to do to enable SSO in your Teams tabs is to specify your API endpoint in the Teams manifest.

To register a Teams application with a basic tab, you can follow this article as a starting point. There is only one additional step to configure Single Sign On: In App Studio, go to Domains and Permissions and click on Set up under Web App single sign-on. Here you have to provide the client ID that you got from the AAD app registration as well as the resource URL you specified (in AAD, it was called Application ID URI, e.g. api://mydomain.com/00000000-0000-0000-0000-00000000000000). That is all you have to do! If you want to configure these properties right in the manifest.json, you can follow this documentation.

Getting your single sign-on token

Now that all setup steps are done, you can implement the actual single sign-on in the tab. This can be done in only 5 lines of code:

microsoftTeams.initialize();
var authTokenRequest = {
    successCallback: function (result) {console.log(result)},
    failureCallback: function (error) {console.log(error)}
};
microsoftTeams.authentication.getAuthToken(authTokenRequest);

result will contain the id_token with the users information. This is a JSON Web Token and can be decrypted using various JWT libraries. This token contains the name, mail-adress and AAD ID of the user. If you only need this information, you are now officially done!

Limitations and workarounds

This new mechanism is optimized around giving you a quick, seamless and verified information around who is using your application. In OAuth2 terms, it authenticates your user (if your app uses Azure Active Directory as the identity control plane). This leaves some use cases open where you still have to implement some workarounds.

Your app must use Azure Active Directory as its Identity provider

This flow only works with returing you the Azure Active Directory information about an user. If your app has its own Identity provider, you can't use this Single Sign On flow to authenticate your users. There of course is a way to still authenticate your users, but you are responsible for doing this and it usually involves a popup asking your users to log in manually. You can check out this article if you want to learn more about how to build such a mechanism.

You cannot make any further requests with the token

In the process, you get an id_token returned. This token is only used to provide information about the user and cannot the used for authorization, meaning giving you access to any other confidential data that you could ask for from Azure Active Directory. In fact, the SSO mechanism only implements half of the On-behalf-of OAuth2 flow. The Microsoft App gives you a token that contains data that was accessed in the name of your app, but you still have to convert it into a token that can make additional requests, as only then it is truly your app that is accessing that data.

Converting a token that another app has scheduled for you into a token that you got yourself is well documented in this article. A very simple implementation could look like this: When you get the token from the client, send it to your tab backend.

fetch("/storeToken?token=" + result)

In your backend (we're using TypeScript in this example) you can then trade the id_token for an access_token by following the OAuth2 specification. The most important value is the client_secret that verifies that you are the actual owner of this app who is allowed to make requests against Azure Active Directory / Microsoft Graph.

app.get("/storeToken", (req,res) => {

const idToken = req.query.token;

request("https://login.microsoftonline.com/common/oauth2/v2.0/token", {

    "method": "POST",

    "headers": {

        "Content-Type": "application/x-www-form-urlencoded"

    },

    "form": {

        "grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",

        "client_id": process.env.CLIENTID,

        "client_secret": process.env.CLIENTSECRET,

        "scope": "user.read",

        "requested_token_use": "on_behalf_of",

        "assertion": idToken

    }

}, (error,response,body) => {

    const access_token = JSON.parse(body)["access_token"];

    })

})

You only get a certain set of scopes

When using SSO with Teams tabs, the token issued will only contain five scopes: user.read, email, profile, openid and offline_access. Oftentimes, this is not enough when your tab wants to learn more about the context of the user or enables them to work with their Graph data. Before you can exchange this token for an access_token with further scopes in your backend, the user has to initially consent that you are allowed to use this data. For implementing this you can refer to this documentation.

Low code, ultimate security - Secure Azure with Managed Identities

Tobias Urban — Wed, 06 Nov 2019 22:34:39 +0000

Are your apps really secure?

TL;DR: Azure Managed Identities are awesome! Give your services an Identity, configure permissions between resources and let Azure handle all access management for you. How to implement this? Here you go.

Can you remember your last side project? For me, it is always a time of hype. I am a cloud enthusiast, so I always have plenty of stuff in my mind on what to do. Let's go with Kubernetes, but Serverless seems pretty cool too, and didn't they blog lately about that new feature in Web Apps that sounded fun? Maybe I just bring in all of them?
And while building a concept and coding the logic of your side project is a real joy, there comes this point where I always start wondering: "Wait, maybe that thing I built could be cool for others as well. Okay, then I definetly have to spend more time on the UI. And I have to make it more secure. I have to make all of it more secure. Why did I choose to bring in all these components?"

Securing the interaction between different resources in a product can be time-consuming. You have to connect all resources by hand, keep secrets up to date in your environment variables, and usually build dependencies in your code to some sort of security service or library (for the Azure context, let's take Key Vault for storing your secrets and certificates as an example). You start investing a lot of time in your security assets, and that's time you could spend on value-bringing features as well.

But luckily, Azure is here to help: With Managed Identities, you can assign an Azure Active Directory Identity to many of your Azure services and give them permissions to other resources, just as you would do with users. That way, you don't have to care about access management, and Azure routes and resolves all access requests internally for you.

What are these managed identities really?

If you have never worked with Azure Active Directory (called AAD in short) before, here's the summary: AAD is Microsofts Identity Management tool. That means, every user, all of his permissions and a lot of other objects in your organizations are listed there and with every sign-in to some resource. AAD knows exactly what you can or cannot do. Nearly all of Microsofts first-party as well as a lot of third-party tools use AAD for their identity management. For example, you can thank AAD for keeping your Office secure, and making sure only you see your OneDrive files.

And exactly this Azure Active Directory can be used to regulate service communication as well: Just give your service an identity (let's call it Alex) and specify all other resources that Alex can talk to.

Okay, how can Alex do that?

Bringing it to life: Assigning an identity

When we gave our service its identity, we created two things in AAD: An Application registration and a service principal. An application registration basically just tells AAD: "Hey, here is some service that wants to call some of your APIs" (This isn't the most accurate or wholesome description ever, but it serves my cause. If you want to go deeper, start here). With that application registration also comes a service principal object. This is a modified version of a basic user in AAD (like Alice and Bob), with the restriction that it only can talk to other Azure resources. This service principal is attached to our application registration, and it is linked to its assigned managed identity. If the managed identity is deleted, the linked service principal is automatically removed as well.

Assigning an identity can be done purely from the Azure web app. As we see in the next chapter, there are two different kinds of identities, but the process is very similar: Navigate to the resource you want to assign an identity to, choose the Identity tab in the resource navigation, and click on Status: On or Add (depending on which kind of identity you want to assign). The whole process is very well documented in this article.

Who am I? User-assigned vs. System-assigned identities

When it comes to managed identities, we can choose from two different creation methods.

System-assigned identities are related directly to one specific Azure resource. You can deploy it via two clicks in all Azure resources that support managed identities. If the resource is deleted, the identity is removed as well.

User-assigned identities on the other hand can be connected to multiple Azure resources. They are created like every other Azure resource and can be used to group multiple resources together to have identical rights and permissions. When of these resources are deleted, the user-assigned identity continues to exist.

Making it talk: Requesting access to other resources

At first, you have to configure which resources our newly created identity (called Alex like you might remember) has access to. If you ever assigned access rights to your co-workers on a specific Azure resource, this is a piece of cake for you. You just do the following:

Navigate to the resource that you want Alex to access
Click on "Access Control (IAM)"
Click on "Add", and then "Role assignment"
Give it whatever role is suitable for your plans, and enter Alex' service principal ID under "Select"
Hit save

And just like that you are all set! For screenshots and a more elaborate description of the process, go through this documentation.

Now we can go to phase two: Giving Alex an access token to actually call other services. Depending on what kind of Azure service Alex is, we have different ways to achieve this (and this article would blow up if we would cover all of them). But to clarify the concept, let's look at the flow for Virtual Machines.

Coming to life: Implementing Managed Identity calls on Virtual Machines

For those of you who are already familiar to OAuth2 login flows, this flow should be somewhat familiar (but with far less security measures included): After we set up our Service Principal from the Azure portal, we call a certain URL and get a JSON Web Token back with which we can call other services. In detail:

From your VM, call http://169.254.169.254/metadata/identity/oauth2/token. Provide these query parameters:

api-version: Specifies the internal API version to use for token acquiry. Must be 2018-02-01 or higher.
resource: The API that you want to call with the token acquired. Could be for example https://management.azure.com/ if you want to call the Azure API, or https://vault.azure.net/ for access to Key Vault.
(Optional) client_id: If you're using user-assigned identities, you must include this parameter with the client ID of your service principal.

Why don't we use any secrets or any other verification to get a token? Well, we are already on the Virtual Machine when we get the token. Therefore you can see this call as machine internal, same like if we would access the file system.

After you successfully requested your token, you will get back a JSON Web Token. This token can be used as an authorization to call other services (specified by resource). And now you're done! Just one single call from your server for all Azure services, instead of implementing huge and service-specific authorization SDKs!

To learn more about other resources and how to obtain an access token, check out this documentation.

Native security: Key Vault references

This is already pretty cool. But it get's even better. A common scenario for service-to-service communication in Azure is the request of secrets. You wouldn't want to store app passwords or certificates in plain text on your server, and leaving them hardcoded as an environment variable is still a huge administrative overhead (and all your App developers can see the secrets). So it is a good idea to outsource all secrets somewhere, and Azure Key Vault is exactly that place. It stores secrets and certificates encrypted by a physical TPM module (if you never heard about that, don't worry: It is just as secure as it sounds), with strong auditing and features like auto-rotation.

The only drawback: How can we access these secrets if they are not on our server? Ironically, we can use REST: Just call an URL, authorize with a secret and get your other secret back. Do you spot the loophole? We trade in all of our secrets for one master-secret, but in the end we still have to manage a secret.

For Azure App Services and through Managed Identities we have a very elegant way of fixing this. Just configure a Managed Identity for your App Service, give that Identity access to your Key Vault and use a Key Vault reference to access your secrets. What is a Key Vault reference? In the environment variables of your App Service (in Azure, they are called App settings), instead of configuring SECRET:pq289gni..., you can put in a reference. This looks like this: SECRET:@Microsoft.KeyVault({secretUri}). With the right permissions in place (provided through Managed identities) your App Service now automatically pulls the secret from Key Vault at startup, and the secret is never exposed anywhere in the whole process.

I hope you are now equipped with the knowledge to build super-robust apps on Azure. If you have any questions/feedback or some information is still missing, please provide some feedback!

Building a Microsoft Teams connector

Tobias Urban — Sat, 02 Nov 2019 20:19:16 +0000

What is a connector?

Teams connectors (or more specifically Office connectors) are inbound webhooks into Microsoft Teams. This means that a connector gives you an URL with which you can post messages in a specified channel at any time.
GitHub for example uses this mechanism to notify your team when a new pull request was accepted into a certain repository, Trello can notify the team about upcoming deadlines. Besides MS Teams, Connectors can also be used in Outlook to notify users via mail.

The basic functionality of a connector

A connector consists (from a developers perspective) of two parts: A configuration page and a backend. The configuration page is displayed directly in Teams and should be used to specify the content that is posted to the channel. So you could for example specify which task lists you want to monitor, about which type of messages you want to be notified, or how often you would like to receive notifications. The second part is the backend. Here you should store the webhook URL and send POST requests to that URL to send messages into the channel.

Configuring your connector for Teams

Besides the implementation itself you will also need a Teams App that the user can install in order to access your connector in the first place. And to create a Teams App, you should use a Teams App. More specifically, App Studio offers you the capabilities to just click through the App creation process and gives you a manifest.json file which contains your app configuration. Although you only need that manifest.json in the end (and you could write it from scratch if you're into that) it is always recommendable to use App Studio. It offers all configuration options available for the manifest and offers built-in error checking.

You will also need to register your connector in the Office 365 connector dashboard. Doing so gives you a connector ID which identifies your connector and gives your users more information about the organization that wants to post content into their channel. Besides some explanatory text for your connector, two settings are especially important: The configuration page (we will hear more about that in the next paragraph) and enabling actions on your card. If you do not enable actions, buttons that post a message to your app won't work (for example, you're posting a message into Teams that reminds you of an important task and you want to offer a button saying "Mark as completed"). When you successfully registered the connector, download the Teams manifest and start right away!

You only have to provide this information to register your connector

The configuration page is a HTML page that you can use to ask the user which data they want to get notified about in their channel. Specifically, you can ask for any information you need from the user, and based on this information you can select which data the channel just subscribed on and therefore which data you will send to the channel. Most of the following guide will be dedicated to writing a configuration page, so let's jump right in.

Developing your first connector

For your first connector, you will need only a configuration page. You can print the webhook URL directly to the configuration page, and then use tools like Postman to send messages to your specified channel. You can find the code for this step here.

To get our webhook URL, we have to register the connector within the channel. We need a Teams app, and this Teams app needs an URL to your configuration page (Note: localhost won't work, for developing you should use tools like ngrok). To interact with Teams from a frontend side, Teams offer the so-called Teams JavaScript SDK. In order to tell Teams if our configuration went successful, we will need the SDK. As we only want a webhook URL in the first step, we don't need any input elements in the HTML. We only need a container to display the webhook URL later on:
<span id="url"></span>.
Now we can start working with the Teams context. Before using the Teams SDK, you always have to initialize it first. You can do this by calling
microsoftTeams.initialize();.
Configuring a connector on the Teams side consists of four steps:

Providing additional information about your connector
Receiving the webhook
Telling Teams what to do when the user hits "Save"
Enabling the "Save" button

To give Teams more information about your connector, you should call microsoftTeams.settings.setSettings({...}) with the settings JSON object as the parameter. You need to provide these settings:

entityId: An unique ID of your connector in the channel. Is needed when you want to reference your connector from within Teams (e.g. you want to create a link to the connector configuration)
configName: The string that will be displayed to users when they look up their existing connector configurations in Teams
contentUrl: The URL which is called whenever the user wants to update the configuration

All together, the call could look like this:

microsoftTeams.settings.setSettings({
    entityId: "sampleConn",
    configName: "sampleConfig",
    contentUrl: "https://e6d84899.ngrok.io"
});

Next, we have to receive the webhook URL from Teams. This is actually a very familiar setting: We call microsoftTeams.settings.getSettings((settings) => {...}). In Teams the settings for your webhook are created as soon as you call setSettings(), so only then we can get the connector settings. getSettings() requires a callback that the settings are parsed to. For the moment we only want to print the webhook URL of the settings to the screen, so the call looks like this:

microsoftTeams.settings.getSettings(s => {
    document.getElementById("url").innerText = s.webhookUrl;
});

Though we now got everything we came for, the webhook isn't activated yet. To activate it, we have to save our configuration. This process consists of two steps: First, we specify what should happen when the user clicks on "Save". To do so, we call microsoftTeams.settings.registerOnSaveHandler((saveEvent) => {...}). In the actual handler, we need to at least call saveEvent.notifySuccess(); to tell Teams that our saving process is successfully completed. And second we have to make the "Save" button clickable by calling microsoftTeams.settings.setValidityState(true);. All together, our calls looks like this:

microsoftTeams.settings.registerOnSaveHandler((saveEvent) => {
    saveEvent.notifySuccess();
});
microsoftTeams.settings.setValidityState(true);

And here you go, your first connector is completed! Open Postman, copy your webhook URL into the URL bar, set your body to application/json and POST this message:

{
"text": "Hi I'm a connector test!"
}

Your first connector message now is available in your channel!

Securing the connector: Adding authentication

Now that you are able to play around with your first connector, you got the idea behind connectors. We can now start thinking about building a connector that actually could run in a production environment. From the configuration page, this means one thing above all: Security. We have to make absolutely sure that only authorized users are able to configure connectors. To do this, you should leverage Azure Active Directory (AAD) and log your users in before they are able to make any configurations. An implementation of this step can be found here.

On the HTML side, you have to insert a new button into your page. Teams will spawn a popup if you want to authenticate your current user, and popups that are not triggered by a direct user interaction are usually blocked. In the example the default text is hidden in another div for UI reasons. This leaves you with this code:

<button id="login" onclick="login()">Authenticate before configuring the connector!</button>
<div id="success" style="display: none;">
    Copy your webhook URL from here to POST messages in this channel: <span id="url"></span><br><br>
    Don't forget to click on "Save" to activate your connector.
</div>

Triggering a login in the frontend SDK is pretty intuitive. Just call microsoftTeams.authentication.authenticate({...}) and specify the login URL, the proportions of the popup as well as success / failure callbacks. The only thing you should keep in mind is that the login URL must be on the same URL on which your configuration page is hosted. So you can't directly redirect on example.secureLogin.com if your page runs on mysite.com, but you have to redirect to mysite.com/login first.

function login() {
        microsoftTeams.authentication.authenticate({
            url: window.location.origin + "/login",
            width: 600,
            height: 535,
            successCallback: function (result) {
                console.log(result);
                configure();
            },
            failureCallback: function (reason) {
                console.error(reason);
            }
        });
    }

When a user hits the /login endpoint, the example just redirects that user to the Azure Active Directory login page without any further checks. Creating a backend to support AAD logins is a (quite intuitive and fast) topic on its own, so to not bloat this article you can find instructions for that here. In the end, we get an access_token that contains some user information and enables you to call Microsoft services to get further information about the user. Though many tutorials directly get this token at the client side, this isn't a wise idea. Access tokens are valid for a hour, and whoever possesses such a token has access to sensitive user information. And as the client (more specifically, a browser) can have all kinds of vulnerabilities (like for example malicious add-ins) that could steal anything that goes over the wire, you shouldn't hand out such a sensitive token to your users.

But how do we pass anything to the configuration page anyways? Right now you have a popup where the user can log in, but this isn't your config page. The answer again lies in the Teams SDK: When the login process has finished, you have to redirect your user to a new HTML page that you host. On this page, you initialize the Teams SDK and call microsoftTeams.authentication.notifySuccess({...}) or microsoftTeams.authentication.notifyFailure() respective of if the login process succeeded. You could pass and access token as well as an id token to the client, but in the example implementation all this sensitive information is kept server-side. So you can send back just a placeholder indicating that everything succeeded (given that we won't need to persist the token anyways, you don't need to give some session ID to the client). The example uses ejs which is a very straightforward rendering engine for Node.js that allows to execute JavaScript whilst rendering HTML pages. The final code could look like this:

microsoftTeams.initialize();

        //notifySuccess() closes the popup window and passes the specified information to the configuration page
        //Usually you would pass the tokens in here, but as we don't want to expose user tokens to the client and we only need a proof that the user is who we claims (authentication), we leave these fields empty
        <% if(successfulAuth) { %>
        microsoftTeams.authentication.notifySuccess({
        idToken: "N/A",
        accessToken: "N/A",
        tokenType: "N/A",
        expiresIn: "N/A"
    })
    <% } else { %>   
        microsoftTeams.authentication.notifyFailure("User could not be verified");
    <% } %>

Calling this will close the popup and pass the information specified to the client. And just like that, you authenticated your user and made your app a lot safer!

Further steps to an awesome connector

If you now send the webhook URL to your server instead of just displaying it to your user, you took every step to create a solid base for your actual connector logic. Now the actual fun part starts: You have to implement some configuration options for the user to choose from when setting up the connector, store the webhook URL in your backend and trigger some event mechanisms whenever an user should be notified. For storing your connector, you should keep a few things in mind:

Next to the webhook URL, you should also keep the channel ID to eventually check (via Microsoft Graph) the members of the channel.
In your backend, you need a scalable and efficient process to trigger messages to the webhook URL. Utilize the Notification Event Pattern or the Observer Pattern and services like Azure Functions or Azure Event Grid.

Congratulations, you are now equipped to build an awesome connector and keep your Teams up-to-date about anything that happens in your application!

Serverless prototyping - A case study

Tobias Urban — Sat, 26 Oct 2019 23:02:20 +0000

Serverless computing is an exciting way of hosting your apps. It offers you unmatched scalability whilst making you only pay what you really used. In this article I will go through a case study on how an architecture for your serverless web app prototype could look like (on Microsoft Azure), some best practices and what it would cost you in the end.

What will be built?

So you decided to use serverless computing, but what now? The good thing first: You're not really forced in any programming language. The Azure functions runtime offers some preferred languages, but due to its open source character you could modify it to work with any language of your choice.

For your web app itself, you will most likely will have a front-end and a back-end. On the front-end site, you gain the most flexibility in terms of where to run your app (Mobile, Desktop, Web, …) with the classic HTML / CSS / JS stack. Plus, if you're fairly new to programming and don't have a deep knowledge in another specific language, those three are very easy to get into. For the backend, you can leverage the rich ecosystems of Node.js, .Net, Java or Python to get started quickly.

How to design your functions?

Now to the interesting part: How could an architecture for our app look like? How do we define a function in our system? Is the operation a+b worth its own function or should we split on logical entities (e.g. search all entries of a database table, bring them in the right format and then return them)? As always, the truth is somewhere in the middle.
From a hosting perspective, we have two metrics that determine how expensive a function will be for us: Execution time and memory consumption. So when splitting functions, it should always be our goal to overally minimize those two factors while keeping network traffic to a fair level without bringing a hell of communication latency into the system.
And note the "overally minimize": With every communication between two functions you bring in a sending (packaging) and a receiving (parsing) step, as well as securing operations, which all again is taking its time (which is neglectable at first but can become a factor if overdone extremely).

We can at least optimize for the networking and deployment bit by identifying functions that are highly dependent on each other. To understand why this is important, we have to peek "under the hood" of Azure Functions:
Whenever we deploy our Function App Code to Azure, that code is stored somewhere with a reference to an ever-active URL that marks our Function APP API endpoint. When someone hits that API, Azure starts looking up for an available VM where it could deploy your app. Once found, it takes your code, puts it onto that VM and starts your main process. Once this is done, all your functions in the app are available.
When traffic gets too high and the compute power on the current VM isn't enough to run your app, the code gets deployed to a second app and requests get distributed by a load balancer.
That means for us: All functions in a single function app live and scale together, and if you have a function that is called once a month and one that is called 50 times a second, both of them will have to be deployed and initialized together in each scaling step.
So to optimize scalability we can split our functions in "high-demand" functions and "just-sometimes" functions. Again, the additional time it takes Azure Functions to mount a few additional functions to a VM won't make the difference - but if we speak about dozens of functions, it may does. And on the other hand, functions which are often called together should be kept on the same deployment - this way you save a lot of networking overhead.

Services besides your functions

If you're developing a web app, you will most likely have a front-end that shows HTML and loads a lot of scripts and static files. As we learned already, Azure Functions always takes your whole codebase and shifts it onto an available VM. What happens now if a Gigabyte of images has to be moved? Exactly, the startup time of your Functions App gets tremendously slower.
So it would be recommendable to outsource as many bits and bytes as somehow possible - and this is where Azure Storage kicks in. With Azure (Blob) Storage you can store all your static files outside of your Functions App where they are always available and don't block scaling of your Web Application, while also being optimized for high-frequency delivery (think about CDNs and stuff like that, you have those options with Blob Storage but not if all your files are baked into your app). Plus, they are easily interchangeable and your app doesn't have to be redeployed every time you alter a picture.

And we have a second storage issue: How do we persist dynamic data (like user inputs)? We need some kind of database to store all of that, and databases need VMs, and VMs cost a lot of money per month, and we don't have money, so is it in the end inevitable anyways to go bankrupt and fail? Again, Azure Storage to the rescue. Azure Table Storage provides a very cheap "pseudo-SQL" database where you can store all kinds of data. I call it pseudo-SQL as it takes objects of key-value pairs as parameters and flattens them all in one big table where every distinct key that exists in at least one object gets a column. And it doesn't offer a rich query language, so you are mostly stuck with storing and retrieving data. As long as you create collections (the NoSQL equivalent of tables) with some sort of meaning, you're good to go (for the start).
And the best thing: The API for Table Storage and Cosmos DB (Azures high-performance database) are exactly the same, so if advanced queries and analytics and answers in the single-digit millisecond become a thing at your organization, having that is only a paramater away.

Are we done now with theory? Yes. You can finally start developing your own Azure Functions driven web app. I would recommend following structure for your project: You will need a function for every page you want to display to the customer (e.g. yourdomain/ and yourdomain/about) where you do the website rendering and send out the final file to the user (Which is also a great way to monitor your traffic and latency for each page). And you will of course need one function for every API endpoint that you provide (e.g. GET yourdomain/api/user, POST yourdomain/api/user, …). If you only have simple CRUD operations for your database, I would leave it with this setup.
If you however want to make more complex calculations that involve parallel computing (e.g. for every item in my array do something wildly complex), you could split up your functions API. If you have long-running orchestration functions that have to wait for other functions, familiarize yourself with durable functions.
Whenever these functions trigger another function, they go to sleep and wake up when all "sub-"functions are done executing. This way, you only pay for the functions which are actively doing stuff.

Architecture done, but why does my app get horribly slow sometimes?

Remember the whole "Spin up a new VM and deploy my code to it"-process? This is called startup of your Functions App, and its one of the main principles of serverless computing (only use your code whenever it is really needed).
Your functions transitions from being in a "cold" state (meaning it isn't deployed) to a "hot" state (meaning it is deployed and ready to serve requests immediatly). This process is constantly optimized to go faster by the Azure developers, but at the time writing this article it can still take up to ten seconds until your app is responding. After receiving a call and brining your App into a hot state, Azure Functions keeps your App hot for five minutes (By the time writing this article). Whenever it receives a new call, the clock starts ticking again for five minutes. After that, it clears your code from the VM and sends your App into cold state.
So theoretically you would need at least one user every five minutes to keep your App alive, which will probably not always be the case.

But there is an (unofficial) trick to simulate this effect: There are so-called time trigger functions which execute always after a specified time period. If you now add one of these time triggered functions to your project and set it off to execute every 4 Minutes 30 (It doesn't matter what the function does, just let it log a char or something like that), you have an activity in your app that keeps your app constantly in a hot state.
And as functions are optimized to be cheap for a huge amount of small compute activities, that roughly 9.000 calls per month that you "waste" actually cost you absolutely nothing (the first million calls in a function app per month are free). So this way you will always keep one VM occupied with your code and have your systems up and running at all times.

Do I have to ping an .azuresomething URL now?

As you have sure noticed Functions provides you with a standard URL from which you can call your service. This URL usually consists of the name of your Functions App.azurewebsites.net and is quite impractical if actual customers should engage with it (but at least its HTTPS-enabled by default, so you got that going for you).
If you want to release your Application, you most likely will want to bring in a custom domain from which your app should be reachable. And with Azure Functions you can do that in just a few clicks. Functions Apps are just basic Web Apps in Azure which can be configured in the exact same way. To access the Web App settings just click on "Platform Features" in the starting page of your Functions App and you're good to go. See this documentation on how to add your custom domain then.

And it get's even better. Usually when you host something with your own domain, you don't have a SSL certificate to enable HTTPS communication. This means that all of your users will most likely see a warning stating that your site is insecure and should not be trusted. With Azure Web Apps you have to possibility to configure a LetsEncrypt plugin that automatically creates and renews SSL certificates for your custom domain. To do this, just follow this article.

Great, but let's get to business. How much will that make?

(Before we start: All pricing detail reference the Western Europe Datacenter, are given in Euro and were taken in March 2019). Remember that we have three parts in our application: We have our Functions App for the logic and an Azure Storage with a Blob Storage and a Table Storage to host our website files and our data. So let's break down the single components:

Calculating cost for the functions apps

The Functions App is meant to support large concurrent tasks. It also offers a very generous free contingent of 1.000.000 API hits and the 400.000 GB-seconds per month.

Wait, what are GB-seconds? Besides of tracking how often your APIs are hit, Azure Functions also measures the RAM consumption of your Application. That means: For every second that your app is running, its RAM impact gets measured. If your App needs less than 1 GB of RAM, one GB-s will be charged. If it uses e.g. 1.3 GB of RAM in that second, two GB-s will be charged (keep that in mind when implementing compute-intensive functions).
But due to the free contingent, you could run nearly ten concurrent functions without a pause through the whole month (assuming that your RAM consumption stays below 1 GB) without paying a dime. And remember, Functions only charge for when your function is running, meaning idle time in hot state is completely free of charge.
Some technical details about this metric: Measurement is done in 128 MB intervals, and one function in the standard tier can take up a maximum of 1.536 MB of RAM.

Although our compute cost is the one kicking in earlier, we still have a second metric. So let's look at the second metric, API hits.
As you might remember we are using time trigger functions every 4 minutes 30 seconds to keep our app alive. That adds up to roughly 9.000 calls a month, leaving 991.000 free calls per month. If we're assuming a typical user of your app checks in twice a day every day of the week and uses during one session 50 function calls (page loading and interactions with the API like creating and deleting elements), you could support nearly 354 concurrent users within the free contingent.

But what happens when you exceed your free budget? Is that when the price trap finally snatches? Not really.

Let's calculate with 500 active users for your app. Using the scenario described above, these users would trigger 1.400.000 function calls each month, so we would pay 0.17€ for an additional million function executions. We will also have to pay for 1.000.000 additional GB-s, which costs us 14€. For 500 active users in our system, this leaves us with 14,17€ of hosting cost. This is a cost of not even 3 cents per user in your system!

As we scale up, the cost per user stays roughly the same: Supporting 1.000 users in our scenario would cost 34,11€ which is 3,4 cents per user, 10.000 users would cost us 390,97€ which is roughly 4 cents per user, and 100.000 users would be 3.961,55€ with again is roughly 4 cents per user.

Calculating storage cost

But don't forget: We are not yet done. There is still the storage. Remember, we had two parts in there: A blob storage to store static files and a table storage to store persistent data. Both storages have the same pricing model. You are paying once for the amount of data you've stored (in GB) and second for the amount of CRUD transactions you did on that data (in 10.000 operations).
For the Blob storage, you pay 0.0166€ per GB per month (in a "hot", meaning very fast access level), as well as 0.004€ per 10.000 read operations and 0.0456€ per 10.000 write operations.
Let's say we're having 4 sites, per site 15 static assets that get loaded, that our whole frontend asset base has a size of 100 MB and that we update four times a month. An average user visits all 4 sites per session, and has the option to upload a profile picture which he of course does. Assume that an unoptimized 4k profile pic has 3.5 MB of data. The profile pic is loaded with every change of the users site and gets re-set once a month.
With these assumptions and the fact that we support 500 users that browse our site twice a day, we would have: 0.3€ for hosting roughly 2 GB of assets, 0.07€ for loading the pages and 0.02€ for re-deploying our front-end and storing the user profile pictures, adding up to 0.39€ for file storage.

Azure table storage has the same metrics, yet different prices. Per GB of data you pay 0,0591€ (in a locally redundant storage, which should be enough for our case) as well as 0.000304€ per 10.000 transactions, regardless of their type. Let's say we host a user database where every entry has 0.5 kb of data (thats roughly 10 columns of 15-char-strings) and where an entry is changed once a month. Furthermore we log everything our users do with a size of 0.05 kb of every log entry. We furthermore have four databases where a user on average stores a hundred elements at 1 kb of data each and which a user accesses for all his entries once per session (imagine a To-Do list of something like that). Let's add that up with our 500 users: We'll pay 0.18€ for 3 GB of data storage as well as 1.97€ for over 64.3 million transactions per month, adding up to 2.15€ in database costs.

Let's put that all together: We've built a killer app and managed to convert 500 users to use it on a daily basis. Per month we're paying 14.17€ for hosting our servers, 2.15€ for hosting and maintaining our data and 0.39€ for storing our static files. This adds up for monthly costs of 16.71€ per month on IT infrastructure costs, leading to costs of 0.034€ per user in our system. So for the rough price of a Netflix family subscription you can bring value to 500 users through our Functions app. And the best thing is as our app is built for scalability, growth in usage and your costs would scale linear, making it real easy to calculate your own pricing.

That is awesome!

I know, I was thinking just the same when deciding to write this article. But please keep in mind: This approach is meant for when you are starting with your product. You will always have limitations that sooner or later will become a bottleneck in your IT organization and if you don't shift your hosting methods soon enough you'll end up building a lot of technical debt. Here a the most striking limitations you have:

You have a RAM limitation of 1.5 GB per function in the standard plan, meaning RAM-intensive processes like rendering, complex image-refactoring or AI are hard to implement and in every case very slow in execution.
Your Functions App is very good at scaling, making you vulnerable for DDOS attacks. You can mitigate that risk by putting your App behind an (Azure) Firewall (and Azure has an eye out for you as well by default).
Your attack surface increases significantly, as you have to protect dozens or hundreds of small functions. You have to validate input, output, access rights and many more in each function you are writing.
Though Azure Functions scale really well, they don't scale infinitely. Right now, a maximum of 10 VMs can be occupied by your Functions App in the Standard Plan, and these VMs only have a fixed amount of RAM to offer, meaning there exists a hard cap on how much workload a Functions App can handle.

As already said, these are all risks that could be partially mitigated or worked around, but only at the cost of really tweaking your code base into something that isn't meant for doing that.
Although all the downsides and grumpiness at the end of the article, I hope this article gives you a good introduction how to quickly set up a scalable, reliable and cheap service that you can use to kick-start your product.

Why serverless is awesome for prototyping

Tobias Urban — Sat, 19 Oct 2019 09:53:24 +0000

With the rise of Cloud Computing new ways of hosting emerged, putting on-demand compute power instead of hardware limitations in the center of every organizations IT department. What started with Virtual Machines that could be spun up and shut down in minutes to optimize costs has evolved to many new technologies that are now de-facto standards in an ever growing number of organizations. Applications became "cloud native" and organizations were "cloud born" - and if you really managed to dodge these buzzwords until now, I recommend you to read this article to catch up with what modern IT Infrastructure is all about.

Breaking systems down with Microservices

These innovations all had similar goals in mind: Make your infrastructure more flexible and at the same time robust whilst minimizing your costs (remember, in the new world it's all about variable software compute power instead of fixed-priced hardware). Technologies like Docker and Kubernetes arised which made splitting up your application in small building blocks a real thing (this is called Micro-Services, another Buzzword that you can read more about here).

The basic idea behind this: Instead of having one app where every line of code references any other line of code (this is called high cohesion), build multiple modules that have well-defined interfaces that other modules can use (which leads to so-called high coupling). This way, you are more flexible in the way you build your application (you may for example want to build your app front-end in Node and the high-performance calculation in Go) and in the way you scale your app (same example, the frontend may just needs to forward user input so 2GB RAM on one or two machines could be enough, whereas your calculation engine may needs 32GB of RAM and on high traffic should scale to 8 machines).

Going even smaller with Nanoservices

And you can spin that idea even further: After Micro-Services followed Nano-Services, with the idea that every deployment you have in your architecture should only do exactly one single thing and is seperated into its own code package. What sounds like the ultimative mess for every IT Admin (and you're not wrong, it has its drawbacks) enables completely new levels of scalability and development flexibility.

This is where so-called serverless computing shines: If all your workload is split in tiny packages, the compute power becomes nearly irrelevant (as a single function usually doesn't need a lot of compute power) and flexibility becomes the top priority. And this what for example Azure Functions or AWS Lambda promise: You write single distinct functions that can be accessed by other services via API and upload the code into the cloud, and the cloud provider deploys your code "just-in-time" when it is called from another service. You as a developer only pay for the time that your code execution took and for how many times your function was called (usually priced at a few cent per a million calls).

This leaves us with two exciting things: We have a cheap service to host our application, and we have a lot of scalability that only charges us for how frequently our app is used. This is great for Start-Ups or prototypes where you don't want to overspend on infrastructure, where you usually don't have highly complex logic in your application and where you yet don't know how heavily your app will be utilized.

Serverless on everything ... right?

Of couse, not everything is perfect in the serverless world. And it will probably not solve all your problems you ever had with hardware management. The most striking reason against serverless computing comes from its core development ideom: By having thousands of standalone functions in your architecture, you create a huge latency overhead just for routing traffic through your systems. Even if you give all your apps into one datacenter, you still have to route the traffic within the datacenter, and if we think about visiting 20-50 stops before we successfully finished one request, this latency becomes significant for your response time.

Most cloud providers also usually offer only standardized VMs on which you can run your functions. For you that means that you have a cap on how much resources one function can consume. Although this shouldn't be an issue for most of your logic, you will have to search for alternatives when it comes to compute-heavy jobs like image transformation or AI calculations.

One additional thing you have to consider when going serverless: Your attack surface multiplies. A lot. You have to secure every single function, and every function on its own is responsible for validating access control, inputs and outputs. Most providers offer easy and compute-saving methods to handle the authentication part, but still you have to take care of securing every single function in your architecture.

And as hard as you try, you will almost never end up in a completely serverless way. Think about the following scenario: One of your functions sends out notifications to all of your users mobile devices. This triggers around a million function calls as your app already got quite popular. Absolutely no problem, serverless will handle everything for you and a few seconds later all is done (for probably just a few cents for all messages together). But you have one additional job for them: For logging purposes, your function writes a statement in a SQL database. If you never worked with SQL DBs: Shooting a million concurrent requests at it is a really bad idea.
So you will have to keep the big picture in mind, and design your architecture around the "weakest link" in your overall architecture. The best individual scalability won't help you a thing if you have one bottleneck that keeps the overall process slow. As a note before you ditch logging forever: there are already services by large cloud providers that even enable serverless storage databases which you can penetrate as much as you want.

So when do I use serverless?

Serverless functions have a lot of great use cases. If you have an app idea and you want to quickly test it and get started fast, serverless is a great way to achieve this. You don't have any fixed hardware cost, your app can scale no matter how many people use your app and you are forced into a dependency injection programming pattern which in the worst case leaves you with a greatly structured monolith if you decide to switch to traditional development. And the most striking concept still is: You can basically develop your service for free. Most cloud providers offer the first million calls to a function per month for free, which is more than enough to get you started. From here, you can start promoting your app and acquiring users. And as you do, your bills scale proportionally to your users, giving you the chance to scale with your users demand (and probably your income from new and paying users).

Serverless is also a go-to choice for all kinds of proxies, helper functions and small handlers. If you want to pipe a request from one service to another (e.g. from a queue to a database), serverless got you covered. And you could even automate your IT infrastructure with functions: Just write the REST calls to start / stop / administrate your services as a serverless function, and define protocols in which situation which operations should be applied. And here you go, a fully automated IT infrastructure right at your fingertips.

Have you ever worked with serverless functions? What did you like, what were your pitfalls?

The (hands-on) beginners guide to Azure Active Directory

Tobias Urban — Thu, 17 Oct 2019 17:44:07 +0000

This article aims to give you a reference implementation of how you can log in your users using their existing Azure Active Directory accounts. Are you curious what Azure Active Directory (AAD) is in the first place? Then check out this article that walks you through the theoretical foundations of AAD.

You can find the final code of this tutorial here.

What we're building

Azure Active Directory is built on industry-standard identity specifications and supports Open ID Connect for modern authentication and authorization. This tutorial will walk you through the OAuth 2.0 authorization code flow, which consists of two basics steps:

First, you have to redirect your user to a Microsoft hosted login page. Microsoft takes care of receiving, handling and validating user credentials, so you don't have to care about any of that.
In the second step, Azure Active Directory will send an authorization code to our web server. We take this code and trade it for a JWT token, which then can be used to authorize calls to other Microsoft services like Microsoft Graph, AAD and more.

Getting started

As a prerequisite for this tutorial we will need some kind of server environment to run our code, in this case we chose Node.js. The first thing to do is to set up the server:

const express = require("express"); const app = express(); const request = require("request"); require("dotenv").config(); //Here goes our code app.listen(8080);

In our first step we have to log in our user and receive a code string that we can trade in for a Bearer token. This tutorial assumes that you have already registered an app in Azure Active Directory, a manual for that can be found here. To get the code we have to redirect our user to the Microsoft login page (where they then log in with their credentials).

app.get("/login", (req, res) => { res.redirect( "https://login.microsoftonline.com/" + process.env.TENANT_ID + "/oauth2/v2.0/authorize?" + "client_id=" + process.env.CLIENT_ID + "&response_type=code" + "&redirect_uri=" + process.env.BASE_URL + process.env.REDIRECT_URL + "&response_mode=query" + "&state= " + "&scope=" + process.env.SCOPE ); })

There are a lot of parameters in this URL, so let's walk through them one by one:

The base url: We need to call https://login.microsoftonline.com/*tenant*/oauth2/v2.0/authorize to enter our user data. The tenant can be either only our own tenant (e.g. myTenant.onmicrosoft.com) or common, which means every account that is part of some Active Directory tenant (e.g. thatOtherTenant.onmicrosoft.com) can log into our app. We further specify that we want to use version 2.0 of the REST API (you can find the differences between v1.0 and v2.0 here).
client_id: The id of your app that is registered in your AAD tenant
response_type: Specifies what Active Directory will return after a successful login action. For the OAuth 2.0 flow we always use code to receive the code we can trade for a Bearer token, in the OpenIdConnect flow we would have to use the value id_token.
redirect_uri: The url to which AAD will redirect the user when they successfully logged in. This url has to be specified in the app registration and your parameter must match this specified url exactly (including the path, so if you specify www.myapp.com as a redirect url in Active Directory and www.myapp.com/callback in your query parameters, the authentication will fail).
response_mode: This parameter determines how our access code will be sent back to our app. It can have the values query, form_post and fragment. In our case where we actually redirect the user we can use the query parameter, if we would for example open a pop-up for the user we could also use the form_post to post the code to our app.
state: Here you can enter any string you wish. This can be used as a challenge for security reasons or if you want to store some information the user gives to you at the beginning of the authentication flow and that should be given back at the end of the flow.
scope: The scopes are the permissions your users will have when they want to call any Microsoft services with the bearer token they receive. You can read more about scopes here.

Furthermore there are optional parameters that you can use (if applicable):

prompt: Here you can specify what the user will see when they try to log into your app. By default, Active Directory will sign your users in, store a cookie on their device and with every further authorization, it will log the user in silently without showing an additional prompt. With the values login, consent and none you can specify that the user either always sees the login form, that they always have to give consent to the app to use their data before using it or that they never see any form.
login_hint: If you already know the email address or user name that your user will take to log in, you can pre-fill this input field in the login mask with the login_hint parameter.
domain_hint: As already stated the Azure AD v2.0 endpoint supports login from consumers (private accounts) and organizations likewise. With this parameter you can specify if only organizations or only consumers can log into your app by giving it the value consumers or organizations.

When our user now logs into the application successfully, our redirect_uri will be called in the format redirect_uri?code=… . We can now listen to that path from our server and directly use the code to trade in for a token. Therefore we have to use a POST call to Azure AD, and we have to implement functionality on what to do with the token when we received it.

app.get(process.env.REDIRECT_URL, (req, res) => { const authCode = req.query.code; if (!authCode) { res.status(500).send("There was no authorization code provided in the query. No Bearer token can be requested"); return; } const options = { method: "POST", url: "https://login.microsoftonline.com/" + process.env.TENANT_ID + "/oauth2/v2.0/token", form: { grant_type: "authorization_code", code: authCode, client_id: process.env.CLIENT_ID, client_secret: process.env.CLIENT_SECRET, redirect_uri: process.env.BASE_URL + process.env.REDIRECT_URL } }; request(options, function (error, response, body) { if (error) throw new Error(error); try { const json = JSON.parse(body); if (json.error) res.status(500).send("Error occured: " + json.error + "\n" + json.error_description); else { res.send(json.access_token); } } catch (e) { res.status(500).send("The token acquirement did not return a JSON. Instead: \n" + body); } }); });

The POST call again has a few mandatory parameters that we have to provide to get the Bearer token.

url: Similar to the first call we can use either our tenant or the common endpoint and we can specify to use v1.0 or v2.0. The url has to match the url from the first call.
grant_type: Used to specify which kind of grant should be given back, must be authorization_code to get a Bearer token. You can look up the different grants here.
code: Our access code we got from our previous call.
client_id: The ID of our app registered in Azure Active Directory.
client_secret: A secret given out by Azure Active Directory used to prove that you are the developer of the app.
redirect_uri: Usually just the current url, as this is the redirect_uri you have specified in your last call and in AAD.

After we executed or POST call, we get an answer from Active Directory containing a JSON object that has the Bearer token in its parameters. If an error occurred the JSON object has an error attribute where you will receive an error message.

Now you have everything to authenticate your users and start working with their Microsoft profiles to build apps based on the Microsoft ecosystem. For example, if you gave your app the scope user.read, you can now do a GET request to Microsoft graph to get the users profile information.

app.get("/me", (req,res) => { const options = { method: "GET", url: "https://graph.microsoft.com/v1.0/me/", header: { Authorization: "Bearer *your-token*" } } request(options, function (error, response, body) { res.send(body); }) })

You can find the whole code (documented) here.

The (theoretical) beginners guide to Azure Active Directory

Tobias Urban — Thu, 17 Oct 2019 17:42:31 +0000

When I started to develop my first small applications, I usually wasn’t concerned too much about security and about how to keep user data safe from exposure. But the deeper I digged into developing apps, the more I figured out how hard it was to design a truly secure way to store sensitive personal data. So I looked for third-party software that could solve this problem for me.

If you’re searching for identity management providers, one of the first platforms you will find is Active Directory (AD), a tool developed by Microsoft that was first launched in 1999 and since then evolved to the leading Identity and Access Management (IAM) software for enterprises. But although this makes Active Directory a very important skill to cover for software engineers, it also means that the software has to cover a lot of different use cases, making it seem very complex at the start. Therefore I hope to make your start with Active Directory a joy by sharing a walk-through tutorial of the login flow (written in Node.js) and by explaining the underlying concepts of Active directory in the process.

TL;DR: Is this tutorial suitable for me? Ask yourself these questions:

Do I want to understand how an IAM software works?
Do I want to explore different Azure Active Directory versions and how they differ?
Do I want to build a login mechanism with Azure Active Directory?
Do I want to learn more about OAuth2?
Do I want to learn how to secure my users sensitive data?

If you can answer any of those questions with “Yes”, this is your tutorial. We will cover the following topics:

What and why is Active Directory
The developers opportunity
Active Directory for everyone with AAD B2C
OAuth2 in a nutshell
Where to learn more

This tutorial is seperated in two articles: A theoretical introduction to Azure Active Directory (You are here) and a hands-on walk-through. You can find an implementation of how to log your user in with Azure Active Directory in this GitHub repository.

What and why is Active Directory

So before we start working with Active Directory we should understand what exactly AD actually is. Basically, AD stores all kind of entity information in an organization. That could be for example data about users, groups, devices, organizations or permissions between those entities. So it is far more than just a pure user management platform, although in this tutorial we will only focus on the user management features.
Historically, Active Directory was an On-Premise solution, meaning that every company using this technology had it installed in their own datacenters, completely isolated from all other Active Directories out there. After some time solutions were provided to connect two specific Active Directory tenants with each other, so that the employees from Company A could for example get access to devices of Company B and vice versa.

With the rise of cloud and Microsoft Azure specifically a new form of AD was provided: Azure Active Directory. Here, all Active Directories are connected with each other, meaning that someone from Company A could for example access an application developed internally at Company B if Company B allows that option in their application (more on how to handle that in the Coding section). This eliminates a lot of configuration overhead and makes it far easier to develop B2B applications as the app developed in your AD tenant can now be accessed by either individual users of another company or can register in the AD tenant of your customer (if you’re interested in that, you can dig deeper into AD scopes and delegated vs app permissions).

Active Directory tenants that are hosted on-premise, meaning not in Azure, are not accessible by Azure by default but can also connect to the Azure Active Directory via Azure AD Connect to replicate their user data to Azure Active Directory.

The developers opportunity

Coming from its long history and its deep integration with the Microsoft ecosystem, Active Directory was the de-facto standard for identity management in companies. And even though on-premise identity management isn't on its prime anymore, a lot of companies stayed with what they know and migrated directly to Azure Active Directory. There are also a lot of reasons to do so even nowadays: Office 365, Dynamics 365 and Azure (the "three clouds" of Microsoft) all work with Azure Active Directory. So whenever you want to use the cloud-hosted Office, you will need an Azure Active Directory account. Also every user with a Microsoft account (for example someone who uses Outlook or has a XBOX account) is also using Azure Active Directory under the hood.

This makes AAD a huge deal for developers. Especially if you develop an application for the enterprise sector, you often have to argue over how to access data and store user information. By integrating your application in Azure Active Directory, you don't have to store any personal user data (e.g. names), your users have a seamless single-sign-on experience and your customers admins still have control over which users and which data your app can reach out to. This spares you a lot of user management development efforts and makes user adoption a lot easier.

Active Directory for everyone with AAD B2C

Active Directory in the first place was a B2B product, designed for companies to handle their internal resources. As more and more companies developed complex B2C apps a new branch of Active Directory was released: Azure Active Directory B2C. While this version was downsized in terms of micro-management of users as well as in its capability to handle resources like devices, rooms and organizations, it came with a lot more features to handle a massive, anonymous amount of users, like self-registration into the active directory via e.g. Facebook, Google, Mail or LinkedIn and advanced features to store custom user metadata.

Furthermore a new version of the Active Directory login API was released, making it possible to authenticate personal Microsoft Accounts (e.g. outlook or msn email-adresses) into Active Directory applications. Therefore there currently are two options to develop B2C apps with Active Directory: You could use the traditional Azure Active Directory and allow users to log in from any domain (including the public Microsoft Active Directory that holds all @live.com, @outlook.com,... users). Or you use Azure Active Directory B2C to build your own Active Directory instance which holds all the users of you app. This approach holds a lot more benefits like expanding AD with custom metadata, and making it possible for people without a Microsoft account to use your app.

OAuth2 in a nutshell

Now that we know a lot more about the "What" of Active Directory, we want to look into the "How". Specifically, how can we authenticate into Azure Active Directory (AAD).

AAD uses the OpenID Connect specification to handle the sign-in process of its users. OpenID Connect is an extension to OAuth 2.0, which was built to standardize identity verification throughout all systems. OAuth 2.0 only specifies the Authorization of users, meaning: How can one user proof that he is allowed to do what he wants to do? Although this is a crucial step in securing Web Apps from malicious access, it still doesn't answer the question: Who is that person that wants to access my app? Therefore, OpenID Connect was written and took care of the Authentication of users. As OpenID Connect (also called OIDC in short) builds on OAuth2.0, every system that implements OIDC always also implements OAuth2.0.

The good thing is: We don't have to implement the Authentication part on ourselves when we start with AAD. Although it is a good idea to think about Authentication more deeply in production apps, the core logic of signing users in and looking up who they are is handled by Azure Active Directory for us. We only have to care about Authorization of our users, so getting to do something with their data.

OAuth2.0 offers a variety of possibilities to authorize your users. The most intuitive one (and the most common for Web Apps) is the authorization code flow. This flow has two steps: In step 1, you redirect your user to a Microsoft hosted login page, where users can log in with their username/e-mail and password. When the login was successful, we enter step 2: Azure Active Directory sends an authorization code to your application. You now have a proof that your user is signed in, and you can trade this authorization code for a so-called JSON Web Token (JWT). And that JWT can be then used to query services like the Microsoft Graph or Azure Active Directory to receive and manipulate data on behalf of the user that signed in.

Where to learn more?

If you want to go into the technical how of this concept, you can check out this article that walks you through an implementation of the authorization code flow.

If you already have an existing AAD integration, there is another exiting offer you can use. Microsoft offers a dedicated store just for AAD integrated apps where you can promote your app completely free of charge. To get started, take a look at the official documentation.

Getting started with WebAuthn - The basic flow

Tobias Urban — Sat, 12 Oct 2019 08:08:49 +0000

New to WebAuthn?

If you have never heard of the WebAuthentication standard before, let me tell you something: It's awesome! I've written a blog article that explains what WebAuthn is and why you should definitely use it. I recommend you to check that out first and then come back to the technical parts.

What will be covered in this article?

This article is meant as a first introduction into how to implement WebAuthn yourself. It won't cover each required methods step by step (although an article about that is currently coming up), but it will walk you through all necessary steps to implement the specification. Here's what you can expect:

What is the basic (developers) idea of WebAuthn
Registering a new user
1. Step 1: The Client
2. Step 2: The Server
Validating a login request
1. Step 1: The Client
2. Step 2: The Server

What is the basic (developers) idea of WebAuthn?

From a web developers view, WebAuthn is actually quite straight-forward and only offers two functionalities: Signing up a new user and logging in an existing user. For both of these functions, it offers a method under navigator.credentials: navigator.credentials.create() and navigator.credentials.get(). The credentials API also offers the methods navigator.credentials.preventSilentAccess() which toggles the auto sign-on capabilities to your application and navigator.credentials.store() which is meant to persist any credential (e.g. username/password) to the browser. Both of these APIs are not part of the WebAuthn standard.

The WebAuthn specification is not yet implemented in all browers (by the time this article got published). Check availability to learn more.

Implementing WebAuthn always requires a client (e.g. a browser) and a server (or, in specification terms, Relying Party). On the client-side, you have to provide some options (we will look into these later) to call create() and get(), which will then provide you a PublicKeyCredential. This credential contains different data depending on if you have created a new credential or requested an existing credential. The credential is delivered in a secure context by most clients, meaning it is only handed out over HTTPS and its attributes cannot be directly sent to a server. That wouldn't be a good idea anyways, as the credential object contains a wild mixture of strings and byte arrays.

Registering a new user

Step 1: Client-side signup

So let's get into the actual doing. Registering a new user starts on the client. Calling navigator.credentials.create() will push a dialog to the user prompting him to choose an authentication method and using this method to verify themself. But before doing this, we have to configure the so-called PublicKeyCredentialCreationOptions.

The CreationOptions consist of multiple pieces of information. These are:

The ID of your app (namingly the URL on which your app is reachable)
The ID, name and username of the current user (The latter two will be shown to the user when he signs in)
A server-side challenge that you can use to ensure this request is actually meant for your app
A list of allowed public key creation options (depending on which algorithms you allow for public key creation some providers will / won't work)
A timeout value in milliseconds
A list of credentials that are not allowed to be created again (basically a security measure to prohibit users creating multiple accounts on the same device)
Additional data which requirements an authenticator (e.g. Windows Hello, Touch ID) has to fulfill to be eligible to create a credential
An indication if you wish to also get data about the authenticator and not only the new user credential
Extensions: Basically every key-value pair you want to include, can be used for information encoding of all sorts

After calling navigator.credentials.create() with these options set, you will receive an Authenticator Attestation Response which also implements the Credential interface (Yep, the specification loves long object names). So basically you get a credential with a credential ID and a type (this field indicates in the W3C specs the opertation you did, namingly webauthn.create or webauthn.get), some client data (like the url at which the credential was created, and your original challenge), and an attestation object. This is a byte sequence, so literally just a list of zeros and ones. This byte array contains all credential-relevant data: The public key, (again) the credential ID and some other, cryptographically relevant information.

Before you can send this credential object to your server for further operations, you have to encode the information contained in the credential into a string that you can then decode again at your server side. You should send all available data in the credential object to your server, as every attribute has some necessity for the validation of the credentials later on.

Step 2: Server-side signup

Once you've sent the data to your server, the specification then puts a lot of verification steps in place to make sure you are dealing with an actually legit request. Your first job is to verify the source of the request is actually legit.

To do so, we got the clientDataJSON that contains all source-relevant information. As this is only a stringified JSON by the time your server receives it, you just have to parse it and are ready to go.

Its type field indicates for which operation this credential was actually issued, so you have to make sure that field is set to "webauthn.create".
You then must verify that your server has actually issued the challenge that is included in the clientData. The challenge is a random string that your server issues whenever some user (automatically) creates new CreationOptions to sign up for your service.
Make sure that the URL that scheduled this credential is actually the URL you would expect to do this. E.g. if your app runs on "https://www.awesomeapp.com" and the origin of the clientData is "http://phishy.scamsite.to", you would probably want to cancel the request.
Some clients will also send you a tokenBinding. This attribute contains information about the TLS connection over which you got your credential, and can be used to validate a secure transfer from the client to your server.

After you are done with this, you can be pretty sure the request was issued by a legit source. So we can move on to the actually interesting part - the validation of the credential we just received. To do this, we have to decode first the attestation and then the authenticatorData.

The attestation is a CBOR encoded byte array that tells you all you need to know about how the credential was issued as well as the credential itself. This is also the part where it starts getting tricky. Every authenticator can specify their own attestation, and with that the data contained as well as the validation process are completely different. By now, this led to six different Attestation Statement Formats - they vary in their data content, their verification methods and even in the format in which the verification relevant data is issued.

We also get the authenticatorData. Here we find the credential itself, information about the circumstances under which the credential was created, and an encrypted version of the Relying Party ID that we specified at creation time in the client.

To get things started, we first decrypt the attestation and the included authenticatorData. The attestation The authenticatorData can be parsed byte by byte as the specification gives a clear format on what byte has which information.
We then verify that the relying party that issued this credential is really us. To do so, we encrypt our relying party ID with the SHA256 algorithm and compare it to the rpIdHash - if the two are identical, everything is okay.
We then take care of the flags - binary values which describe the checks that were done before the credential was issued. We have one flag for userPresent. This indicates if the presence of the user was verified by showing at least one popup that the user actively clicked before the credential was created. Another flag stands for userVerified, indicating if the user has passed the check of the authenticator (for example provided a matching fingerprint). The server can decide whether or not to stop the process if one of these flags is false. In the scenario of sign-up, there are very limited use cases where a valid request didn't check for the user presence or their validity, so you should enforce a strict checking (as we see later in the verify section, there are use cases where users don't have to be present in order to log in).
Now we check if the public key was created with an algorithm that we allowed in the creationOptions. To do so, we can take the credentialPublicKey.kty value and check it against a list of allowed algorithms we specified beforehand.
We also have to make sure we didn't receive any unwanted extensions. To do so, we look through the extensions attribute and compare all extensions to a list of white-labeled extensions that we want in our credential. This is also a good time to process the extensions you specified.

Now comes the fun part: We are done with verifying the context and can now start to verify the attestation itself. To do so, we have six different approaches that are all described in the specification. To not double this already huge article even more, I will only describe the general idea here:

At first, we determine which format our attestation has. We have a fmt attribute that tells us in a string representation which format the attestationStatement resembles. But it is actually more secure to also go through all attributes of attestationStatementand check if each attribute that the format specifies is included.
The attestation contains a signature. This is the unique proof that an authenticator is really who he claims to be, and every authenticator has their own methods to validate this signature.
The attestationStatement also contains an AAGUID(authenticator ID). You can use this ID to make a call to an external, trustworthy service (like FIDO) and if the ID matches their record, they will return a certificate or public key back to you (called a trust anchor). This can be used to determine if the authenticator is who they claim to be.
The attestation formats use different methods to encrypt their data. You now should use the trust anchor to validate the trustworthiness of the attestation.

If you made it this far, you actually did it: You made absolutely sure that the request was legit and that there is an actual user who wants to use your service and wants to sign up for it. All you have left to do now is to store the users credentials in your database.

First, check if the credential ID is already in use. If there already is a record for this ID in your database, deny the request and indicate that the client should issue a new credential.
Go ahead and register the new user.

We made it! We actually registered an user to our database, 100% compliant with all current standards. You could of course just skip steps 1 - 14 (excluding step 5) and save yourself a lot of stress and scripting, but then you couldn't be sure that you are actually operating securely in your app. And as a heads up, validating a login is way easier then signing up a new user!

Validating a login request

Step 1: Client-side validation

Similar to registering a new user, we have to first build up PublicKeyCredentialRequestOptions. We don't have to specify as many parameters, though. The necessary information that you need to get a stored users credentials is:

A server-side challenge that you can use to ensure this request is actually meant for your app
A timeout value in milliseconds
The ID of your app (namingly the URL on which your app is reachable)
Additional data which requirements an authenticator (e.g. Windows Hello, Touch ID) has to fulfill to be eligible to create a credential
A list of credentials that you would expect to receive (all credential IDs that you have stored about the user)

After calling navigator.credentials.get() with these options set, you will receive an Authenticator Assertion Response which also implements the Credential interface (so, basically the same as with registering a new user). We get the credential ID (as string and encoded as a byte array), client data (same as in registering an user), authenticator data (reduced version of what we know from registration), a signature and an userHandle, being the ID of the user on which this credential was registered (these are new). We again encode all this information to send it securely to our server.

Step 2: Server-side validation

Now that we got all information on our server, the first thing we have to get is our user credential record from our own database. We will need the public key that should belong to that request, so we look up the credential ID in our database. After that, we can do the request origin verification we already know from registering an user. We parse the clientDataJSON and check the following (if any of these sound new to you, go to Registering a new user - Server Part):

Is the operation (clientData.type) webauthn.get?
Do we recognize the provided challenge?
Did we expect the request to come from the given origin url?
Do we have a tokenBinding object and does it match our expectations regarding the TLS connection?

After this, we can validate the context of the request (again, same procedure as in the signup process). First we decrypt the authenticator data and then we check the following:

Does the Resource Provider ID match our expectations?
Is there an user presence flag set?
Is there an user verification flag set?
Does the request send us any extensions? Did we expect those?

In the last two steps, we get to the core of verifying: Does the credential match our records? To do so, we have to first create a sha265 encrypted hash of our clientDataJSON. Then we decrypt the signature that the client has sent us, and we create a joint byte array of the initial authenticator data and the hash of clientDataJSON. If we now verify this Byte Array with the public key in our records, we should get exactly the signature value (which was created in the same procedure and signed by the private key). If not, that means our public key doesn't match the private key of the authenticator and we should therefore deny the request.

In a last step, we look at the signCount of the request. This attribute indicates how often the key was already used to log in. We initialized the signCount with a value of 0 at signup time, and if the signCount provided is lower than the number we stored, we should also deny the request.

We did it!

And that's it! We now have a secure and intuitive method for our users to access their most private data. And the best thing: It even works completely offline!
If you want to see an implementation of this protocol, you can check out this GitHub repository.
And if you want to dig even deeper into how to build WebAuthn, stay tuned! I will publish an article walking through developing a WebAuthn server step by step soon.

Building towards a web without passwords

Tobias Urban — Sun, 06 Oct 2019 17:54:30 +0000

Think of all the online accounts that you have...

Basically all your online passwords in one picture

With nearly every one of these comes a (new) password. And for most of the people out there, passwords in the end are all the same. And as they get hacked in one app, all their online presences lie open.

In the last years, a few approaches came up to tackle that problem. Apps with large userbases like Facebook, Microsoft or Google are offering social logins that link your account directly with your social media account (therefore making them your primary Identity Provider), sparing you one more password to remember. But you are still trusting on that social media account staying secure, and you are giving out a lot of data to that Identity Provider (which is up to your judgement if that is a good thing).

Another approach are password managers. With just one (hopefully more secure) password you can access randomly generated passwords to all your accounts, making each secret unique and therefore independent from your other accounts. But again you are relying on a single source of truth, and if your master password is stolen, all of your accounts are breached.You could of course add additional layers of security with Multi-Factor Authentication, but these often require additional hardware and can be costly to implement for the App creator (e.g. when you have to pay for SMS fees).

Introducing: WebAuthentication

The major OS providers Microsoft, Google and Apple have tackled this problem already a few years ago when they introduced password-less authentication on their systems. It is nowadays a de-facto standard for new devices to have some sort of fingerprint or face recognition that offers you to access your devices without remembering any passwords. And the best feature: Usually your biometric data is stored directly on the device, making it impossible to breach some database of face or finger data.

With biometrics you are the password

Now that this technology has matured and got the go-to choice for most users in their everyday life, the question arised: How do we get that into the web? And frankly, the W3C tackled exactly this question with their WebAuthn specification. The specification is currently only a draft, but most of the major players have already adopted the proposed standards and it is useable on Windows, Android, iPhone (to some extend) and MacOS already.

The basic idea is quite simple: Instead of asking for a password, the browser uses native login methods (e.g. Windows Hello, Touch ID) to verify the user. The application then gets a huge package of encrypted information that can be used to verify that it was actually the user who tried to log in (and not some hacker). After successful registration, the browser then stores a private key on the users device, not exposable in any way. The server gets the according public key and a credential ID that it can store securely instead of a password.

If you are interested in the experience from the perspective of an user, check out this demo.

How is this more secure?

Every user who logs into an app is protected by multiple layers: First, the app has to find the user in its system. The app should have stored the credential ID next to the user ID. The credential ID is useful only for the browser in which the user signed in. If the browser recognizes the credential ID, it will then prompt the user for authentication. If the user passes the authentication (e.g. by providing a security key USB stick or by using Windows Hello / Touch ID), the browser can send some verifiable data to the app. This data will then be verified by the server with the public key it got at sign-up time. If this process succeeds, then the login is successful.

A good representation of how few Authn trusts your login approach

From an app standpoint, WebAuthn offers another awesome benefit: You don't have to store any passwords. All you get from an user is a public key and a credential ID. Both work only if the user in on their device, on your page URL. So let's assume your servers get breached and all credentials are stolen. In the classic password world, this means major security risks for everyone who ever worked on your app. With WebAuthn, your users can just move on, and as long as their device and face/finger/Security Key don't get stolen, their access to your app is still safe. And all other applications where they use WebAuthn to log in are not impacted by this at all.

So can I just use this everywhere now?

WebAuthentication is an exciting protocol that offers a new level of security to users who want to take an extra step in protecting their online data. But by design, this extra step comes with a few inconveniences that your users have to take. By nature, all credentials are stored on the clients device, more specifically in the app or browser that your user has used to log into your services. That means, as soon as your users switches to a new device or to a new browser on their current device, he has to sign up again with new login credentials for your service.

For scenarios like this you will always depend on other, more universal authentication methods. Let's say an user loses their device, so there is no way to log in with the credentials that you have currently stored. In moments like this, you should always be able to fall back to e.g. email verification, or a standard password.

Generally speaking, although WebAuthn does look promising in enabling a new layer of security, it is just a tool. It is always the responsibility of the app providers to ensure a secure environment for your users, and WebAuthn should be one of many locks that you put in front of your users data.

Enough Intro talk, where can I learn more?

This article is just the kick-off to a series of technical posts about WebAuthn that I plan to launch. You can find the articles here:
Getting started with WebAuthn: The basic flow
Securing your WebAuthn server: Response validation (Coming soon)
WebAuthn Step by Step: A specification rundown in code (Coming soon)

I have also prepared the code for the demo as a learning implementation on my Github. I tried to keep the documentation as extensive as possible, so you can just read through the source code and learn more about implementing this protocol. On the GitHub page you can also find a list of resources by other parties who have written great examples of how to get started with WebAuthn. And make sure to check out the official documentation as well!