Forem: Mikhail Shilkov

The Best Interview is No Interview: How I Get Jobs Without Applying

Mikhail Shilkov — Thu, 02 Jul 2020 06:16:48 +0000

Interviews

I think I'm reasonably good at passing job interviews.

When I was open for a job change a couple of years ago, I interviewed at eight local companies here in the Netherlands. I got job offers from seven of them. Of course, I have enough experience, and the market was scorching and candidate-friendly.

Besides, Dutch interviews are not as extensive compared to major tech companies in the U.S. For me, it took from 40 minutes to 4 hours of conversations to get to offer negotiation. There would typically be no whiteboard coding or take-home assignments.

I had two interviews at the U.S. companies too. I got offers from both of them.

Amazon flew me to Stockholm for a hiring event. Before jumping on a plane, I spent a day or two studying interviews at Amazon and an algorithms book. Amazon hired a hotel room and hosted four 1-hour sessions with me. There was a whiteboard standing, and every new interviewer would give me a coding or design challenge to solve within that hour. I wasn't well-calibrated for such interviews, and couldn't quite tell if I'm performing well. Apparently, I was okay, because I got an open offer for Canada, U.S., Luxembourg, and Germany to work on some Amazon projects.

I also had an interview with Jet.com—Amazon's competitor in retail—for a position in Dublin. Two online live-coding interviews, four onsite interviews with whiteboards: a tad stressful but also fun. Coincidentally, the onsite in Dublin was two weeks after the Amazon interview, and the offers came literally on the same day.

No-Interviews

There's one big problem with this narrative. I declined all of those nine offers.

I didn't feel like these were enough of a step forward for me. Offers from Amazon and Jet were cool, but not revolutionary enough to relocate away from a well-arranged life.

There were more declined offers in the past. In fact, in my whole 18-years (oh gosh) career as a software developer, I only accepted a single job offer after an interview process. I left that company nine months later, despite being already promoted in my role.

How did I get other jobs? Honestly, it felt like they happened by chance.

I got my first paid gig from my father when I was an 18-year-old student. He showed me screenshots of the applications from work and asked me if I can create that kind of apps. He knew nothing about software development but needed a bunch of utilities to calibrate hardware that they were working on.

Sure, I knew Delphi and C++ Builder, so I could make form-based apps. I prototyped one and got my first $100 paycheck. I started working for the company and spent the next three years learning the tech and taking over existing code. By the end of that engagement, I was writing all software in their stack from C code on microcontrollers with directly addressable memory to 60-screen Windows apps.

There wasn't much to learn there anymore. Still being a student, I hopped on a three-week bootcamp organized by a software consultancy company. They were teaching free classes to promote the best students to FTEs. I scored a second place on the overall leaderboard and got hired at the double my previous salary.

I spent five years there, learning from many amazing people. I tried myself in different roles from an individual coder to a technology "expert" to a team lead to a project manager. The company has been doing contracting projects for European customers ("outsourcing").

In the end, I quit the company to try contracting on my own. A friend of mine connected me to somebody in the Netherlands who needed help with his ongoing projects. The guy just gave me the first task, I completed it, he paid. The second task, the first lengthy project, the second project, and I got rolling.

I've been working in my bedroom for the next four years. My contact in the Netherlands got hired by Qualcomm and switched me to their projects. Eventually, Qualcomm decided to hire me full-time, and my family moved from Russia to the Netherlands.

Most certainly, Qualcomm had an extensive process for hiring, so I did fly over for an onsite "interview". The interview lasted 5 minutes: I said hi to everyone and then worked on the project for the rest of the day.

Follow The Passion

The other day, I interviewed somebody for a developer role at Pulumi, where I currently work. They had a screening interview first, and now going through five rounds of tech interviews. Live-coding, design questions, debugging problems, tell-me-about-the-time-when situations. Very common for anyone applying for a software engineering job in the U.S.

Except, I never had a single interview at Pulumi. I jumped on their product as a user as soon as the first beta went public. I enjoyed the product, published blog posts, used it for my pet projects, gave one of the first Pulumi talks in Europe.

When I was in Seattle for the Microsoft MVP summit, I pinged Pulumi Slack and grabbed a beer with Luke, our CTO. Soon after, I started doing paid contracting for Pulumi: writing blogs, docs, examples, then fixing tiny issues here and there. Eventually, I went to Seattle for the Pulumi 1.0 launch and a week onsite. Finally, after 12 months as a user and 6 months as a contractor, I got the contract signed for full-time employment.

I had no interviews. When I first started, I had zero experience with Go and very limited TypeScript, our main programming languages. I'm not a compiler guru, not ingrained into DevOps ecosystem, never worked on developer tooling or open-source SDKs.

However, I thoroughly enjoy my job. It means a lot to me. I love learning from exceptional people, and I have a lot of freedom.

Off The Beaten Track

There were two categories of companies that I interacted with. The first group invited me for an interview, and the second group allowed me to collaborate without a formal hiring process. I consistently chose to work in the latter group and enjoyed my time there. Why?

Every such job meant going out of a local maximum, out of comfort zone. Working on calibration software without any experience with metrology. Jumping between consulting projects, teams, and roles. Relocating to another country to join a multi-national company. Becoming one of the two non-U.S. employees at a DevOps tooling startup without ever seeing bash before.

None of these were "the next logical step" in my career. Maybe I could pass interviews at those companies, perhaps I wouldn't. That's mostly irrelevant because there wouldn't be a sequence of events to land me at a business-as-usual interview at one of those companies.

Expand Your Circle

There's a lot of arguments about the interview process being broken. A candidate spends several hours solving computer science puzzles on a whiteboard to get a job of fixing bugs in a React app. Companies may be too strict at not accepting great candidates and hiring unfit candidates at the same time. Perhaps that's all true.

I'm actually making a different point. A successful job search doesn't have to be centered around an interview. If my experience generalizes to others at all, here are some activities that may help you get the next great job:

Build a strong professional network. And not just a bunch of hiring managers. Diverse connections might perform even better since you'd get access to a broad set of opportunities. You don't know what you are looking for anyway.
Keep learning and be curious, even a topic seems to bring no immediate value. The goal is expanding your surface area of interest, curiosity, and skills.
Take part-time gigs, contracting, one-off apps. They probably won't make you rich on their own. Still, successful low-risk engagement with somebody may lead to longer happy and fulfilling relationships.
Build a public profile. Publish your code, write blogs, share your interests, design, run, and publish experiments. More than once was I surprised about the way this helped me with jobs, networking, learning, or getting a consulting gig.

And if you are wearing a hiring hat, try to give your potential future workers a chance to engage without full-blown employment commitment. Asking me to code a simple but useful real-life application. Running a free class for software engineering students. Hiring temporary contractors from distant countries. Those companies get me as their employee.

Enough about myself! I'm keen to learn about your experience. Whether you can relate to my story or disagree with my take, please leave a comment below.

How do AWS CloudFormation, Azure Resource Manager, and GCP Deployment Manager compare?

Mikhail Shilkov — Mon, 07 Oct 2019 12:16:15 +0000

They are all Infrastructure as Code tools of the respective cloud providers.

I'd love to learn about your personal experience. Missing features, developer experience, error handling, gaps, whatever!

Visualizing Serverless Cold Starts

Mikhail Shilkov — Wed, 03 Jul 2019 08:48:06 +0000

I wrote a lot about cold starts of serverless functions. The articles are full of charts and numbers which are hopefully useful but might be hard to internalize. I decided to come up with a way to represent colds starts visually.

I created HTTP functions that serve geographic maps (map credit Open Street Map). The map is a combination of small square tiles; each tile is 256 by 256 pixels. My selected map view consists of 12 tiles, so 12 requests are made to the serverless function to load a single view.

During each experiment, I load the map and then zoom-in three times. The very first view hits the function in a cold state. Subsequently, the zoomed views are loaded from the warm function. There is a timer next to the map which shows the total time elapsed since the beginning until the last tile arrives.

Cold Starts Visualized

All functions are implemented in Node.js and run in the geographically closest region to me (West Europe).

The functions load map tiles from the cloud storage (AWS S3, Google Cloud Storage, and Azure Blob Storage). So the duration is increased by loading SDK at startup and the storage read latency.

AWS Lambda

The following GIF movie is a recording of the experiment against an AWS Lambda:

The cold view took 1.9 seconds to load, while the warm views were between 200 and 600 milliseconds. The distinction is fairly visible but not extremely annoying: the first load feels like a small network glitch.

Google Cloud Functions

This GIF shows the experiment against a Google Cloud Function:

Loading the initial view took an extra second compared to AWS. It's not a dealbreaker, but the delay of 3 seconds is often quoted as psychologically important.

The tiles seem to appear more gradually; read more on that below.

Azure Functions

Here is another movie, this time for an Azure Function:

As expected from my previously published measurements, Azure Functions take significantly longer to start. A user has enough time to start wondering whether the map is broken.

I expect better results from Functions implemented in C#, but that would not be an apples-to-apples comparison.

How do providers handle parallel requests?

The map control fires 12 requests in parallel. All functions talk HTTP/2, so the old limit on the number of connections does not apply. Let's compare how those parallel requests are processed.

AWS Lambda

Each instance of AWS Lambda can handle a single request at a time. So, instead of hitting just one cold start, we hit 12 cold starts in parallel. To illustrate this, I’ve modified the function to color-code each tile based on the Lambda instance ID and to print that ID on the image:

AWS Lambda provisioned 12 instances to serve 12 requests in parallel

Effectively, the measured durations represent the roundtrip time for the slowest out of 12 parallel requests. It's not the average or median duration of the cold start.

Google Cloud Functions

Google uses the same one-execution-per-instance model, so I expected GCP Cloud Functions to behave precisely the same as AWS Lambda. However, I was wrong:

Google Cloud Function provisioned 3 instances to serve 12 parallel requests

Only three instances were created, and two of them handled multiple requests. It looks like GCP serializes the incoming requests and spreads them through a limited set of instances.

Azure Functions

Azure Functions have a different design: each instance of a function can handle multiple parallel requests at the same time. Thus, in theory, all 12 tiles could be served by the first instance created after the cold start.

In practice, multiple instances are created. The picture looks very similar to GCP:

Azure Function provisioned 4 instances to serve 12 parallel requests

There were four active instances, but the same one handled 9 out of 12 requests. This behavior seems to be quite consistent between multiple runs.

Conclusion

I hope that these visualizations are useful to get a better feel of the cold starts in serverless functions.

However, they are just examples. Don't treat the numbers as exact statistics for a given cloud provider. If you are curious, you can learn more in Serverless Cold Starts series of articles.

Globally-distributed Serverless Application in 100 Lines of Code. Infrastructure Included!

Mikhail Shilkov — Wed, 03 Jul 2019 07:29:13 +0000

Pulumi is excellent at connecting multiple cloud components into a cohesive application. In my previous post, I introduced the way to mix JavaScript or TypeScript serverless functions directly into the cloud infrastructure programs.

Today, I will build a serverless application with both the data store and the HTTP endpoint located close to end users to ensure prompt response time. The entire application runs on top of managed Azure services and is defined as a single Pulumi program in TypeScript.

Baseline

I'm going to build a URL shortener: a simple HTTP endpoint which accepts a shortcode in the URL and then redirects a user to the full URL associated with the given short code.

I start simple and make a non-distributed version of the application first. It consists of two main components: a Cosmos DB container to store URL mappings and two Azure Functions to handle HTTP requests:

The URL Shortener app deployed to a single location

Let's define this infrastructure in a Pulumi program.

Cosmos DB Collection

Arguably, Cosmos DB is overkill in such a simple scenario: we just need a key-value store, so Table Storage would suffice. However, Cosmos DB comes handy at the multi-region setup later on.

Next to a standard resource group definition, I create a Cosmos DB account with location set to a single region and Session consistency level:

import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure";
import * as cosmos from "@azure/cosmos";

const location = "westus"; // To be changed to multiple configurable locations

let resourceGroup = new azure.core.ResourceGroup("UrlShorterner", {
   location,
});

let cosmosdb = new azure.cosmosdb.Account("UrlStore", {
   resourceGroupName: resourceGroup.name,
   location,
   geoLocations: [{ location, failoverPriority: 0 }],
   offerType: "Standard",
   consistencyPolicy: {
       consistencyLevel: "Session",
       maxIntervalInSeconds: 5,
       maxStalenessPrefix: 100,
   },
});

Cosmos DB has a hierarchical structure of Accounts, Databases, and Containers. Unfortunately, Cosmos DB containers can’t be defined as Pulumi resources yet. As a workaround, I defined a helper function getContainer to create a database and a container using Cosmos SDK, see here.

Function App

Serverless Azure Functions are going to handle the HTTP layer in my application. I use the technique of serverless functions as callbacks to define Azure Functions inline inside my Pulumi program:

const fn = new azure.appservice.HttpEventSubscription("GetUrl", {
   resourceGroup,
   location,
   route: "{key}",
   callback: async (_, request: azure.appservice.HttpRequest) => {
       const endpoint = cosmosdb.endpoint.get();
       const masterKey = cosmosdb.primaryMasterKey.get();

       const container = await getContainer(endpoint, masterKey);

       const key = request.params['key'];
       try {
           const response = await container.item(key.toString()).read();
           return {
               status: 301,
               headers: { "location": response.body.url },
               body: '',
           };
       } catch (e) {
           return { status: 404, body: 'Short URL not found' }
       }
   }
});

export const url = fn.url;

I supply the template {key} as route parameter so that the function accepts wildcard URLs and extract the wildcard value as a key parameter available inside the request object. Then, I use the received key to look up the full URL. The URL is returned to the client as a header of the 301 Redirect response. 404 Not Found is returned in case the requested document does not exist—the naughty Cosmos SDK throws an error in this scenario.

Note how the Cosmos DB parameters endpoint and primaryMasterKey are used directly inside the function. There is no need to manage the keys manually or explicitly put them into application settings: the generated keys are injected into the code at the time of deployment. In practice, a better idea is to store the key in a KeyVault and read it via application settings, but I'll leave this for a separate article.

A function to add a URL to the database looks quite similar to the above:

const fn = new azure.appservice.HttpEventSubscription("AddUrl", {
   resourceGroup,
   methods: ["POST"],
   callback: async (_, request: azure.appservice.HttpRequest) => {
       const endpoint = cosmosdb.endpoint.get();
       const masterKey = cosmosdb.primaryMasterKey.get();
       const container = await getContainer(endpoint, masterKey, location);

       await container.items.create(request.body);
       return { status: 200, body: 'Short URL saved' };
   }
});
export const addEndpoint = fn.url;

The key differences are:

Using POST as the accepted HTTP method
Creating a new item in Cosmos DB
Using request.body as the payload to save

I might need to add some validation in a real application—skipped for now.

Trying It Out

Now, when a user navigates to an address like https://geturl-xyz.azurewebsites.net/api/URLKEY, they get redirected to the full URL associated with the key. The URL is not quite short yet, but we could make it so with a custom domain configuration.

Presumably, we expect our URL shortener to be popular around the world. Therefore, one key aspect is to make sure that the service responds fast and enables a smooth user experience. Let's measure the response time of our first version from different spots around the world:

Location	Response time
San Francisco	133 ms
New York	272 ms
London	383 ms
Frankfurt	420 ms
Tel-Aviv	470 ms
Hong-Kong	437 ms
Brisbane	449 ms

The further we get from the West US region, the slower the responses become.

How can we do better?

Bring Compute and Data Closer to Users

It's hard to beat the speed of light, so if we want to respond fast, we need to bring both code and data close to any target user.

Luckily, the most involved aspect of that—the data distribution—can be handled by Cosmos DB multi-region accounts. We need to take care of deploying the code in multiple locations and routing the traffic to the nearest one. Here is the plan:

The multi-region URL Shortener app

I don't care too much about the latency of Add URL function, so I left it out of the picture.

Configuring the locations

The above picture shows three Azure regions, but since I'm using a general-purpose programming language, I can handle any number of locations in the same way. Actually, I put the list of target regions in Pulumi config file and read it at execution time:

const config = new pulumi.Config();
const locations = config.require("locations").split(',');
const primaryLocation = locations[0];

The order of regions defines the priority.

Multi-region Cosmos DB Account

Since locations is a simple array, I can map this array to produce the array of geoLocations to be set for my Cosmos DB:

let cosmosdb = new azure.cosmosdb.Account("url-cosmos", {
   location: primaryLocation,
   geoLocations: locations.map((location, failoverPriority) => ({
       location,
       failoverPriority
   })),
   /* ... other properties stay the same ... */

Right here, the full power of a programming language at my fingertips!

Multi-region Serverless App

Azure Traffic Manager is a DNS-based traffic load balancer that enables you to distribute traffic optimally to services across global Azure regions while providing high availability and responsiveness.

Sounds like what I need for my global application! Let's define a Traffic Manager profile:

const profile = new azure.trafficmanager.Profile("UrlShortEndpoint", {
   resourceGroupName: resourceGroup.name,
   trafficRoutingMethod: 'Performance',
   dnsConfigs: [{
       relativeName: "shorturls",
       ttl: 60,
   }],
   monitorConfigs: [{
       protocol: 'HTTP',
       port: 80,
       path: '/api/ping',
   }]
});

I set the routing method to Performance:

Select Performance when you have endpoints in different geographic locations and you want end users to use the "closest" endpoint in terms of the lowest network latency.

Now, I need to create a Function App in each of the target regions. That's easy to achieve with a for..of loop:

for (const location of locations) {
   const fn = new azure.appservice.HttpEventSubscription(`GetUrl-${location}`, {
       resourceGroup,
       location,
       route: "{key}",
       callback: /* ... the function stays the same ... */
   });
}

Finally, I set up an Endpoint for each Function App to bind them to the Traffic Manager. I do so in the same loop as above:

for (const location of locations) {
   const fn = new azure.appservice.HttpEventSubscription(`GetUrl-${location}`, {
       /* ... function definition as shown above ... */
   });

   const app = fn.functionApp;

   new azure.trafficmanager.Endpoint(`tme${location}`, {
       resourceGroupName: resourceGroup.name,
       profileName: profile.name,
       type: 'azureEndpoints',
       targetResourceId: app.id,
       target: app.defaultHostname,
       endpointLocation: app.location,
   });
}

I assign the properties of each Function App to the corresponding Endpoint with a simple object initializer expression.

Everything is in place. I can export the Traffic Manager endpoint:

export const endpoint = pulumi.interpolate `http://${profile.fqdn}/api/{key}`;

Time to re-run the latency tests.

Results

I deployed the distributed infrastructure to five Azure regions: West US, East US, West Europe, East Asia, and Australia East. The results are shown in the table below, next to the values from the initial run:

Location	Single region	Multiple regions
San Francisco	133 ms	140 ms
New York	272 ms	152 ms
London	383 ms	150 ms
Frankfurt	420 ms	130 ms
Tel-Aviv	470 ms	307 ms
Hong-Kong	437 ms	149 ms
Brisbane	449 ms	529 ms

We got a much smoother distribution of response time. There's no region close enough to Tel-Aviv, so its latency is still high-ish. I checked Brisbane, and the traffic was somehow directed to the East US region, so the latency hasn't improved at all. For today, I guess I'll stick to the hypothesis "Australian internet is slow" and advise to always test your target scenarios without trusting the providers blindly.

More importantly, I walked you through a scenario of developing an end-to-end application with Pulumi. I highlighted the power of leveraging familiar techniques coming from the TypeScript realm. You can find the full code in Pulumi examples.

Cloud brings superpowers to developer's hands. You just need to use those efficiently.

Hosting a Static Website on Azure with Pulumi

Mikhail Shilkov — Wed, 03 Jul 2019 07:03:49 +0000

Static websites are back in the mainstream these days. Website generators like Jekyll, Hugo, or Gatsby, make it fairly easy to combine templates and markdown pages to produce static HTML files. Static assets are the simplest thing to serve and cache, so the whole setup ends up being fast and cost-efficient.

Many platforms offer services to host such static websites. This post explains the steps to create the infrastructure to do so on Microsoft Azure.

Setting up the infrastructure to serve a static website doesn't sound like it would be all that difficult, but when you consider HTTPS certificates, content distribution networks, and attaching it to a custom domain, integrating all the components can be quite daunting.

Fortunately, this is a task where Pulumi shines. Pulumi's code-centric approach not only makes configuring cloud resources easier to do and maintain, but it also eliminates the pain of integrating multiple services.

Overview

Our goal is to create a static website with a custom domain—I'll use an imaginary demo.pulumi.com for this article. In 2019, my website has to support HTTPS, so we need to create a custom TLS certificate too.

The final solution consists of several Azure services:

Static files will be stored in a Blob Container inside a Storage Account
The Storage Account will have Static Website feature enabled to have some basic URL rewrite rules
We'll put an Azure CDN Endpoint in front of the container to support the custom domain over TLS
Azure CDN will self-manage the TLS certificate
Our custom DNS provider will have the rule to point to the CDN endpoint (that's a manual step)

The diagram below outlines the interaction of these components:

Static website running on Azure and defined in Pulumi

Let's break down how to configure each component using Pulumi.

Resource Group

Let's start a new Pulumi program, import the Pulumi packages, and define a new resource group:

import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure";

const resourceGroup = new azure.core.ResourceGroup("demo-rg", {
   location: azure.WestEurope,
});

Storage Account

The Storage Account will contain our static website's assets:

const storageAccount = new azure.storage.Account("demopulumi", {
   resourceGroupName: resourceGroup.name,
   accountReplicationType: "LRS",
   accountTier: "Standard",
   accountKind: "StorageV2",
});

The only trick here is to make sure that the account kind is V2; otherwise, it won't support the static website feature.

Static Website Hosting in Azure Storage

Any storage container could be exposed as a web endpoint. However, that's not flexible enough. The URL would always include the container name and the exact file name, so the user would have to ask for https://demo.pulumi.com/containername/index.html instead of simply https://demo.pulumi.com/.

Enabling Static Website Hosting in Azure Storage improves this experience. A dedicated container $web gets automatically created, which also has special treatment for index and 404 documents.

The bad news is that Static Website Hosting is not a part of Azure Resource Manager API, and therefore, it's not available out-of-the-box in ARM templates, Terraform, or Pulumi. We can enable this feature with Azure CLI, so the solution is to create a dynamic Pulumi resource which enables Pulumi experience while delegating the work to the CLI. You can find the full source code for the dynamic resource in this example, but the usage is quite trivial:

const staticWebsite = new StorageStaticWebsite("demopulumi-static",
   { accountName: storageAccount.name },
   { parent: storageAccount });

// Web endpoint to the website
export const staticEndpoint = staticWebsite.endpoint;

The last line exports the static website endpoint. It will look something like https://demopulumi01234abc.z6.web.core.windows.net/—no custom domain yet, but already functional—once we deploy some static files in there.

Static Files

Now, it's time to upload the static files to the $web Blob Container.

For this demo, I've created a folder wwwroot with two files in it: index.html and 404.html. I can upload those files with the following Pulumi snippet:

["index.html", "404.html"].map(name =>
   new azure.storage.Blob(name, {
       name,
       resourceGroupName: resourceGroup.name,
       storageAccountName: storageAccount.name,
       storageContainerName: staticWebsite.webContainerName,
       type: "block",
       source: `./wwwroot/${name}`,
       contentType: "text/html",
   })
);

In practice, your static website might contain hundreds or thousands of files. At that point, you might want to split the file upload operation from Pulumi and do it as a separate step in your CI/CD pipeline.

Azure CDN

To make the static website files available over our custom domain and HTTPS, we need to create an Azure CDN Endpoint and to point it to the storage account:

const cdn = new azure.cdn.Profile("demo-cdn", {
   resourceGroupName: resourceGroup.name,
   sku: "Standard_Microsoft",
});

const endpoint = new azure.cdn.Endpoint("demo-cdn-ep", {
   name: "demopulumi",
   resourceGroupName: resourceGroup.name,
   profileName: cdn.name,
   isHttpAllowed: true,
   isHttpsAllowed: true,
   originHostHeader: staticWebsite.hostName,
   origins: [{
       name: "blobstorage",
       hostName: staticWebsite.hostName,
   }],
});

// CDN endpoint to the website. Allow it some time after the deployment to get ready.
export const cdnEndpoint = pulumi.interpolate`https://${endpoint.hostName}/`;

I specified an explicit name for the CDN Endpoint: demopulumi. This name has to be globally unique because it is a part of the endpoint URL https://demopulumi.azureedge.net. Please pick a custom name before running the program.

I pointed the CDN origin to the static website name with staticWebsite.hostName. As usual, it's easy to link resources in Pulumi code!

You might need to wait a few minutes before your content is visible as the CDN configuration is not immediately executed. I've set isHttpAllowed to true because HTTP is available sooner than HTTPS; feel free to switch it off for your production configuration.

Configure a Domain DNS Rule

You've probably registered your domain with some third-party provider. Follow the instructions of your provider to configure a CNAME rule for the website's DNS. For a custom domain demo.pulumi.com, the CNAME entry demo would be linked to the endpoint demopulumi.azureedge.net.

CNAME entries don't support "naked" domains like pulumi.com. If you want to set up a top-level domain to be served from the static Azure website, you'd have to use an Alias DNS record, which isn't supported by some DNS providers. Please check with your provider for available options.

The following step assumes that a CNAME record is configured; otherwise, it would fail with a validation error.

Custom Domain and TLS

The final step is to point our custom domain to the CDN endpoint and provision a TLS certificate to enable HTTPS support. Once again, these operations are not parts of the ARM API surface, so another dynamic resource was created to support them.

The usage is quite straightforward, just make sure to use your own domain in the following snippet:

const customDomain = new CDNCustomDomainResource("cdn-custom-domain", {
   resourceGroupName: resourceGroup.name,
   // Ensure that there is a CNAME record for demo pointing.pulumi.com to demopulumi.azureedge.net.
   // You would do that in your domain registrar's portal.
   customDomainHostName: "demo.pulumi.com",
   profileName: cdn.name,
   endpointName: endpoint.name,
   // This will enable HTTPS through Azure's one-click automated certificate deployment.
   // The certificate is fully managed by Azure from provisioning to automatic renewal
   // at no additional cost to you.
   httpsEnabled: true,
}, { parent: endpoint });

Bring It Live!

And we are done! Run pulumi up and make sure that all resources get created successfully.

Start testing with staticEndpoint—it's the first one to become available. cdnEndpoint might return some 404's at first, be patient. Then, try your custom domain with HTTP. Finally, the TLS certificate takes quite a while to be registered, so come back in an hour or so to test HTTPS.

While a "static website" may sound simple, we went through a rather complicated process to wire all the components together. Some Azure services and features are more straightforward to automate than others, but in the end, we combined all of them into one cohesive Pulumi program to host a static website, served over HTTPS and from a worldwide CDN. That's the power of a general-purpose programming language applied to the task of sophisticated infrastructure automation. Once the reusable components are in place, reliable and reproducible deployments become a reality.

From YAML to TypeScript: Developer's View on Cloud Automation

Mikhail Shilkov — Wed, 20 Feb 2019 11:57:50 +0000

The rise of managed cloud services, cloud-native and serverless applications brings both new possibilities and challenges. More and more practices from software development process like version control, code review, continuous integration, and automated testing are applied to the cloud infrastructure automation.

Most existing tools suggest defining infrastructure in text-based markup formats, YAML being the favorite. In this article, I'm making a case for using real programming languages like TypeScript instead. Such a change makes even more software development practices applicable to the infrastructure realm.

Sample Application

It's easier to make a case given a specific example. For this essay, I define a URL Shortener application, a basic clone of tinyurl.com or bit.ly. There is an administrative page where one can define short aliases for long URLs:

URL Shortener sample app

Now, whenever a visitor goes to the base URL of the application + an existing alias, they get redirected to the full URL.

This app is simple to describe but involves enough moving parts to be representative of some real-world issues. As a bonus, there are many existing implementations on the web to compare with.

Serverless URL Shortener

I'm a big proponent of the serverless architecture: the style of cloud applications being a combination of serverless functions and managed cloud services. They are fast to develop, effortless to run and cost pennies unless the application gets lots of users. However, even serverless applications have to deal with infrastructure, like databases, queues, and other sources of events and destinations of data.

My examples are going to use Amazon AWS, but this could be Microsoft Azure or Google Cloud Platform too.

So, the gist is to store URLs with short names as key-value pairs in Amazon DynamoDB and use AWS Lambdas to run the application code. Here is the initial sketch:

URL Shortener with AWS Lambda and DynamoDB

The Lambda at the top receives an event when somebody decides to add a new URL. It extracts the name and the URL from the request and saves them as an item in the DynamoDB table.

The Lambda at the bottom is called whenever a user navigates to a short URL. The code reads the full URL based on the requested path and returns a 301 response with the corresponding location.

Here is the implementation of the Open URL Lambda in JavaScript:

const aws = require('aws-sdk');
const table = new aws.DynamoDB.DocumentClient();

exports.handler = async (event) => {
  const name = event.path.substring(1);

  const params = { TableName: "urls", Key: { "name": name } };
  const value = await table.get(params).promise();

  const url = value && value.Item && value.Item.url;
  return url 
    ? { statusCode: 301, body: "", headers: { "Location": url } }
    : { statusCode: 404, body: name + " not found" };
};

That's 11 lines of code. I'll skip the implementation of Add URL function because it's very similar. Considering a third function to list the existing URLs for UI, we might end up with 30-40 lines of JavaScript in total.

So, how do we deploy the application?

Well, before we do that, we should realize that the above picture was an over-simplification:

AWS Lambda can't handle HTTP requests directly, so we need to add AWS API Gateway in front of it.
We also need to serve some static files for the UI, which we'll put into AWS S3 and proxy it with the same API Gateway.

Here is the updated diagram:

API Gateway, Lambda, DynamoDB, and S3

This is a viable design, but the details are even more complicated:

API Gateway is a complex beast which needs Stages, Deployments, and REST Endpoints to be appropriately configured.
Permissions and Policies need to be defined so that API Gateway could call Lambda and Lambda could access DynamoDB.
Static Files should go to S3 Bucket Objects.

So, the actual setup involves a couple of dozen objects to be configured in AWS:

All cloud resources to be provisioned

How do we approach this task?

Options to Provision the Infrastructure

There are many options to provision a cloud application, each one has its trade-offs. Let's quickly go through the list of possibilities to understand the landscape.

AWS Web Console

AWS, like any other cloud, has a web user interface to configure its resources:

AWS Web Console

That's a decent place to start—good for experimenting, figuring out the available options, following the tutorials, i.e., for exploration.

However, it doesn't suit particularly well for long-lived ever-changing applications developed in teams. A manually clicked deployment is pretty hard to reproduce in the exact manner, which becomes a maintainability issue pretty fast.

AWS Command Line Interface

The AWS Command Line Interface (CLI) is a unified tool to manage all AWS services from a command prompt. You write the calls like

aws apigateway create-rest-api --name 'My First API' --description 'This is my first API'

aws apigateway create-stage --rest-api-id 1234123412 --stage-name 'dev' --description 'Development stage' --deployment-id a1b2c3

The initial experience might not be as smooth as clicking buttons in the browser, but the huge benefit is that you can reuse commands that you once wrote. You can build scripts by combining many commands into cohesive scenarios. So, your colleague can benefit from the same script that you created. You can provision multiple environments by parameterizing the scripts.

Frankly speaking, I've never done that for several reasons:

CLI scripts feel too imperative to me. I have to describe "how" to do things, not "what" I want to get in the end.
There seems to be no good story for updating existing resources. Do I write small delta scripts for each change? Do I have to keep them forever and run the full suite every time I need a new environment?
If a failure occurs mid-way through the script, I need to manually repair everything to a consistent state. This gets messy real quick, and I have no desire to exercise this process, especially in production.

To overcome such limitations, the notion of the Desired State Configuration (DSC) was invented. Under this paradigm, one describes the desired layout of the infrastructure, and then the tooling takes care of either provisioning it from scratch or applying the required changes to an existing environment.

Which tool provides DSC model for AWS? There are legions.

AWS CloudFormation

AWS CloudFormation is the first-party tool for Desired State Configuration management from Amazon. CloudFormation templates use YAML to describe all the infrastructure resources of AWS.

Here is a snippet from a private URL shortener example kindly provided at AWS blog:

Resources:
  S3BucketForURLs:
    Type: "AWS::S3::Bucket"
    DeletionPolicy: Delete
    Properties:
      BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ]
      WebsiteConfiguration:
        IndexDocument: "index.html"
      LifecycleConfiguration:
        Rules:
          -
            Id: DisposeShortUrls
            ExpirationInDays: !Ref URLExpiration
            Prefix: "u"
            Status: Enabled

This is just a very short fragment: the complete example consists of 317 lines YAML. That's an order of magnitude more than the actual JavaScript code that we have in the application!

CloudFormation is a powerful tool, but it demands quite some learning to be done to master it. Moreover, it's specific to AWS: you won't be able to transfer the skill to other cloud providers.

Wouldn't it be great if there was a universal DSC format? Meet Terraform.

Terraform

HashiCorp Terraform is an open source tool to define infrastructure in declarative configuration files. It has a pluggable architecture, so the tool supports all major clouds and even hybrid scenarios.

The custom text-based Terraform .tf format is used to define the configurations. The templating language is quite powerful, and once you learn it, you can use it for different cloud providers.

Here is a snippet from AWS Lambda Short URL Generator example:

resource "aws_api_gateway_rest_api" "short_urls_api_gateway" {
  name        = "Short URLs API"
  description = "API for managing short URLs."
}
resource "aws_api_gateway_usage_plan" "short_urls_admin_api_key_usage_plan" {
  name         = "Short URLs admin API key usage plan"
  description  = "Usage plan for the admin API key for Short URLS."
  api_stages {
    api_id = "${aws_api_gateway_rest_api.short_urls_api_gateway.id}"
    stage  = "${aws_api_gateway_deployment.short_url_api_deployment.stage_name}"
  }
}

This time, the complete example is around 450 lines of textual templates. Are there ways to reduce the size of the infrastructure definition?

Yes, by raising the level of abstraction. It's possible with Terraform's modules, or by using other, more specialized tools.

Serverless Framework and SAM

The Serverless Framework is an infrastructure management tool focused on serverless applications. It works across cloud providers (AWS support is the strongest though) and only exposes features related to building applications with cloud functions.

The benefit is that it's much more concise. Once again, the tool is using YAML to define the templates, here is the snippet from Serverless URL Shortener example:

functions:
  store:
    handler: api/store.handle
    events:
      - http:
          path: /
          method: post
          cors: true

The domain-specific language yields a shorter definition: this example has 45 lines of YAML + 123 lines of JavaScript functions.

However, the conciseness has a flip side: as soon as you veer outside of the fairly "thin" golden path—the cloud functions and an incomplete list of event sources—you have to fall back to more generic tools like CloudFormation. As soon as your landscape includes lower-level infrastructure work or some container-based components, you're stuck using multiple config languages and tools again.

Amazon's AWS Serverless Application Model (SAM) looks very similar to the Serverless Framework but tailored to be AWS-specific.

Is that the end game? I don't think so.

Desired Properties of Infrastructure Definition Tool

So what have we learned while going through the existing landscape? The perfect infrastructure tools should:

Provide reproducible results of deployments
Be scriptable, i.e., require no human intervention after the definition is complete
Define the desired state rather than exact steps to achieve it
Support multiple cloud providers and hybrid scenarios
Be universal in the sense of using the same tool to define any type of resource
Be succinct and concise to stay readable and manageable
~~Use YAML-based format~~

Nah, I crossed out the last item. YAML seems to be the most popular language among this class of tools (and I haven't even touched Kubernetes yet!), but I'm not convinced it works well for me. YAML has many flaws, and I just don't want to use it.

Have you noticed that I haven't mentioned Infrastructure as code a single time yet? Well, here we go (from Wikipedia):

Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools.

Shouldn't it be called "Infrastructure as definition files", or "Infrastructure as YAML"?

As a software developer, what I really want is "Infrastructure as actual code, you know, the program thing". I want to use the same language that I already know. I want to stay in the same editor. I want to get IntelliSense auto-completion when I type. I want to see the compilation errors when what I typed is not syntactically correct. I want to reuse the developer skills that I already have. I want to come up with abstractions to generalize my code and create reusable components. I want to leverage the open-source community who would create much better components than I ever could. I want to combine the code and infrastructure in one code project.

If you are with me on that, keep reading. You get all of that with Pulumi.

Pulumi

Pulumi is a tool to build cloud-based software using real programming languages. They support all major cloud providers, plus Kubernetes.

Pulumi programming model supports Go and Python too, but I'm going to use TypeScript for the rest of the article.

While prototyping a URL shortener, I explain the fundamental way of working and illustrate the benefits and some trade-offs. If you want to follow along, install Pulumi.

How Pulumi Works

Let's start defining our URL shortener application in TypeScript. I installed @pulumi/pulumi and @pulumi/aws NPM modules so that I can start the program. The first resource to create is a DynamoDB table:

    import * as aws from "@pulumi/aws";

    // A DynamoDB table with a single primary key
    let counterTable = new aws.dynamodb.Table("urls", {
        name: "urls",
        attributes: [
            { name: "name", type: "S" },
        ],
        hashKey: "name",
        readCapacity: 1,
        writeCapacity: 1
    });

I use pulumi CLI to run this program to provision the actual resource in AWS:

> pulumi up

Previewing update (urlshortener):

     Type                   Name             Plan
 +   pulumi:pulumi:Stack    urlshortener     create
 +    aws:dynamodb:Table    urls             create

Resources:
    + 2 to create

Do you want to perform this update? yes
Updating (urlshortener):

     Type                   Name             Status
 +   pulumi:pulumi:Stack    urlshortener     created
 +    aws:dynamodb:Table    urls             created

Resources:
    + 2 created

The CLI first shows the preview of the changes to be made, and when I confirm, it creates the resource. It also creates a stack—a container for all the resources of the application.

This code might look like an imperative command to create a DynamoDB table, but it actually isn't. If I go ahead and change readCapacity to 2 and then re-run pulumi up, it produces a different outcome:

> pulumi up

Previewing update (urlshortener):

     Type                   Name             Plan
     pulumi:pulumi:Stack    urlshortener        
 ~   aws:dynamodb:Table     urls             update  [diff: ~readCapacity]

Resources:
    ~ 1 to update
    1 unchanged

It detects the exact change that I made and suggests an update. The following picture illustrates how Pulumi works:

How Pulumi works

index.ts in the red square is my program. Pulumi's language host understands TypeScript and translates the code to commands to the internal engine. As a result, the engine builds a tree of resources-to-be-provisioned, the desired state of the infrastructure.

The end state of the last deployment is persisted in the storage (can be in pulumi.com backend or a file on disk). The engine then compares the current state of the system with the desired state of the program and calculates the delta in terms of create-update-delete commands to the cloud provider.

Help Of Types

Now I can proceed to the code that defines a Lambda function:

// Create a Role giving our Lambda access.
let policy: aws.iam.PolicyDocument = { /* Redacted for brevity */ };
let role = new aws.iam.Role("lambda-role", {
    assumeRolePolicy: JSON.stringify(policy),
});
let fullAccess = new aws.iam.RolePolicyAttachment("lambda-access", {
    role: role,
    policyArn: aws.iam.AWSLambdaFullAccess,
});

// Create a Lambda function, using code from the `./app` folder.
let lambda = new aws.lambda.Function("lambda-get", {
    runtime: aws.lambda.NodeJS8d10Runtime,
    code: new pulumi.asset.AssetArchive({
        ".": new pulumi.asset.FileArchive("./app"),
    }),
    timeout: 300,
    handler: "read.handler",
    role: role.arn,
    environment: { 
        variables: {
            "COUNTER_TABLE": counterTable.name
        }
    },
}, { dependsOn: [fullAccess] });

You can see that the complexity kicked in and the code size is growing. However, now I start to gain real benefits from using a typed programming language:

I'm using objects in the definitions of other object's parameters. If I misspell their name, I don't get a runtime failure but an immediate error message from the editor.
If I don't know which options I need to provide, I can go to the type definition and look it up (or use IntelliSense).
If I forget to specify a mandatory option, I get a clear error.
If the type of the input parameter doesn't match the type of the object I'm passing, I get an error again.
I can use language features like JSON.stringify right inside my program. In fact, I can reference and use any NPM module.

You can see the code for API Gateway here. It looks too verbose, doesn't it? Moreover, I'm only half-way through with only one Lambda function defined.

Reusable Components

We can do better than that. Here is the improved definition of the same Lambda function:

import { Lambda } from "./lambda";

const func = new Lambda("lambda-get", {
    path: "./app",
    file: "read",
    environment: { 
       "COUNTER_TABLE": counterTable.name
    },
});

Now, isn't that beautiful? Only the essential options remained, while all the machinery is gone. Well, it's not completely gone, it's been hidden behind an abstraction.

I defined a custom component called Lambda:

export interface LambdaOptions {
    readonly path: string;
    readonly file: string;

    readonly environment?:  pulumi.Input<{
        [key: string]: pulumi.Input<string>;
    }>;    
}

export class Lambda extends pulumi.ComponentResource {
    public readonly lambda: aws.lambda.Function;

    constructor(name: string,
        options: LambdaOptions,
        opts?: pulumi.ResourceOptions) {

        super("my:Lambda", name, opts);

        const role = //... Role as defined in the last snippet
        const fullAccess = //... RolePolicyAttachment as defined in the last snippet

        this.lambda = new aws.lambda.Function(`${name}-func`, {
            runtime: aws.lambda.NodeJS8d10Runtime,
            code: new pulumi.asset.AssetArchive({
                ".": new pulumi.asset.FileArchive(options.path),
            }),
            timeout: 300,
            handler: `${options.file}.handler`,
            role: role.arn,
            environment: {
                variables: options.environment
            }
        }, { dependsOn: [fullAccess], parent: this });
    }
}

The interface LambdaOptions defines options that are important for my abstraction. The class Lambda derives from pulumi.ComponentResource and creates all the child resources in its constructor.

A nice effect is that one can see the structure in pulumi preview:

Previewing update (urlshortener):

     Type                                Name                  Plan
 +   pulumi:pulumi:Stack                 urlshortener          create
 +     my:Lambda                         lambda-get            create
 +       aws:iam:Role                    lambda-get-role       create
 +       aws:iam:RolePolicyAttachment    lambda-get-access     create
 +       aws:lambda:Function             lambda-get-func       create
 +     aws:dynamodb:Table                urls                  create

The Endpoint component simplifies the definition of API Gateway (see the source):

const api = new Endpoint("urlapi", {
    path: "/{proxy+}",
    lambda: func.lambda
});

The component hides the complexity from the clients—if the abstraction was selected correctly, that is. The component class can be reused in multiple places, in several projects, across teams, etc.

Standard Component Library

In fact, Pulumi team came up with lots of high-level components that build abstractions on top of raw resources. The components from the @pulumi/cloud-aws package are particularly useful for serverless applications.

Here is the full URL shortener application with DynamoDB table, Lambdas, API Gateway, and S3-based static files:

import * as aws from "@pulumi/cloud-aws";

// Create a table `urls`, with `name` as primary key.
let urlTable = new aws.Table("urls", "name");

// Create a web server.
let endpoint = new aws.API("urlshortener");

// Serve all files in the www directory to the root.
endpoint.static("/", "www");

// GET /url/{name} redirects to the target URL based on a short-name.
endpoint.get("/url/{name}", async (req, res) => {
    let name = req.params["name"];
    let value = await urlTable.get({name});
    let url = value && value.url;

    // If we found an entry, 301 redirect to it; else, 404.
    if (url) {
        res.setHeader("Location", url);
        res.status(301);
        res.end("");
    }
    else {
        res.status(404);
        res.end("");
    }
});

// POST /url registers a new URL with a given short-name.
endpoint.post("/url", async (req, res) => {
    let url = req.query["url"];
    let name = req.query["name"];
    await urlTable.insert({ name, url });
    res.json({ shortenedURLName: name });
});

export let endpointUrl = endpoint.publish().url;

The coolest thing here is that the actual implementation code of AWS Lambdas is intertwined with the definition of resources. The code looks very similar to an Express application. AWS Lambdas are defined as TypeScript lambdas. All strongly typed and compile-time checked.

It's worth noting that at the moment such high-level components only exist in TypeScript. One could create their custom components in Python or Go, but there is no standard library available. Pulumi folks are actively trying to figure out a way to bridge this gap.

Avoiding Vendor Lock-in?

If you look closely at the previous code block, you notice that only one line is AWS-specific: the import statement. The rest is just naming.

We can get rid of that one too: just change the import to import * as cloud from "@pulumi/cloud"; and replace aws. with cloud. everywhere. Now, we'd have to go to the stack configuration file and specify the cloud provider there:

config:
  cloud:provider: aws

Which is enough to make the application work again!

Vendor lock-in seems to be a big concern among many people when it comes to cloud architectures heavily relying on managed cloud services, including serverless applications. While I don't necessarily share those concerns and am not sure if generic abstractions are the right way to go, Pulumi Cloud library can be one direction for the exploration.

The following picture illustrates the choice of the level of abstraction that Pulumi provides:

Pulumi abstraction layers

Working on top of the cloud provider's API and internal resource provider, you can choose to work with raw components with maximum flexibility, or opt-in for higher-level abstractions. Mix-and-match in the same program is possible too.

Infrastructure as Real Code

Designing applications for the modern cloud means utilizing multiple cloud services which have to be configured to play nicely together. The Infrastructure as Code approach is almost a requirement to keep the management of such applications reliable in a team setting and over the extended period.

Application code and supporting infrastructure become more and more blended, so it's natural that software developers take the responsibility to define both. The next logical step is to use the same set of languages, tooling, and practices for both software and infrastructure.

Pulumi exposes cloud resources as APIs in several popular general-purpose programming languages. Developers can directly transfer their skills and experience to define, build, compose, and deploy modern cloud-native and serverless applications more efficiently than ever.

Making Sense of Azure Durable Functions

Mikhail Shilkov — Wed, 09 Jan 2019 21:46:56 +0000

Stateful Workflows on top of Stateless Serverless Cloud Functions—this is the essence of the Azure Durable Functions library. That's a lot of fancy words in one sentence, and they might be hard for the majority of readers to understand.

Please join me on the journey where I'll try to explain how those buzzwords fit together. I will do this in 3 steps:

Describe the context of modern cloud applications relying on serverless architecture;
Identify the limitations of basic approaches to composing applications out of the simple building blocks;
Explain the solutions that Durable Functions offer for those problems.

Microservices

Traditionally, server-side applications were built in a style which is now referred to as Monolith. If multiple people and teams were developing parts of the same application, they mostly contributed to the same code base. If the code base were structured well, it would have some distinct modules or components, and a single team would typically own each module:

Multiple components of a monolithic application

Usually, the modules would be packaged together at build time and then deployed as a single unit, so a lot of communication between modules would stay inside the OS process.

Although the modules could stay loosely coupled over time, the coupling almost always occurred on the level of the data store because all teams would use a single centralized database.

This model works great for small- to medium-size applications, but it turns out that teams start getting in each other's way as the application grows since synchronization of contributions takes more and more effort.

As a complex but viable alternative, the industry came up with a revised service-oriented approach commonly called Microservices. The teams split the big application into "vertical slices" structured around the distinct business capabilities:

Multiple components of a microservice-based application

Each team then owns a whole vertical—from public communication contracts, or even UIs, down to the data storage. Explicitly shared databases are strongly discouraged. Services talk to each other via documented and versioned public contracts.

If the borders for the split were selected well—and that's the most tricky part—the contracts stay stable over time, and thin enough to avoid too much chattiness. This gives each team enough autonomy to innovate at their best pace and to make independent technical decisions.

One of the drawbacks of microservices is the change in deployment model. The services are now deployed to separate servers connected via a network:

Challenges of communication between distributed components

Networks are fundamentally unreliable: they work just fine most of the time, but when they fail, they fail in all kinds of unpredictable and least desirable manners. There are books written on the topic of distributed systems architecture. TL;DR: it's hard.

A lot of the new adopters of microservices tend to ignore such complications. REST over HTTP(S) is the dominant style of connecting microservices. Like any other synchronous communication protocol, it makes the system brittle.

Consider what happens when one service becomes temporary unhealthy: maybe its database goes offline, or it's struggling to keep up with the request load, or a new version of the service is being deployed. All the requests to the problematic service start failing—or worse—become very slow. The dependent service waits for the response, and thus blocks all incoming requests of its own. The error propagates upstream very quickly causing cascading failures all over the place:

Error in one component causes cascading failures

The application is down. Everybody screams and starts the blame war.

Event-Driven Applications

While cascading failures of HTTP communication can be mitigated with patterns like a circuit breaker and graceful degradation, a better solution is to switch to the asynchronous style of communication as the default. Some kind of persistent queueing service is used as an intermediary.

The style of application architecture which is based on sending events between services is known as Event-Driven. When a service does something useful, it publishes an event—a record about the fact which happened to its business domain. Another service listens to the published events and executes its own duty in response to those facts:

Communication in event-driven applications

The service that produces events might not know about the consumers. New event subscribers can be introduced over time. This works better in theory than in practice, but the services tend to get coupled less.

More importantly, if one service is down, other services don't catch fire immediately. The upstream services keep publishing the events, which build up in the queue but can be stored safely for hours or days. The downstream services might not be doing anything useful for this particular flow, but it can stay healthy otherwise.

However, another potential issue comes hand-in-hand with loose coupling: low cohesion. As Martin Fowler notices in his essay
What do you mean by "Event-Driven":

It's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of the larger-scale flow.

Given many components that publish and subscribe to a large number of event types, it's easy to stop seeing the forest for the trees. Combinations of events usually constitute gradual workflows executed in time. A workflow is more than the sum of its parts, and understanding of the high-level flow is paramount to controlling the system behavior.

Hold this thought for a minute; we'll get back to it later. Now it's time to talk cloud.

Cloud

The birth of public cloud changed the way we architect applications. It made many things much more straightforward: provisioning of new resources in minutes instead of months, scaling elastically based on demand, and resiliency and disaster recovery at the global scale.

It made other things more complicated. Here is the picture of the global Azure network:

Azure locations with network connections

There are good reasons to deploy applications to more than one geographical location: among others, to reduce network latency by staying close to the customer, and to achieve resilience through geographical redundancy. Public Cloud is the ultimate distributed system. As you remember, distributed systems are hard.

There's more to that. Each cloud provider has dozens and dozens of managed services, which is the curse and the blessing. Specialized services are great to provide off-the-shelf solutions to common complex problems. On the flip side, each service has distinct properties regarding consistency, resiliency and fault tolerance.

In my opinion, at this point developers have to embrace the public cloud and apply the distributed system design on top of it. If you agree, there is an excellent way to approach it.

Serverless

The slightly provocative term serverless is used to describe cloud services that do not require provisioning of VMs, instances, workers, or any other fixed capacity to run custom applications on top of them. Resources are allocated dynamically and transparently, and the cost is based on their actual consumption, rather than on pre-purchased capacity.

Serverless is more about operational and economical properties of the system than about the technology per se. Servers do exist, but they are someone else's concern. You don't manage the uptime of serverless applications: the cloud provider does.

On top of that, you pay for what you use, similar to the consumption of other commodity resources like electricity. Instead of buying a generator to power up your house, you just purchase energy from the power company. You lose some control (e.g., no way to select the voltage), but this is fine in most cases. The great benefit is no need to buy and maintain the hardware.

Serverless compute does the same: it supplies standard services on a pay-per-use basis.

If we talk more specifically about Function-as-a-Service offerings like Azure Functions, they provide a standard model to run small pieces of code in the cloud. You zip up the code or binaries and send it to Azure; Microsoft takes care of all the hardware and software required to run it. The infrastructure automatically scales up or down based on demand, and you pay per request, CPU time and memory that the application consumed. No usage—no bill.

However, there's always a "but". FaaS services come with an opinionated development model that applications have to follow:

Event-Driven: for each serverless function you have to define a specific trigger—the event type which causes it to run, be it an HTTP endpoint or a queue message;
Short-Lived: functions can only run up to several minutes, and preferably for a few seconds or less;
Stateless: as you don't control where and when function instances are provisioned or deprovisioned, there is no way to store data within the process between requests reliably; external storage has to be utilized.

Frankly speaking, the majority of existing applications don't really fit into this model. If you are lucky to work on a new application (or a new module of it), you are in better shape.

A lot of the serverless applications may be designed to look somewhat similar to this example from the Serverless360 blog:

Sample application utilizing "serviceful" serverless architecture

There are 9 managed Azure services working together in this app. Most of them have a unique purpose, but the services are all glued together with Azure Functions. An image is uploaded to Blob Storage, an Azure Function calls Vision API to recognize the license plate and send the result to Event Grid, another Azure Function puts that event to Cosmos DB, and so on.

This style of cloud applications is sometimes referred to as Serviceful to emphasize the heavy usage of managed services "glued" together by serverless functions.

Creating a comparable application without any managed services would be a much harder task, even more so, if the application has to run at scale. Moreover, there's no way to keep the pay-as-you-go pricing model in the self-service world.

The application pictured above is still pretty straightforward. The processes
in enterprise applications are often much more sophisticated.

Remember the quote from Martin Fowler about losing sight of the large-scale flow. That was true for microservices, but it's even more true for the "nanoservices" of cloud functions.

I want to dive deeper and give you several examples of related problems.

Challenges of Serverless Composition

For the rest of the article, I'll define an imaginary business application for booking trips to software conferences. In order to go to a conference, I need to buy tickets to the conference itself, purchase the flights, and book a room at a hotel.

In this scenario, it makes sense to create three Azure Functions, each one responsible for one step of the booking process. As we prefer message passing, each Function emits an event which the next function can listen for:

Conference booking application

This approach works, however, problems do exist.

Flexible Sequencing

As we need to execute the whole booking process in sequence, the Azure Functions are wired one after another by configuring the output of one function to match with the event source of the downstream function.

In the picture above, the functions' sequence is hard-defined. If we were to swap the order of booking the flights and reserving the hotel, that would require a code change—at least of the input/output wiring definitions, but probably also the functions' parameter types.

In this case, are the functions really decoupled?

Error Handling

What happens if the Book Flight function becomes unhealthy, perhaps due to the outage of the third-party flight-booking service? Well, that's why we use asynchronous messaging:after the function execution fails, the message returns to the queue and is picked up again by another execution.

However, such retries happen almost immediately for most event sources. This might not be what we want: an exponential back-off policy could be a smarter idea. At this point, the retry logic becomes stateful: the next attempt should "know" the history of previous attempts to make a decision about retry timing.

There are more advanced error-handling patterns too. If executions failures are not intermittent, we may decide to cancel the whole process and run compensating actions against the already completed steps.

An example of this is a fallback action: if the flight is not possible (e.g., no routes for this origin-destination combination), the flow could choose to book a train instead:

Fallback after 3 consecutive failures

This scenario is not trivial to implement with stateless functions. We could wait until a message goes to the dead-letter queue and then route it from there, but this is brittle and not expressive enough.

Parallel Actions

Sometimes the business process doesn't have to be sequential. In our reservation scenario, there might be no difference whether we book a flight before a hotel or vice versa. It could be desirable to run those actions in parallel.

Parallel execution of actions is easy with the pub-sub capabilities of an event bus: both functions should subscribe to the same event and act on it independently.

The problem comes when we need to reconcile the outcomes of parallel actions, e.g., calculate the final price for expense reporting purposes:

Fan-out / fan-in pattern

There is no way to implement the Report Expenses block as a single Azure Function: functions can't be triggered by two events, let alone correlate two related events.

The solution would probably include two functions, one per event, and the shared storage between them to pass information about the first completed booking to the one who completes last. All this wiring has to be implemented in custom code. The complexity grows if more than two functions need to run in parallel.

Also, don't forget the edge cases. What if one of the function fails? How do you make sure there is no race condition when writing and reading to/from the shared storage?

Missing Orchestrator

All these examples give us a hint that we need an additional tool to organize low-level single-purpose independent functions into high-level workflows.

Such a tool can be called an Orchestrator because its sole mission is to delegate work to stateless actions while maintaining the big picture and history of the flow.

Azure Durable Functions aims to provide such a tool.

Introducing Azure Durable Functions

Azure Functions

Azure Functions is the serverless compute service from Microsoft. Functions are event-driven: each function defines a trigger—the exact definition of the event source, for instance, the name of a storage queue.

Azure Functions can be programmed in several languages. A basic Function with a Storage Queue trigger implemented in C# would look like this:

[FunctionName("MyFirstFunction")]
public static void QueueTrigger(
    [QueueTrigger("myqueue-items")] string myQueueItem, 
    ILogger log)
{
    log.LogInformation($"C# function processed: {myQueueItem}");
}

The FunctionName attribute exposes the C# static method as an Azure Function named MyFirstFunction. The QueueTrigger attribute defines the name of the storage queue to listen to. The function body logs the information about the incoming message.

Durable Functions

Durable Functions is a library that brings workflow orchestration abstractions to Azure Functions. It introduces a number of idioms and tools to define stateful, potentially long-running operations, and manages a lot of mechanics of reliable communication and state management behind the scenes.

The library records the history of all actions in Azure Storage services, enabling durability and resilience to failures.

Durable Functions is open source, Microsoft accepts external contributions, and the community is quite active.

Currently, you can write Durable Functions in 3 programming languages: C#, F#, and Javascript (Node.js). All my examples are going to be in C#. For Javascript, check this quickstart and these samples. For F# see the samples, the F#-specific library and my article A Fairy Tale of F# and Durable Functions.

Workflow building functionality is achieved by the introduction of two additional types of triggers: Activity Functions and Orchestrator Functions.

Activity Functions

Activity Functions are simple stateless single-purpose building blocks that do just one task and have no awareness of the bigger workflow. A new trigger type, ActivityTrigger, was introduced to expose functions as workflow steps, as I explain below.

Here is a simple Activity Function implemented in C#:

[FunctionName("BookConference")]
public static ConfTicket BookConference([ActivityTrigger] string conference)
{
    var ticket = BookingService.Book(conference);
    return new ConfTicket { Code = ticket };
}

It has a common FunctionName attribute to expose the C# static method as an Azure Function named BookConference. The name is important because it is used to invoke the activity from orchestrators.

The ActivityTrigger attribute defines the trigger type and points to the input parameter conference which the activity expects to get for each invocation.

The function can return a result of any serializable type; my sample function returns a simple property bag called ConfTicket.

Activity Functions can do pretty much anything: call other services, load and save data from/to databases, and use any .NET libraries.

Orchestrator Functions

The Orchestrator Function is a unique concept introduced by Durable Functions. Its sole purpose is to manage the flow of execution and data among several activity functions.

Its most basic form chains multiple independent activities into a single
sequential workflow.

Let's start with an example which books a conference ticket, a flight itinerary, and a hotel room one-by-one:

3 steps of a workflow executed in sequence

The implementation of this workflow is defined by another C# Azure Function, this time with OrchestrationTrigger:

[FunctionName("SequentialWorkflow")]
public static async Task Sequential([OrchestrationTrigger] DurableOrchestrationContext context)
{
    var conference = await context.CallActivityAsync<ConfTicket>("BookConference", "ServerlessDays");
    var flight = await context.CallActivityAsync<FlightTickets>("BookFlight", conference.Dates);
    await context.CallActivityAsync("BookHotel", flight.Dates);
}

Again, attributes are used to describe the function for the Azure runtime.

The only input parameter has type DurableOrchestrationContext. This context is the tool that enables the orchestration operations.

In particular, the CallActivityAsync method is used three times to invoke three activities one after the other. The method body looks very typical for any C# code working with a Task-based API. However, the behavior is entirely different. Let's have a look at the implementation details.

Behind the Scenes

Let's walk through the lifecycle of one execution of the sequential workflow above.

When the orchestrator starts running, the first CallActivityAsync invocation is made to book the conference ticket. What actually happens here is that a queue message is sent from the orchestrator to the activity function.

The corresponding activity function gets triggered by the queue message. It does its job (books the ticket) and returns the result. The activity function serializes the result and sends it as a queue message back to the orchestrator:

Messaging between the orchestrator and the activity

When the message arrives, the orchestrator gets triggered again and can proceed to the secondactivity. The cycle repeats—a message gets sent to Book Flight activity, it gets triggered, does its job, and sends a message back to the orchestrator. The same message flow happens for the third call.

Stop-resume behavior

As discussed earlier, message passing is intended to decouple the sender and receiver in time. For every message in the scenario above, no immediate response is expected.

On the C# level, when the await operator is executed, the code doesn't block the execution of the whole orchestrator. Instead, it just quits: the orchestrator stops being active and its current step completes.

Whenever a return message arrives from an activity, the orchestrator code restarts. It always starts with the first line. Yes, this means that the same line is executed multiple times: up to the number of messages to the orchestrator.

However, the orchestrator stores the history of its past executions in Azure Storage, so the effect of the second pass of the first line is different: instead of sending a message to the activity it already knows the result of that activity, so await returns this result back and assigns it to the conference variable.

Because of these "replays", the orchestrator's implementation has to be deterministic: don't use DateTime.Now, random numbers or multi-thread operations; more details here.

Event Sourcing

Azure Functions are stateless, while workflows require a state to keep track of their progress. Every time a new action towards the workflow's execution happens, the framework automatically records an event in table storage.

Whenever an orchestrator restarts the execution because a new message arrives from its activity, it loads the complete history of this particular execution from storage. Durable Context uses this history to make decisions whether to call the activity or return the previously stored result.

The pattern of storing the complete history of state changes as an append-only event store is known as Event Sourcing. Event store provides several benefits:

Durability—if a host running an orchestration fails, the history is retained in persistent storage and is loaded by the new host where the orchestration restarts;
Scalability—append-only writes are fast and easy to spread over multiple storage servers;
Observability—no history is ever lost, so it's straightforward to inspect and analyze even after the workflow is complete.

Here is an illustration of the notable events that get recorded during our sequential workflow:

Log of events in the course of orchestrator progression

Billing

Azure Functions on the serverless consumption-based plan are billed per execution + per duration of execution.

The stop-replay behavior of durable orchestrators causes the single workflow "instance" to execute the same orchestrator function multiple times. This also means paying for several short executions.

However, the total bill usually ends up being much lower compared to the potential cost of blocking synchronous calls to activities. The price of 5 executions of 100 ms each is significantly lower than the cost of 1 execution of 30 seconds.

By the way, the first million executions per month are at no charge, so many scenarios incur no cost at all from Azure Functions service.

Another cost component to keep in mind is Azure Storage. Queues and Tables that are used behind the scenes are charged to the end customer. In my experience, this charge remains close to zero for low- to medium-load applications.

Beware of unintentional eternal loops or indefinite recursive fan-outs in your orchestrators. Those can get expensive if you leave them out of control.

Error-handling and retries

What happens when an error occurs somewhere in the middle of the workflow? For instance, a third-party flight booking service might not be able to process the request:

One activity is unhealthy

This situation is expected by Durable Functions. Instead of silently failing, the activity function sends a message containing the information about the error back to the orchestrator.

The orchestrator deserializes the error details and, at the time of replay, throws a .NET exception from the corresponding call. The developer is free to put a try .. catch block around the call and handle the exception:

[FunctionName("SequentialWorkflow")]
public static async Task Sequential([OrchestrationTrigger] DurableOrchestrationContext context)
{
    var conf = await context.CallActivityAsync<ConfTicket>("BookConference", "ServerlessDays");
    try
    {
        var itinerary = MakeItinerary(/* ... */);
        await context.CallActivityAsync("BookFlight", itinerary);
    }
    catch (FunctionFailedException)
    {
        var alternativeItinerary = MakeAnotherItinerary(/* ... */);
        await context.CallActivityAsync("BookFlight", alternativeItinerary);
    }
    await context.CallActivityAsync("BookHotel", flight.Dates);
}

The code above falls back to a "backup plan" of booking another itinerary. Another typical pattern would be to run a compensating activity to cancel the effects of any previous actions (un-book the conference in our case) and leave the system in a clean state.

Quite often, the error might be transient, so it might make sense to retry the failed operation after a pause. It's a such a common scenario that Durable Functions provides a dedicated API:

var options = new RetryOptions(
    firstRetryInterval: TimeSpan.FromMinutes(1),                    
    maxNumberOfAttempts: 5);
options.BackoffCoefficient = 2.0;

await context.CallActivityWithRetryAsync("BookFlight", options, itinerary);

The above code instructs the library to

Retry up to 5 times
Wait for 1 minute before the first retry
Increase delays before every subsequent retry by the factor of 2 (1 min, 2 min, 4 min, etc.)

The significant point is that, once again, the orchestrator does not block while awaiting retries. After a failed call, a message is scheduled for the moment in the future to re-run the orchestrator and retry the call.

Sub-orchestrators

Business processes may consist of numerous steps. To keep the code of orchestrators manageable, Durable Functions allows nested orchestrators. A "parent" orchestrator can call out to child orchestrators via the context.CallSubOrchestratorAsync method:

[FunctionName("CombinedOrchestrator")]
public static async Task CombinedOrchestrator([OrchestrationTrigger] DurableOrchestrationContext context)
{
    await context.CallSubOrchestratorAsync("BookTrip", serverlessDaysAmsterdam);
    await context.CallSubOrchestratorAsync("BookTrip", serverlessDaysHamburg);
}

The code above books two conferences, one after the other.

Fan-out / Fan-in

What if we want to run multiple activities in parallel?

For instance, in the example above, we could wish to book two conferences, but the booking order might not matter. Still, when both bookings are completed, we want to combine the results to produce an expense report for the finance department:

Parallel calls followed by a final step

In this scenario, the BookTrip orchestrator accepts an input parameter with the name of the conference and returns the expense information. ReportExpenses needs to receive both expenses combined.

This goal can be easily achieved by scheduling two tasks (i.e., sending two messages) without awaiting them separately. We use the familiar Task.WhenAll method to await both and combine the results:

[FunctionName("ParallelWorkflow")]
public static async Task Parallel([OrchestrationTrigger] DurableOrchestrationContext context)
{
    var amsterdam = context.CallSubOrchestratorAsync("BookTrip", serverlessDaysAmsterdam);
    var hamburg   = context.CallSubOrchestratorAsync("BookTrip", serverlessDaysHamburg);

    var expenses = await Task.WhenAll(amsterdam, hamburg);

    await context.CallActivityAsync("ReportExpenses", expenses);
}

Remember that awaiting the WhenAll method doesn't synchronously block the orchestrator. It quits the first time and then restarts two times on reply messages received from activities. The first restart quits again, and only the second restart makes it past the await.

Task.WhenAll returns an array of results (one result per each input task), which is then passed to the reporting activity.

Another example of parallelization could be a workflow sending e-mails to hundreds of recipients. Such fan-out wouldn't be hard with normal queue-triggered functions: simply send hundreds of messages. However, combining the results, if required for the next step of the workflow, is quite challenging.

It's straightforward with a durable orchestrator:

var emailSendingTasks =
    recepients
    .Select(to => context.CallActivityAsync<bool>("SendEmail", to))
    .ToArray();

var results = await Task.WhenAll(emailSendingTasks);

if (results.All(r => r)) { /* ... */ }

Making hundreds of roundtrips to activities and back could cause numerous replays of the orchestrator. As an optimization, if multiple activity functions complete around the same time, the orchestrator may internally process several messages as a batch and restart the orchestrator function only once per batch.

Other Concepts

There are many more patterns enabled by Durable Functions. Here is a quick list to give you some perspective:

Waiting for the first completed task in a collection (rather than all of them) using the Task.WhenAny method. Useful for scenarios like timeouts or competing actions.
Pausing the workflow for a given period or until a deadline.
Waiting for external events, e.g., bringing human interaction into the workflow.
Running recurring workflows, when the flow repeats until a certain condition is met.

Further explanation and code samples are in the docs.

Conclusion

I firmly believe that serverless applications utilizing a broad range of managed cloud services are highly beneficial to many companies, due to both rapid development process and the properly aligned billing model.

Serverless tech is still young; more high-level architectural patterns need to emerge to enable expressive and composable implementations of large business systems.

Azure Durable Functions suggests some of the possible answers. It combines the clarity and readability of sequential RPC-style code with the power and resilience of event-driven architecture.

The documentation for Durable Functions is excellent, with plenty of examples and how-to guides. Learn it, try it for your real-life scenarios, and let me know your opinion—I'm excited about the serverless future!

Acknowledgments

Many thanks to Katy Shimizu, Chris Gillum, Eric Fleming, KJ Jones, William Liebenberg, Andrea Tosato for reviewing the draft of this article and their valuable contributions and suggestions. The community around Azure Functions and Durable Functions is superb!