Forem: Marcos Garcia

How to build Azure Load Tests and Automate It via Pipelines

Marcos Garcia — Wed, 25 Jun 2025 05:56:43 +0000

Introduction

Load testing is essential when building scalable applications in the cloud. Azure Functions should scale up naturally, but how can you know if they handle spikes in demand? In this blog, we'll explore how to use Azure Load Testing to stress-test your Azure Functions and automate the process using CI/CD pipelines like Azure DevOps.

By the end of this guide, you'll know how to:

Prepare an Azure Function for load testing
Create a request to the Azure Function in Postman
Create and run a load test in Azure
Automate the load test in the pipeline
Set performance thresholds to fail builds

Use Case / Problem Statement

Imagine you're building an API using Azure Functions that will be called thousands of times per minute. How do you ensure your app can handle that load without failing or slowing down? Without proper load testing, performance regressions may go unnoticed until it's too late.

Prerequisites

Azure Subscription
Azure Load Testing resource
An Azure Function App with at least one HTTP-triggered function
Azure DevOps or GitHub repository with pipeline access
Apps: Code Editor of choice, Postman, JMeter

Step 1: Prepare The Azure Function for Testing

Make sure the Azure Function is deployed and has a public endpoint. Secure it using Function Keys or authentication as needed.
We will use Postman first to make sure there's proper access to the function and your IP is whitelisted.

Download Postman. Open your terminal/shell and run winget install Postman.Postman if you are on windows or if on Fedora/HREL, use the following commands: sudo dnf install snapd,sudo ln -s /var/lib/snapd/snap /snap,sudo snap install postman. Check snapcraft.io for more details.
Get the token ready. An app registration will be needed for this step. I went through it in more detail in this dev.to post

Make sure to provide the correct values in the request body. Replace the Tenant id, Client id, and Client secret according to your App Registration, but scope must reflect the Azure Function name (see in the screenshot below).

Optional: add a post response script to Postman to avoid token copy and paste:
```
const jsonData = pm.response.json();

if (jsonData.access_token) {
    pm.globals.set("AzureToken", jsonData.access_token);
}
```
Build a request to your Azure Function.

Make sure to add the Bearer token manually to the headers or in the Authorization tab, and to use the correct URL, which should be the "Default domain" from the Function overview page, and the chosen route, in my case it's defined in the Function code:
If this request succeeds, proceed to the Load Test creation, otherwise, I suggest troubleshooting it first. I'd take a close look at the permissions granted to the App Registration, managed identities, RBAC, Authentication Settings, and network.

If the Function is configured with "Restricted Access" in the authentication settings, find your public IP address and add to the white-listed IPs at Networking > Public network access "Enabled with access restrictions".

Step 2: Create a Load Test in Azure Load Testing

Create a Load Testing resource in Azure.
Install JMeter. If using windows install Apache JMeter using winget install DEVCOM.JMeter and on Fedora/Linux or another HREL distro, dnf install jmeter.
To open the application, simply run jmeter in your shell/terminal of choice. This command may vary depending on the distro/OS.

I created a template to get started. The file can be overwhelming, but contains as many features I find useful for the next step.

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.6.3">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Function‑API Load Test Template">
      <boolProp name="TestPlan.serialize_threadgroups">true</boolProp>
      <boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments">
        <collectionProp name="Arguments.arguments">
          <elementProp name="function_host" elementType="Argument">
            <stringProp name="Argument.name">function_host</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;FUNCTION_HOST&apos;) ?: &apos;your‑function.azurewebsites.net&apos;)}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
          <elementProp name="tenant_id" elementType="Argument">
            <stringProp name="Argument.name">tenant_id</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;AAD_TENANT_ID&apos;) ?: __GetSecret(&apos;aadTenantId&apos;))}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
          <elementProp name="client_id" elementType="Argument">
            <stringProp name="Argument.name">client_id</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;AAD_CLIENT_ID&apos;) ?: __GetSecret(&apos;aadClientId&apos;))}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
          <elementProp name="client_secret" elementType="Argument">
            <stringProp name="Argument.name">client_secret</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;AAD_CLIENT_SECRET&apos;) ?: __GetSecret(&apos;aadClientSecret&apos;))}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
          <elementProp name="aad_scope" elementType="Argument">
            <stringProp name="Argument.name">aad_scope</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;AAD_SCOPE&apos;) ?: &apos;api://YOUR‑APP‑ID/.default&apos;)}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
          <elementProp name="NumberOfThreads" elementType="Argument">
            <stringProp name="Argument.name">NumberOfThreads</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;THREADS&apos;) ?: &apos;50&apos;)}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
          <elementProp name="RampUpInSeconds" elementType="Argument">
            <stringProp name="Argument.name">RampUpInSeconds</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;RAMP_UP&apos;) ?: &apos;30&apos;)}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
          <elementProp name="DurationInSeconds" elementType="Argument">
            <stringProp name="Argument.name">DurationInSeconds</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;DURATION&apos;) ?: &apos;60&apos;)}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
          <elementProp name="userid_fetch_path" elementType="Argument">
            <stringProp name="Argument.name">userid_fetch_path</stringProp>
            <stringProp name="Argument.value">${__groovy(System.getenv(&apos;PATH_FETCH&apos;) ?: &apos;api/org/userid/fetch&apos;)}</stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
        </collectionProp>
      </elementProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="AAD Token Thread Group">
        <intProp name="ThreadGroup.num_threads">1</intProp>
        <intProp name="ThreadGroup.ramp_time">1</intProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller">
          <stringProp name="LoopController.loops">1</stringProp>
          <boolProp name="LoopController.continue_forever">false</boolProp>
        </elementProp>
      </ThreadGroup>
      <hashTree>
        <JSR223Sampler guiclass="TestBeanGUI" testclass="JSR223Sampler" testname="Acquire AAD Token">
          <stringProp name="scriptLanguage">groovy</stringProp>
          <stringProp name="script">
import org.apache.http.client.methods.HttpPost
import org.apache.http.impl.client.HttpClients
import org.apache.http.entity.StringEntity
import org.apache.http.util.EntityUtils
import groovy.json.JsonSlurper

def client = HttpClients.createDefault()

def tokenUrl = &quot;https://login.microsoftonline.com/${vars.get(&apos;tenant_id&apos;)}/oauth2/v2.0/token&quot;

def post = new HttpPost(tokenUrl)
post.setHeader(&apos;Content-Type&apos;, &apos;application/x-www-form-urlencoded&apos;)

def body = [
    grant_type: &apos;client_credentials&apos;,
    client_id : vars.get(&apos;client_id&apos;),
    client_secret: vars.get(&apos;client_secret&apos;),
    scope: vars.get(&apos;aad_scope&apos;)
].collect { k, v -&gt; &quot;${k}=${URLEncoder.encode(v, &apos;UTF-8&apos;)}&quot; }.join(&apos;&amp;&apos;)

post.setEntity(new StringEntity(body))

def response = client.execute(post)

def json = new JsonSlurper().parseText(EntityUtils.toString(response.entity))

props.put(&apos;access_token&apos;, json.access_token)
log.info(&apos;AAD access token fetched.&apos;)
</stringProp>
          <stringProp name="cacheKey">false</stringProp>
          <stringProp name="parameters"></stringProp>
          <stringProp name="filename"></stringProp>
        </JSR223Sampler>
        <hashTree/>
      </hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="API Main Thread Group">
        <stringProp name="ThreadGroup.num_threads">${NumberOfThreads}</stringProp>
        <stringProp name="ThreadGroup.ramp_time">${RampUpInSeconds}</stringProp>
        <stringProp name="ThreadGroup.duration">${DurationInSeconds}</stringProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
        <boolProp name="ThreadGroup.scheduler">true</boolProp>
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller">
          <intProp name="LoopController.loops">-1</intProp>
          <boolProp name="LoopController.continue_forever">false</boolProp>
        </elementProp>
      </ThreadGroup>
      <hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Fetch Record Id">
          <stringProp name="HTTPSampler.domain">${function_host}</stringProp>
          <stringProp name="HTTPSampler.protocol">https</stringProp>
          <stringProp name="HTTPSampler.path">${user_fetch_path}</stringProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments">
              <elementProp name="" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.value">{&quot;firstName&quot;:&quot;John&quot;,&quot;lastName&quot;:&quot;Doe&quot;,&quot;email&quot;:&quot;sample@test.com&quot;,&quot;dateOfBirth&quot;:&quot;1980‑09‑01T00:00:00Z&quot;}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="Bearer Header #1">
            <collectionProp name="HeaderManager.headers">
              <elementProp name="Auth" elementType="Header">
                <stringProp name="Header.name">Authorization</stringProp>
                <stringProp name="Header.value">Bearer ${__P(access_token)}</stringProp>
              </elementProp>
            </collectionProp>
          </HeaderManager>
          <hashTree/>
        </hashTree>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

Open the .jmx file in JMeter, there should be test structure like this:

Note that all the variables are in the topmost section, the Test Plan definition. To testing locally, replace the values for the token to look just like Postman, and hit "Start".

Quick explanation for the variables section:
- ${__groovy(...)}: return the value evaluated by the Groovy code
- ${__GetSecret(...)}: return the secret value defined in the config file.
- ${__P(access_token)}: return the value of a JMeter property
- props.put("access_token", json.access_token): set value to a JMeter property
- System.getenv('FUNCTION_HOST'): Fetches the value of the environment variable
- ?:: Ternary/Elvis Operator
Before proceeding to the pipeline you should upload it to the Azure Load Test resource and configure it.

Open the resource
Go to Tests > Create > Create test:
Once created open the Test plan tab and upload the file.
To authenticate the Load test, go back to the Overview page, then access Settings > Identity, and choose the desired method (User assigned recommended). Back in the Test plan there should be the selected option at the bottom.
In the Parameters tab set the environment variables and secrets:

Make sure the Key Vault is referenced at the bottom of the page.

The rest of it should be already configured, but for good measure it's recommended to reference the Managed identity once again in the Monitoring tab and to add a vnet configuration if the Function is on a private setting.
Click "Apply", access the test and hit "Run". Configure the variables however you'd like and describe it in the Test run description. I use this pattern: "NumberOfThreads 250 RampUpInSeconds 15 DurationInSeconds 600 EngineInstances 1" just to quickly identify each run. Please note that Debug mode runs on a different setting. Try to find a number that start showing a small percentage of errors and identify the performance thresholds of your Azure Function for benchmarking:

Download the result files from the Ellipsis (⋯) in the test results page for further troubleshooting.

Step 4: Automate It in The Pipeline

Use Azure DevOps (ADO) or GitHub Actions to trigger the load test on pull requests or deployments. I'm going to be using ADO for this but any should work.

Two .yml files are needed to automate it. A Load Test config and a pipeline yml file, they reference each other and should be added to the repository, just like the jmx file. If you are not used to pipelines I recommend going through the basics and ADO Pipeline or Github Actions documentation, as you might want to change these files according to your company's needs.

For the configuration file, it can be downloaded from the results of the previous test run. Access the Test runs and click the Ellipsis (⋯), then "download input file".

The config file should be ready to be used, but I'm adding one here in case as an example:

version: v0.1
testId: processor-loadtest
displayName: Processor Load Test
description: Load test for the Processor Azure Function
testType: JMX
testPlan: Processor-LoadTest.jmx
engineInstances: 2
subnetId: /subscriptions/id/resourceGroups/dev-network-rg/providers/Microsoft.Network/virtualNetworks/vnet-functions/subnets/snet-azurevirtualmachines
publicIPDisabled: true
splitAllCSVs: False
failureCriteria:
  clientMetrics:
  - avg(response_time_ms) > 60000
  - percentage(error) > 5
autoStop:
  errorPercentage: 80
  timeWindow: 60
  maximumVirtualUsersPerEngine: 5000
env:
  - name: NumberOfThreads
    value: 50
  - name: RampUpInSeconds
    value: 30
  - name: DurationInSeconds
    value: 60
secrets:
  - name: tokenAuthClientSecret
    value: https://dev-kv.vault.azure.net/secrets/tokenAuthClientSecret/id
  - name: tokenAuthClientId
    value: https://dev-kv.vault.azure.net/secrets/tokenAuthClientId/id
  - name: tokenAuthTenantId
    value: https://dev-kv.vault.azure.net/secrets/tokenAuthTenantId/id
referenceIdentities:
- kind: KeyVault
  type: UserAssigned
  value: /subscriptions/id/resourceGroups/dev-rg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/fndev_processor
- kind: Metrics
  type: SystemAssigned

The pipeline file can also be created with the help of ADO/Github, but I'm adding it as another example of what it should look like:

trigger: none   # Disable automatic CI trigger
pr: none        # Disable PR trigger

name: ManualLoadTestPipeline

stages:
  - stage: LoadTesting
    displayName: 'Start Azure Load Test'
    jobs:
      - job: RunAndPublishLoadTest
        displayName: 'Run and Publish Load Test Results'
        timeoutInMinutes: 15
        steps:

          - task: AzureLoadTest@1
            timeoutInMinutes: 10
            displayName: 'Run Load Test'
            inputs:
              azureSubscription: $(serviceConnection)
              loadTestConfigFile: 'Processor/LoadTests/Dev-Processor-LoadTestConfig.yml'
              loadTestResource: 'dev-load-testing'
              resourceGroup: 'dev-rg'
            condition: succeeded()

          - publish: Processor/LoadTests
            artifact: loadTestResults

          - task: PublishTestResults@2
            displayName: 'Publish Load Test Results'
            inputs:
              testResultsFormat: 'JUnit'
              testResultsFiles: 'Processor/LoadTests/results/*.xml'
              searchFolder: '$(System.DefaultWorkingDirectory)'
              mergeTestResults: true

Don't forget to add a Service Connection and all necessary condiguration to connect ADO and Azure. Azure DevOps itself doesn’t natively support pipeline auth via managed identity for Azure Load Testing, but that could change in the future.

After reviewing the .yml files and replacing the placeholders with the correct values based on your folder structure and Azure configuration, install the Azure Load Test extension in ADO. After that, it should be found in here: https://dev.azure.com/<your-org>/_settings/extensions.

At this point, the pipeline is ready to run. After that, the results will be available for inspection in Azure. Look for a new test in the Load Test resource with the same name as the "displayName" from the config file. If you configured a results folder in the repo, they should also be published after each run.

Final considerations: With this pipeline working and the test running as expected, consider adding it to your development lifecycle. This could run in your pipelines after successful builds on dev, after all the unit tests pass, before deploying from dev to QA or UAT, etc. There are many possible applications for load tests, including SQL queries test directly to a database with JDBC, Azure Kubernetes Service, and Microservices in general, and for the small effort to set it up there's a big value in return.

Best Practices & Considerations

Don't load test production unless necessary
Use deployment slots or staging environments
Account for cold starts in consumption plans
Monitor backend services if the Function triggers other processes

Troubleshooting

Common issues:

Subnet permissions errors (e.g., join/action errors)
Rate limiting (HTTP 429)
Authentication failures (HTTP 403)
Configuration issue (usually HTTP 500)

Solutions:

Ensure correct role assignments and firewall rules
Add retry policies
Use Application Insights for deeper diagnostics

Conclusion

Azure Load tests ensures that serverless applications scale up gracefully. Integrating it into a CI/CD pipeline identifies regression early and gives the team confidence during production deployments.

Moving Sharepoint access from Dynamics 365 OnPrem to Cloud with Graph API

Marcos Garcia — Sat, 18 Jan 2025 23:45:20 +0000

I recently stumbled on a bug I completely underestimated, and lived to tell the story. It involved several unexpected issues, like version incompatibility, broken builds, technical debt, and all the classic problems legacy code brings.
I gained valuable insights along the way. Not only about the code but also dealing with clients and tight deadlines.
Let me take you through this journey tracking down this bug, and the lessons I learned.

Background

My problem started with the bug title, "Online Termination of employee not working". I completely misjudged the situation from the start. There were other bugs I had to fix for this client with very similar titles. Most issues were fixed by tracking down what is trying to access a null value or over-complicated logic that was falling on edge cases.

For the record, this client has been migrating their Dynamics 365 from OnPrem to the Cloud environment for 3 years, lost most of their staff in the process and was planning to go live just 1 week after that bug was assigned to me. I guess no one ever bothered to QA this, or it wasn't a core feature (but it was).

Diving into the code, I realized they were using Sharepoint to store termination letters and many other documents. I noticed that the code would fail at the ClientContext, part of the Microsoft.SharePoint.Client package.
Simple fix, I thought. Find out why it fails and close the bug. Regrettably, Microsoft decided to deprecate parts of the Sharepoint Client API.
Here's a quick peek at the OnPrem code:

private SharePointDocumentLocation CreateSharepointFolderForApplication(IOrganizationService service, org_application application, Guid applicationId)
{
    SharePointDocumentLocation spDocLoc = new SharePointDocumentLocation
    {
        Name = "Documents on org 1",
        ParentSiteOrLocation = new EntityReference(SharePointDocumentLocation.EntityLogicalName, Guid.Parse("00000000-0000-0000-0000-000000000000")),
        RelativeUrl = application.org_name + "_" + applicationId.ToString(),
        RegardingObjectId = new EntityReference(org_application.EntityLogicalName, applicationId)
    };
    Guid _spDocLocId = service.Create(spDocLoc);

    SecureString securePassword = new SecureString();
    char[] arrPassword = SharePointPWD.ToCharArray();
    foreach (char c in arrPassword)
    {
        securePassword.AppendChar(c);
    }
    ClientContext context = new ClientContext("https://sharepoint.org.com/org/")
    {
        Credentials = new NetworkCredential("s_mywebdoc", securePassword)
    };
    Web web = context.Web;
    context.Load(web);
    CreteFolder(context, "Application", application.org_name + "_" + applicationId.ToString());
    xLog.AddLog("Done creating sharepoint folder", 1);

    return spDocLoc;
}

At this point I was starting to realize it would not be that simple, but I was still in denial.

Since ClientContext wasn't an option anymore, or anything from Microsoft.SharePoint.Client for that matter, I had to find another package that would replace it.

I quickly found out about Microsoft Graph client and started to like it. Again, I thought, "simple fix". Replace the deprecated client with the new and close the bug.

I could not fathom my naivety. One does not simply introduce new packages to legacy code without paying the price.

To venture into legacy code is to tread lightly, for even the smallest change may awaken the wrath of unforeseen version incompatibility. -Tolkien J.R.R

Since the birth of .NET Core 1.0 in 2016, Microsoft has continuously emphasized its compatibility but has not figured out a way to allow Dynamics 365 plugins to be written using any version other than 4.6.2 (or 4.8.1 for the daring).

Long story short, there were conflicts with some packages, like CrmSdk.CoreAssemblies and CrmSdk.Workflow. And when everything seemed to work, either the build would not complete without errors or the dll could not be deployed to Dataverse. Trying to debug it was a huge waste of time. The use of ILMerge probably caused some of the issues, but the outcome would be the same. And of course I wasn't going to introduce Package Assemblies to this legacy code base one week away from go live (as if I didn't make a similar mistake before).

Scraping the whole idea of adding another package to the project, I did a quick git reset --hard HEAD and decided to go with the good ol' System.Net.Http.HttpClient. And there's no other way to start other than getting that sweet auth token and start playing with the API on Postman.

After going through the trouble of setting up an App Registration in Azure for Sharepoint, and start hitting the endpoint with unexpected responses, I noticed the deprecated feature also apply in this case.
Fortunately, I could just add the Graph API permissions and change my request. I'll add all the relevant requests down below:

The How To Guide

Set up an App Registration:

You probably don't need to add all of those. For a more step by step guide I recommend following this blog post.

Get your Auth Token. You should be ready to start making your requests now:

After getting all your variables in place, I recommend setting another variable for the token and adding this script. Use {{token}} for simplicity, but you may need a more specific name.

// Parse the JSON response body
const jsonData = pm.response.json();

if (jsonData.access_token) {
    pm.globals.set("token", jsonData.access_token);
}

Create a request. Once you got a token, create a new request and add it to your header value, while the key will be Authorization:

Notice the URL is different from the one we used to get the token. Use https://graph.microsoft.com/v1.0/ as base URL.
We are also using parameters to search for the sitename. You can replace that with your site of choice, or get all of them. Later, we will see how this is relevant for the code we are writing.

I don't have a script for you, but you might want to copy the id of the chosen SharePoint site and also add to a variable. I decided to use {{QA-SiteId}}.

Get the Drive id. By the way, I don't understand how these names make sense since Microsoft uses these terms interchangeably sometimes (site, drive, list, folder). Also I'm not a Sharepoint developer, so take my view of the names with a grain of salt.

What I recommend is getting all your Drive IDs first:
{{BaseUrl}}sites/{{QA-SiteId}}/drives/

Response:

Then copy the id from line 5 from the response and change/duplicate your request to search for the id every time. You might want to create another variable:

Having the specific ID also helps with request URL. You can make a bit shorter if you have the drive ID. It would look like this: {{BaseUrl}}drives/{{ApplicationsDriveId}}.

Get folder id. This is necessary for this implementation, but may not be for you.

We need the parent folder for that drive.

The folder ID should look like this:

Copy documents into another folder: Documentation for this can be found here.

What the code should look like

For this clients' architecture, the folder is created beforehand, then the contents are copied. So we start with CreateSharepointFolderForApplication and move to the rest of the implementation:

Create SharePoint folder:

private SharePointDocumentLocation CreateSharepointFolderForApplication(
            IOrganizationService service,
            Guid applicationId,
            string applicationNameId)
        {
            CrmServiceContext svcContext = new CrmServiceContext(service);

            string parentSite = svcContext.SharePointDocumentLocationSet
                .Where(site => site.RelativeUrl == "application")
                .Select(site => site.Id)
                .FirstOrDefault().ToString();

            SharePointDocumentLocation spDocLoc = new SharePointDocumentLocation
            {
                Name = "Documents",
                ParentSiteOrLocation = new EntityReference(SharePointDocumentLocation.EntityLogicalName, Guid.Parse(parentSite)),
                RelativeUrl = applicationNameId + "_" + applicationId.ToString(),
                RegardingObjectId = new EntityReference(application.EntityLogicalName, applicationId)
            };
            service.Create(spDocLoc);

            return spDocLoc;
        }

Get you access token: string accessToken = MyWebBusinessLogic.GetSharePointAccessToken(service).GetAwaiter().GetResult();

public static async Task<string> GetSharePointAccessToken(IOrganizationService service)
{
    var tenantId = GetEnvironmentVariable(service, "org_sharePointAppRegTenantId");
    var clientId = GetEnvironmentVariable(service, "org_sharePointAppRegClientId");
    var clientSecret = GetEnvironmentVariable(service, "org_sharePointAppRegClientSecret");
    string authority = $"https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/token";

    using (HttpClient client = new HttpClient())
    {
        var requestData = new FormUrlEncodedContent(new[]
        {
            new KeyValuePair<string, string>("grant_type", "client_credentials"),
            new KeyValuePair<string, string>("client_id", clientId),
            new KeyValuePair<string, string>("client_secret", clientSecret),
            new KeyValuePair<string, string>("scope", "https://graph.microsoft.com/.default")
        });
        HttpResponseMessage response = await client.PostAsync(authority, requestData);

        if (!response.IsSuccessStatusCode)
        {
            throw new HttpRequestException($"Error fetching token: {response.ReasonPhrase}");
        }

        string responseContent = await response.Content.ReadAsStringAsync();
        dynamic tokenData = JsonConvert.DeserializeObject(responseContent);
        return tokenData.access_token;
    }

}

The environment variable can be fetched this way:

public static string GetEnvironmentVariable(IOrganizationService service, string schemaName)
        {
            // Query the EnvironmentVariableDefinition entity by Schema Name
            QueryExpression query = new QueryExpression("environmentvariabledefinition");
            query.ColumnSet = new ColumnSet("environmentvariabledefinitionid");
            query.Criteria.AddCondition("schemaname", ConditionOperator.Equal, schemaName);

            var envVarDef = service.RetrieveMultiple(query).Entities.FirstOrDefault();

            if (envVarDef == null)
            {
                throw new InvalidOperationException($"Environment Variable '{schemaName}' not found.");
            }

            var envVarDefId = envVarDef.Id;

            // Query the EnvironmentVariableValue entity for the value
            QueryExpression valueQuery = new QueryExpression("environmentvariablevalue");
            valueQuery.ColumnSet = new ColumnSet("value");
            valueQuery.Criteria.AddCondition("environmentvariabledefinitionid", ConditionOperator.Equal, envVarDefId);

            var envVarValue = service.RetrieveMultiple(valueQuery).Entities.FirstOrDefault();

            return envVarValue?.GetAttributeValue<string>("value") ?? string.Empty;
        }

Get the folder ids

public static async Task<string> GetFolderIdAsync(string accessToken, string driveId, string folderName)
{
    using (var client = new HttpClient())
    {
        client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);

        var url = $"https://graph.microsoft.com/v1.0/drives/{driveId}/root:/{folderName}";
        var response = await client.GetAsync(url);

        if (response.IsSuccessStatusCode)
        {
            var jsonResponse = await response.Content.ReadAsStringAsync();

            using (JsonDocument document = JsonDocument.Parse(jsonResponse))
            {
                if (document.RootElement.TryGetProperty("id", out JsonElement idElement))
                {
                    return idElement.GetString(); //Return the ID
                }
                throw new InvalidOperationException("ID not found in the response.");
            }
        }
        else
        {
            throw new HttpRequestException($"Request failed with status code: {response.StatusCode} - {response.ReasonPhrase}");
        }
    }
}

public static async Task<string> GetParentFolderIdAsync(string accessToken, string driveId)
{
    using (var client = new HttpClient())
    {
        client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);

        var folderUrl = $"https://graph.microsoft.com/v1.0/drives/{driveId}/root/children?$top=1";
        var response = await client.GetAsync(folderUrl);

        if (response.IsSuccessStatusCode)
        {
            var jsonResponse = await response.Content.ReadAsStringAsync();

            using (JsonDocument document = JsonDocument.Parse(jsonResponse))
            {
                if (document.RootElement.TryGetProperty("value", out JsonElement valueArray) && valueArray.GetArrayLength() > 0)
                {
                    if (valueArray[0].TryGetProperty("parentReference", out JsonElement parentReference) &&
                        parentReference.TryGetProperty("id", out JsonElement parentId))
                    {
                        return parentId.GetString(); // Return the parent folder ID
                    }
                }

                throw new InvalidOperationException("Parent folder ID not found in the response.");
            }
        }
        else
        {
            throw new HttpRequestException($"Request failed with status code: {response.StatusCode} - {response.ReasonPhrase}");
        }
    }
}

Copy the folder content

public async Task<HttpResponseMessage> CopyFolderAsync(ITracingService tracingService, string accessToken, string srcFolderDriveId, string srcFolderId, string desFolderId, string desFolderName, string applicationDriveQAId)
{
    tracingService.Trace($"Source Folder ID: {srcFolderDriveId}, Destination Folder ID: {desFolderId}, Destination Folder Name: {desFolderName}");

    try
    {
        using (HttpClient client = new HttpClient())
        {
            tracingService.Trace("Setting Authorization header with access token.");
            client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);

            tracingService.Trace($"Preparing to copy source folder (ID: {srcFolderDriveId}) to destination folder (ID: {desFolderId}).");
            //E.g. srcFolderDriveId = b!3ek... , srcFolderName
            string copyUrl = $"https://graph.microsoft.com/v1.0/drives/{srcFolderDriveId}/items/{srcFolderId}/copy";

            var copyPayload = new JObject
            {
                { "parentReference", new JObject {
                    { "driveId", applicationDriveQAId }, 
                    { "id", desFolderId } } 
                },
                { "name", desFolderName }
            };

            var copyContent = new StringContent(copyPayload.ToString(), Encoding.UTF8, "application/json");

            tracingService.Trace($"Sending POST request to copy folder. URL: {copyUrl}");
            HttpResponseMessage copyResponse = await client.PostAsync(copyUrl, copyContent);

            if (!copyResponse.IsSuccessStatusCode)
            {
                string copyErrorMsg = $"Error copying folder: {copyResponse.StatusCode} - {copyResponse.ReasonPhrase}";
                tracingService.Trace(copyErrorMsg);
                throw new Exception(copyErrorMsg);
            }

            return copyResponse;
        }
    }
    catch (Exception ex)
    {
        tracingService.Trace($"An error occurred in CopyFolderAsync: {ex.Message}");
        throw;
    }
}

Now we are ready to put everything together:

HttpResponseMessage copyResponse = null;
try
{
    var myWebSubmissionDriveId = MyWebBusinessLogic.GetEnvironmentVariable(service, "org_myWebSubmissionDriveId");
    var applicationDriveId = MyWebBusinessLogic.GetEnvironmentVariable(service, "org_applicationDriveId");

    string srcFolderName = results.org_name;
    string desFolderName = response.GetAttributeValue<string>("RelativeUrl");

    string srcFolderId = MyWebBusinessLogic.GetFolderIdAsync(
        accessToken,
        myWebSubmissionDriveId,
        srcFolderName).GetAwaiter().GetResult();

    string desFolderId = MyWebBusinessLogic.GetParentFolderIdAsync(
        accessToken,
        applicationDriveId).GetAwaiter().GetResult();

    copyResponse = myWebLogic.CopyFolderAsync(tracingService,
        accessToken,
        myWebSubmissionDriveId,
        srcFolderId,
        desFolderId,
        desFolderName,
        applicationDriveId).GetAwaiter().GetResult();

}
catch (Exception spEx)
{
    xLog.AddLog("ProcessSubmission Sharepoint Exception: " + spEx.Message, 3);
}
finally
{
    tracingService.Trace($"Copy folder reponse: {JsonConvert.SerializeObject(copyResponse)}");

}

Final remarks

This code shows the implementation I came up with for my client's needs. Moving away from On-Prem to Cloud in Dynamics 365 can be a lengthy process, but tackling issues like hardcoded IDs, even for non-sensitive data, and tightly coupling is important. Going forward, I guarantee my client will have a much better pattern for the next developers to base their code on.

There's still room for improvement, like Dependency Injection for the HTTP Client and better error handling. But overall, I'm proud of the result. The goal was achieved with the minimum required effort while respecting the client's budget and also providing an elegant and easy-to-maintain solution.