Forem: Stas Sultanov

Serving RSA and ECDSA from One ASP.NET Core Kestrel Endpoint

Stas Sultanov — Mon, 30 Mar 2026 07:57:39 +0000

Most web servers are built for clients whose security behavior is handled by mainstream general-purpose operating systems.
In that world, TLS handling by the web server is usually straightforward: one server name, one endpoint, one certificate chain, and broad interoperability across the signature schemes those stacks support.

The situation is different when clients fall outside that mainstream.
In IoT, device fleets, industrial gateways, legacy SDKs, or application-to-application integrations, clients often have hard restrictions on which server authentication algorithms, certificate public key types, and certificate signature schemes they can use.

One important subset of those cases concerns the signature algorithms supported by the client.
Cryptographic algorithms evolved over time, and different generations of clients ended up with support for different sets of algorithms.
As a result, one client may support only RSA, another may support only ECDSA, and a third may support both.

When implementing a web server that must serve all those clients, this creates a specific requirement for the server.
During the TLS handshake, it must inspect the client's capabilities and present a certificate with a signature algorithm that the client supports.

This article is about how to handle that requirement directly in ASP.NET Core Kestrel, without moving certificate management into a reverse proxy, external gateway, or other edge tier.

Note: ASP.NET Core implementation, and most of the information available on this topic, focus on SNI-based certificate selection, where different server names map to different certificates, which is not the case here.
Because of that, most people and LLM-based AI assistants conclude that this scenario is impossible to implement in ASP.NET Core Kestrel.

Note: The reader should have a deep understanding of TLS, especially the negotiation phase of the handshake.

Visualization

The visualization below illustrates the exact case addressed in this article.
The backend exposes one endpoint and clients have different capabilities.

Open in mermaid.live

The Common Approach

This case is not unique and occurs frequently in some areas.
It is often addressed by offloading TLS certificate management to a dedicated edge service placed in front of the web server.
More specifically, that edge service is often a reverse proxy such as NGINX or HAProxy, a managed edge gateway such as Azure Application Gateway, or a Kubernetes Gateway implementation.

While this approach certainly works, it also brings drawbacks, such as:

one more network hop,
one more system to provision,
one more failure domain,
one more place where TLS configuration can drift,
one more operational surface that costs money and must be managed.

In mutual TLS scenarios, this is even less attractive.
Once certificate negotiation and identity extraction move to an edge service, authentication logic gets split across services.
The edge service now owns part of the security model, while the application owns another part.
This is especially problematic in regulated or protocol-driven environments that require strict end-to-end client authentication and certificate binding at the application layer.
In zero-trust environments, it may also be unacceptable to leave traffic unencrypted between the edge service and the web server.

Visualization

Open in mermaid.live

The Optimal Approach

For the specific case discussed in this article, the optimal design is to keep certificate selection inside the application host.

That produces a much cleaner model:

There is no extra hop and no extra failure point.
TLS management stays with the host that actually owns the endpoint.
Certificate selection is implemented exactly where the handshake happens.
In mutual TLS (mTLS) scenarios, client certificate negotiation and application-side identity handling are in one place.

Visualization

Open in mermaid.live

How Certificate Selection Works During TLS

Before moving forward, it is important to clarify how server-side certificate selection works.

This process must occur during the initial phase of the TLS handshake: after the client sends ClientHello and before the server responds with ServerHello.

The server-side flow is as follows:

Receive the incoming TLS record.
Parse the record as TLSPlaintext and verify that it contains a Handshake message with a ClientHello body.
Extract the client capabilities relevant to certificate selection from ClientHello.
Select the appropriate certificate based on those capabilities and your server’s selection policy.
Respond with ServerHello and continue handshake, including Certificate message.

How to get client capabilities

To determine which certificates are compatible, the ClientHello message must be inspected.

The exact location of the information needed for certificate selection depends on the TLS version:

Version	Primary Source	Secondary Source
1.2	`signature_algorithms` extension	`cipher_suites` field
1.3	`signature_algorithms_cert` extension	`signature_algorithms` extension

How the Server Chooses a Certificate

The actual certificate to present is chosen by considering both the client’s capabilities and the web server’s certificate selection policy.

The specific selection policy is determined by the server implementation and may depend on organizational requirements or security policies.

Typical strategies include:

Prefer the certificate with the strongest algorithm supported by both server and client.
Present a default certificate if it matches the client’s capabilities.
If no compatible certificate is available, abort the handshake.

This is the core mechanism for dynamic certificate selection based on client capabilities.

Implement using ASP.NET Core Kestrel

Since ASP.NET Core 2.1, Kestrel provides the ability to configure TLS handshake behavior via HttpsConnectionAdapterOptions.

The HttpsConnectionAdapterOptions provides a ServerCertificateSelector property that allows configuration of a callback that is called during the TLS negotiation phase.

Historically, this API was designed for SNI-based certificate selection, where the server name influences which certificate is returned.
However, nothing prevents using a different certificate-selection logic, such as client capabilities provided in ClientHello and implementation-specific policies.

The method assigned to ServerCertificateSelector accepts an argument of type ConnectionContext.

The ConnectionContext implements the IMemoryPoolFeature interface, which exposes a MemoryPool property.

The MemoryPool property can be used to access the raw TLS records as bytes sent by the client during the handshake. The relevant TLS record is a TLSPlaintext struct that carries a Handshake message with a ClientHello body. This enables custom parsing of handshake data if needed.

Starting with ASP.NET Core 10.0, HttpsConnectionAdapterOptions also provides the TlsClientHelloBytesCallback property.
This callback enables inspection of the incoming ClientHello before the certificate selection callback is invoked.

These APIs provide all the necessary hooks to implement dynamic certificate selection in Kestrel based on client capabilities, not just SNI.

Reference Implementation

Everything required to implement the optimal approach described in this article has already been implemented by me in the following GitHub repository:

tls-server-certificate-selection

This is not a toy demo.
The repository contains production-grade code for parsing ClientHello and extracting the data required for certificate selection.

It also contains tests that can be used to study the behavior in detail, including integration tests that show how to wire the mechanism into ASP.NET Core Kestrel.

More specifically, the repository provides:

low-level parsing of ClientHello,
extraction of the TLS data relevant to certificate selection,
a reusable implementation that can be inspected independently from the article,
Kestrel-based integration tests that demonstrate end-to-end certificate selection behavior.

The repository shows the implementation in full, with tests that make the behavior easy to verify and explore.

Conclusion

If you have one logical endpoint and heterogeneous non-browser clients, deploying an extra TLS tier should not be your default response.

When the requirement is simply:

same host,
same application,
different certificate algorithms per client capability,

Kestrel can solve it where the problem actually lives: inside the server during TLS negotiation.

If you found this article useful, feel free to buy the author a cup of coffee ☕.

[Boost]

Stas Sultanov — Tue, 09 Dec 2025 20:31:49 +0000

Why Every SaaS Company Needs a Solutions Architect

Stas Sultanov ・ Oct 28

#architecture #career #saas

Why Every SaaS Company Needs a Solutions Architect

Stas Sultanov — Tue, 28 Oct 2025 16:00:00 +0000

A SaaS company’s business model depends entirely on the IT solution it builds and operates. If the solution is well-made, it supports the business and enables growth. If it’s made poorly, the business starts to struggle.

That struggle shows up in familiar ways:

outages, overloads, and peak‑time failures that erode customer trust
runaway infrastructure bills that eat margin
slow evolution that delays getting new functionality to market

This isn’t bad luck. It’s a structural issue that won’t fix itself.

What causes the problem?

The root cause is a stream of suboptimal, often simply bad, decisions made through the lifecycle of the IT solution.

Under business pressure or through lack of experience, teams pick the quickest path. It buys a short‑term win and hides a long‑term cost. People act tactically; no one holds the strategic line.

In the moment, choices look reasonable: unblock a deal, hit a date, calm an incident. Each one quietly adds long‑term risk that surfaces later. The consequences arrive as outages, rising run costs, and slower change — long after the original decision is forgotten.

From the outside, the fix seems obvious: “Just start making optimal decisions”.
In reality, it isn’t that simple.

Why is making optimal decisions hard?

In a nutshell, every meaningful decision within the lifecycle of a SaaS product is a trade‑off across four forces:

Business
What must be achieved to win the deal, protect margin, satisfy contractual promises, and keep revenue flowing.
Domain
What must be respected by the product — processes, rules, compliance obligations, regulatory constraints, and industry expectations.
Software
What can be built, tested, and maintained by the current teams and tech stack.
Infrastructure
What can support the solution and host the software at scale — with the required capacity, resilience, security, performance, and predictable run cost.

These four forces naturally form two sides of a single scale:

Demand = Business + Domain
Demand is what the business wants and what must be respected.
Capability = Software + Infrastructure
Capability is what the engineers can actually build and run.

Every decision within a SaaS product’s lifecycle must keep Demand and Capability in balance. That’s inherently hard — and in most SaaS companies, no one is explicitly accountable for that balance.

Why does no one own the balance?

Even though every decision must balance Demand and Capability, inside most SaaS companies there is no shared understanding of the four forces that define that balance — Business, Domain, Software, and Infrastructure. Because these forces aren’t explicitly named or understood, the need to balance them is invisible. As a result, nobody is assigned to own it.

Instead, each group defends its own priority:

Leadership pushes new ideas and commercial commitments.
Sales optimizes for contract wins.
Product management optimizes for functionality and deadlines.
Engineering optimizes for shipping code.
Operations optimizes for stability and keeping production up.

Individually, each of these priorities is rational. Together, they pull the solution apart.

This is the structural gap.

The job of holding Business, Domain, Software, and Infrastructure in balance must belong to a dedicated role.

That role is the Solutions Architect.

What does a Solutions Architect actually do?

A real Solutions Architect (SA) is the business safeguard against self‑inflicted damage.

The SA’s responsibility is to translate business intent into something that the company can actually build, run, and afford — and then defend that shape against erosion.

In practical terms, an effective SA does the following:

Confirms that what is being promised to a customer can in fact be delivered without destroying future roadmap or operational stability.
States clearly when a shortcut creates a future liability (technical, contractual, compliance, or cost).
Describes the viable path: “Yes, this is possible — here is how it must be done so it won’t explode six months from now”.
Forces explicit trade‑offs and documented decisions instead of silent debt.

The SA keeps Demand and Capability in balance.

The SA provides margin protection, reputational protection, and delivery‑risk control.

How does the SA do this in practice?

To keep a SaaS product running, a solution architect must continually translate business intent into real, tangible value that integrates with what already exists and reliably operates in production—without unexpected costs, disruptions, or deviations from the roadmap.

The SA does this through an ongoing process that looks like this:

Capture Business intent: understand revenue impact, contractual promises, deadlines, constraints, and success criteria.
Anchor it in the Domain: map real workflows, compliance, obligations, and rules that cannot be broken.
Check it against Software capabilities: confirm that it can be built with the current stack, safely, on time, and make visible the debt or risk created.
Validate it against Infrastructure realtiy: ensure that it can run at scale, securely, cost‑effectively, and with the uptime and cost profile the company is willing to own.
Negotiate and document the solution shape: define how the solution must look so it is truly buildable and operable, with explicit trade‑offs instead of silent debt.
Stay through implementation: ensure what gets delivered still matches that agreed shape and has not been quietly degraded into something unstable, unmaintainable, or unprofitable.

This process cannot be reliably split across multiple roles. When it is fragmented, every group protects its own priority, and nobody owns the balance.

Only when a single role owns this process end to end does the balance hold and the product remain viable.

That single accountable role is the Solutions Architect.

Who can play this role?

A real Solutions Architect is rare.
This is not a theoretical strategist.
This is a person who has already lived the consequences of bad decisions across multiple environments and can recognize them before they repeat.

A strong SA typically has:

Business
Direct exposure to commercial pressure. Ideally has acted as a co‑founder, carried responsibility for a product or engagement P&L, or directly supported revenue‑critical deals. Understands money, contract pressure, and margin, not just technology.
Domain
Several years inside the actual business domain (energy, finance, logistics, healthcare, etc.). Understands compliance traps and which “minor requirements” are in fact legally or operationally non‑negotiable.
Software
Over a decade or even two of building, debugging, and operating production systems in the same languages and frameworks that the IT solution uses. Not “familiar with”, but battle-tested. Has carried on-call responsibility for that stack.
Infrastructure
A decade of running workloads on the actual platform in use (specific cloud environment, network model, security posture). Understands scaling limits, cost levers, and real failure modes under load.
Motivation
The SA must care that the solution actually fits the business and survives in production — and must be rewarded like someone protecting revenue, margin, and reputation.

This is not a role that can be filled by “a smart engineer with potential in six months”.

This profile is rare.

How can a company get a Solutions Architect?

The realistic way is to bring in a senior Solutions Architect from the market as a dedicated contractor for 12–36 months, depending on the IT solution.

The company must bring in someone who has already seen these failure patterns in multiple environments, already paid the price for getting them wrong, and is motivated to protect the business.

Note that SAs, by nature, don’t stay long at a single company. The judgment this role requires comes from switching products and environments — seeing different business models, domains, stacks, and failure modes. That breadth is hard to accumulate inside one product, so senior SAs tend to rotate to build and maintain the experience the role demands.

Why isn’t there an alternative?

When companies understand what the Solutions Architect actually does, they assume that they can easily cover the capability with a workaround.

What happens in practice is always some variation of the same assumption which is failure by nature:

1. Give it to the CTO

Assumption: the Chief Technology Officer is “the technical one,” so can also do the SA job.

The CTO role is crucial, but it’s an executive function: organization design, budget, hiring, vendor strategy, certifications, board work, and customer engagement. The expertise required for CTO is fundamentally different from that of an SA.

Even if a CTO personally meets the SA bar, they almost never have the time or focus to do deep SA work week in, week out.
Using a CTO as a substitute for an SA usually means neither job is done properly.

2. Cover with multiple people

Assumption: “Senior people together can cover it”.

Companies put Engineers, Operations, Product Management, and Compliance in one room, aim for consensus, and expect the group to collectively “own” decisions.

On paper, this seems to cover Business, Domain, Software, and Infrastructure. In reality, it fails because no single person is accountable for the integrated balance. Each participant protects their own priority, consensus dilutes responsibility, and there is still no owner of the balance in real time.

Shared responsibility becomes no one’s responsibility.

3. Level up an internal employee

Assumption: mentor a strong internal engineer or delivery lead into an SA over time.

This rarely succeeds. Correct judgment in this role comes from years of exposure to different pressures and failures:

multiple customers,
multiple business models,
multiple domains with different regulatory traps,
multiple technology stacks and scaling problems,
multiple production outages and cost crises.

A single SaaS company usually cannot generate that breadth fast enough. It isn’t about intelligence; it’s accumulated scar tissue under different conditions.

4. Borrow from a vendor

Assumption: use an architect provided by an implementation partner or service provider.

A vendor architect’s primary loyalty is to their employer’s commercial safety and delivery success — not to SaaS company.

Even if the SA genuinely wants to help, they will always be forced to propose solutions that work for their employer first.

Conclusion

Any SaaS company that operates a live product needs a Solutions Architect.

Without an SA, no one is accountable for keeping Business and Domain expectations aligned with what Software and Infrastructure can actually deliver and run. That’s when costs spike, promises break, and reputation is damaged.

With an SA, that balance is actively protected. The product stays deliverable, operable, and economically viable.

A senior SA is not overhead. It is a required function for any serious SaaS business.

If you found this article useful, feel free to buy the author a cup of coffee ☕.

To var or Not

Stas Sultanov — Tue, 27 May 2025 12:54:30 +0000

var is a C# keyword introduced in version 3.0 (year 2007) to simplify variable declarations by eliminating redundant type specifications.

Despite its straightforward purpose and clear benefits, many developers still passionately debate its use.

Some argue that var should never be used. Others say it's acceptable, but only when the type is immediately clear. Discussions flare around readability, maintainability, and clarity — as if we're still programming on punch cards.

Every time I come across these arguments, I can't help but wonder:
Seriously, what year are these people living in?

Critics of var frequently overlook one crucial reality:
For decades, we've been working with code via powerful IDEs.

Modern IDEs like Visual Studio, Rider, or VS Code provide instant clarity about every aspect of your code, including variables declared using var.

The IDE always knows the variable's type — just hover your pointer.

The idea that var hides the type is simply outdated.
Such concerns might have had merit decades ago, during the era of monochrome displays and basic text editors, but today they're irrelevant.

Some gifted individuals still claim it’s hard to understand the type behind var — usually while reviewing pull requests or browsing code on GitHub’s web interface.

My take? If you’re doing serious code review or exploration, you should be using a proper tool — like VS Code with GitHub extensions.

So, the real problem isn’t var. It’s using the wrong tools for the job.

Conclusion

Use var wherever it is possible.

P.S.

Back in the 1980s, C++ developers started prefixing class members with m_ to distinguish them from method parameters and variables.
It made sense at the time — there were no IDEs, no semantic highlighting, no code navigation. It was a practical workaround for working in primitive environments.

By the late 1990s, IDEs like Visual Studio made that convention obsolete. Syntax highlighting and intelligent tooling eliminated the need for such manual tricks.
Yet some developers still cling to m_.
Not because it helps — but because it’s what they’re used to.

That’s not engineering discipline. That’s fossilized thinking.

The resistance to var today is cut from the same cloth. It's not about clarity. It's about being stuck in habits that modern tools have already outgrown.

For years, Microservice Architecture has been marketed as the ultimate solution for scalability, flexibility, and efficiency within IT Solutions. But does it really live up to its promises ❓

Stas Sultanov — Wed, 14 May 2025 06:07:20 +0000

Stas Sultanov

May 6 '25

Microservice Architecture — The Wrong Turn

#microservices #architecture #programming #devops

Comments

7 min read

Azure Application Insights — The Most Underused Service

Stas Sultanov — Tue, 13 May 2025 06:45:10 +0000

Application Insights is one of the most critical Azure services for managing and reducing operational risks in IT solutions.

Yet, despite being available for over a decade, it remains misunderstood, poorly utilized, and underappreciated by most organizations.

In practice, teams frequently treat it merely as a log repository, while business leaders question its costs and perceive little return on investment.

The Goal: Clarify the real purpose of Application Insights, highlight why organizations often fail to leverage its full potential, and outlines how to effectively use it to protect your business from operational disruptions.

What Application Insights Really Is

At its core, Application Insights is an observability service, but not simply a tool for monitoring performance or capturing errors. It is a strategic asset for reducing operational risks in IT systems.

Its most valuable—and often neglected—capability is predictive analytics, which identifies, correlates, and anticipates issues before they become serious incidents.

From a business standpoint, predictive analytics is essential. It empowers companies to proactively manage risks, minimize impact, and maintain continuous business operations.

In my experience, this capability has repeatedly proven invaluable, preventing downtime and preserving customer trust.

In addition to predictive insights, Application Insights provides comprehensive visibility into system behavior, answering crucial questions like:

How stable is our IT solution?
How do users interact with our system?
Are there undetected issues impacting customer experience?
Where is degraded performance affecting our revenue?
Which integrations or dependencies are negatively impacting system performance?

When implemented effectively, Application Insights goes beyond basic telemetry—it becomes an essential business safeguard.

The Reasons Behind Misuse

In most companies, Application Insights runs quietly in the background. It collects telemetry that no one reviews and is typically configured by people who don’t understand its purpose.

Here are the key reasons why organizations fail to realize its full potential:

Pigeon Architects

Many so-called Cloud Architects focus more on theory than implementation. Their main contribution is placing the Application Insights icon on a diagram — without understanding its role or verifying its integration.
They often:

Fail to define a clear observability strategy
Don’t understand the full capabilities of Application Insights
Lack hands-on experience
Can’t connect telemetry with business outcomes

We call them “Pigeon Architects” because they fly in, drop diagrams, and disappear — leaving behind pretty but useless architecture.

Unaware Developers

Modern development practices often favor abstraction layers, but rarely acknowledge their downsides.

These abstractions can hide essential capabilities of the tools they encapsulate. This is exactly what happens with Application Insights.

For example, .NET developers typically rely on ILogger, which reduces Application Insights to a basic log collector — distorting the data model, correlation, and context.

Without a clear observability strategy, developers typically push everything into Application Insights. This leads to bloated telemetry, increased Azure costs, and reduced return on investment.

Microsoft’s “Install and Forget” Mentality

To make adoption easier, Microsoft promotes the idea that Application Insights requires just a simple install and a line of configuration. While this increases adoption, it also creates a dangerous illusion: that the service works out-of-the-box without any need for understanding.

As a result, developers integrate the tool without knowing how it works — and businesses never benefit from its full capabilities.

What To Do

Observability must be regarded as a business imperative, not just a technical afterthought.

Organizations should engage experts who understand both business needs and the practical implementation of telemetry tools.

Key actions include:

Hire hands-on architects: Experts who understand telemetry intricacies, can align it with business objectives, and directly assist in implementation.
Assign clear architectural accountability: Architects should own not only the design, but also validate and ensure successful practical implementation.
Integrate telemetry with business goals.

The true value lies not just in having Application Insights but understanding precisely how and why to use it effectively.

Final Thought

Application Insights is more than a monitoring feature—it's a strategic business safeguard.

When used intentionally, it becomes your early-warning system, identifying potential disruptions before they impact your business.

Ignoring Application Insights means firefighting reactively. Embracing it ensures you proactively prevent operational crises.

Understand it. Integrate it. Use it purposefully.

Only then can you turn a perceived cost into genuine operational risk reduction and measurable business value.

If you found this article useful, feel free to buy the author a cup of coffee ☕.

Thinking about working with a Microsoft Cloud Solution Provider? Before you commit, take a moment to understand the real risks that rarely get discussed.

Stas Sultanov — Mon, 12 May 2025 14:10:17 +0000

Stas Sultanov

May 7 '25

5 Risks of Using Terraform for Azure

#azure #devops #microsoft #terraform

Comments

4 min read

Wake up, IT...

Stas Sultanov — Mon, 12 May 2025 10:16:27 +0000

Today, the major part of the IT industry that focuses on delivering solutions for businesses has become a grotesque machine, meticulously engineered to bleed businesses dry while delivering little to no real value.

Developers, managers, architects — entire organizations — have abandoned any pretense of delivering meaningful outcomes. Their only concern is how to extract as much money as possible from businesses. Delivering value is irrelevant; all they care about is that the cash must flow.

To sustain this deception, they imitate work: overengineering trivialities, conducting endless refinements, staging meaningless ceremonies, and rolling out hollow “improvements” that devour time, budgets, and attention — often producing negative value.

Buzzwords, architecture patterns, frameworks, certifications, and cargo cult methodologies dominate the landscape, creating a smokescreen of frantic, purposeless activity.

Beneath it all hides a brutal reality:

Most IT initiatives aimed at supporting businesses do not produce real business outcomes.

The stagnation we see in IT today is no accident.

It is the inevitable consequence of systemic rot.

Businesses are pushing back, reflecting on the billions wasted on meaningless delivery. Layoffs, budget cuts, and the mad rush to replace developers and managers with AI are not anomalies — they are a response.

Businesses are done subsidizing an industry that burns money without delivering anything of substance.

IT solutions exist for one reason only — to make money, save money, or protect money.

This is the simple truth that only a minority understands.

Businesses do not fund IT departments because they admire technology. They invest because IT, when done properly, becomes a weapon — increasing profits, slashing operational costs, accelerating decisions, and annihilating competition.

IT is not a playground. Not a lifestyle brand. Not a safe space for tech enthusiasts.

IT is a business weapon.

A tool for dominance. A tool for survival.

Within the IT solution lifecycle, every decision made and every line of code written must be ruthlessly judged by three filters:

Does it increase revenue?
Does it cut costs?
Does it reduce risk?

If the answer is no — it must not exist.

If you are a developer, a manager, or a so-called “tech leader”, you must understand one thing:

Your role is to help the business succeed.

Developers obsessed with shiny frameworks and coding aesthetics are dead weight unless they deliver measurable business outcomes.

A good developer is not the one who masters trendy frameworks or polishes code to academic perfection. A good developer is the one who brings measurable value to the business in the most efficient way — measured in time and cost.

Managers are no different. Drowning in Agile ceremonies, Jira tickets, and SAFe certifications will not save you. If you cannot tie your role directly to profit, cost reduction, or risk mitigation, you are ballast. Disposable ballast.

Modern project management has become a self-sustaining scam — a theater of the absurd, where delivery metrics are worshiped while real-world outcomes rot and die unnoticed.

Real professionals tie their existence to three brutal outcomes:

Increasing revenue.
Reducing costs.
Mitigating risks.

Everything else is waste.

Everything else must be eliminated.

The IT industry is collapsing under the weight of its own lies.

Developers who don't understand business will be replaced — by those who do, or even by machines.

Project managers who can't prove real value will vanish.

Companies that fund IT for the sake of “innovation”, without clear traceability to business results, will die.

This is not a hypothetical.

It is already happening.

The future belongs to those who reconnect IT to its only legitimate purpose:

Business first.

Wake up, IT!

Everyone’s worried about AI taking our jobs. I’m worried about AI taking control of our minds. In just 3 steps, AI becomes the Telescreen of the 21st century — far more advanced than anything Orwell imagined. Here’s how it happens.

Stas Sultanov — Mon, 12 May 2025 09:53:15 +0000

Stas Sultanov

May 5 '25

AI — A Telescreen We Deserve

#ai #devdiscuss #programming #productivity

Comments

3 min read

Microsoft CSP — 5 Risks You May Face

Stas Sultanov — Sat, 10 May 2025 08:32:40 +0000

If you’re reading this article, chances are you’ve already heard about Microsoft Cloud Solution Provider, aka CSP. If not, here’s a quick primer: a CSP is a Microsoft-certified partner company authorized to resell Microsoft services like Azure and M365, along with related professional services.

While the benefits of working with a CSP are well-promoted, less attention is given to the potential downsides. This article sheds light on the risks involved in going the CSP route and provides actionable advice on how to mitigate them.

Note: I have extensive experience on both sides of the CSP relationship. I’ve been directly involved in helping companies achieve Microsoft CSP status, and I’ve also worked extensively with various CSPs as a customer. The insights shared in this article are based on real-world experience and may not apply universally to every CSP.

Risk #1 – You Might Be Provided with a Suboptimal Solution

Microsoft financially incentivizes partners to resell its products and services, including Azure and M365. This can sometimes result in CSPs designing solutions that prioritize revenue over your actual needs.

For example, with M365, you might be sold licenses that significantly exceed your requirements. With Azure, the solution might be over-engineered, with unnecessary services or excessive capacity—just to increase margins.

Mitigation: Involve internal or third-party experts to review your M365 licensing and Azure architecture. An unbiased second opinion can save you a lot of money.

Risk #2 – You Might Receive Low-Quality Service

Qualified cloud engineers are expensive and in short supply. While Microsoft doesn’t strictly require CSPs to assign certified or experienced professionals to every client, some CSPs may start you off with senior staff and then quietly replace them with less qualified personnel.

They may claim senior engineers are still “involved” (e.g., reviewing work a few hours a week), but in practice, this rarely works effectively.

Mitigation: Insist on having qualified experts assigned throughout the engagement. Ask for proof of experience and ensure continuity.

Risk #3 – You May Lose Visibility into Your Cloud Spending

With Pay-As-You-Go subscriptions or Microsoft Customer Agreements, Azure spending data is typically available within a day. But this is not the case for subscriptions managed through a CSP.

This lack of visibility means you can’t track spending in near real-time, making cost optimization difficult and reactive. You may only discover a spike in usage when the monthly invoice arrives. Some CSPs offer their own portals for cost tracking, but not all do.

Microsoft has promised to improve this, but at the time of writing, limitations still exist.

Mitigation: Choose a CSP that provides near real-time spending visibility and allows you to manage M365 purchases directly.

Risk #4 – Slower Issue Resolution

When you have a problem with Azure, you’d normally open a support request directly through the portal. However, if your Azure subscription is CSP-managed, you’ll need to go through your CSP instead.

This introduces an extra layer of communication. Often, the CSP simply relays your issue to Microsoft, acting as a middleman—and delaying resolution in the process.

Mitigation: Consider purchasing a direct Azure support plan to retain access to Microsoft support channels.

Risk #5 – Your Services May Be Tied to the Wrong Region

CSPs are restricted in where they can resell services like Microsoft 365. While it usually makes sense to work with a CSP based in your country, some businesses choose providers from other countries to reduce costs.

This can lead to significant issues. For instance, if your company is in the UK but your CSP is in Turkey, the CSP may create your Entra tenant with Turkey set as the country. This affects currency settings and regional availability.

At first, this might not matter much—since you're paying the CSP directly—but it becomes a serious issue if you decide to switch to purchasing directly from Microsoft. Changing the Entra tenant country later is extremely difficult. Microsoft support will often recommend deleting the tenant and creating a new one—obviously not a viable option for an operating business.

Mitigation: Be very careful when selecting the country during Entra tenant creation. Ensure it matches your business location and long-term plans.

Conclusion

Personally, I do not recommend purchasing Microsoft services through a CSP unless there is a compelling reason to do so. If you're considering a CSP for access to skilled professionals, consider hiring those professionals directly or engaging them on a contract basis instead. This gives you greater control over quality, cost, and long-term risks.

Ultimately, buying directly from Microsoft remains the safest, most transparent option.

Take a moment to review your current cloud strategy—does a CSP really offer the value you expect?

If you found this article useful, feel free to buy the author a cup of coffee ☕.

Microsoft Power Platform — Plugins Telemetry

Stas Sultanov — Fri, 09 May 2025 07:47:51 +0000

Plugins play a key role within many IT solutions built on top of Microsoft Power Platform.

Plugin can encapsulate business logic, enforce rules, and enable integration with APIs, external services, and cloud components — making it a critical part of an IT solution.

Just like any other component with an IT solution, plugins need telemetry tracking. Telemetry helps engineering teams diagnose issues, understand execution flow, and maintain visibility across the entire system.

In the Microsoft ecosystem, Azure Monitor — particularly its Application Insights feature — is commonly used to collect and analyze telemetry data. However, integrating Power Platform plugin with Application Insights is not straightforward.

This article explains how to implement telemetry tracking for Power Platform plugins with Application Insights.

Why the Built-In Mechanism Is Not Good

Microsoft Power Platform promotes using the ILogger interface combined with automatic data export to an instance of Application Insights.

Several key limitations make this approach insufficient:

Limited Functionality — The ILogger interface introduces an abstraction layer over Application Insights, concealing much of its advanced functionality.
Delayed Telemetry Delivery — Exported telemetry may take up to 24 hours to appear, significantly delaying issue detection and resolution.
No Authentication Support — The export mechanism does not support authentication for accessing the Application Insights instance.
Lack of Granularity — There is no way to isolate specific plugin telemetry.
No Multi-Instance Publishing — Telemetry cannot be routed to multiple Application Insights instances.

In short, the built-in mechanism in the Power Platform is both technically constrained and operationally limited.

Direct Integration Is the Way to Go

Within the Power Platform plugin, the most efficient way to work with telemetry is to directly integrate with Application Insights using a dedicated client library.

That was the exact initial reason why the Azure.Monitor.Telemetry library was created — to provide an efficient way to integrate with Application Insights.

The library is lightweight, high-performance, and built specifically to operate under the constraints imposed by the Power Platform plugin runtime. It avoids all the pitfalls of Microsoft libraries while giving you full control over the telemetry pipeline.

If you’re curious why the standard Microsoft libraries are not suitable for Power Platform plugins, I encourage you to read this article, where I explain in detail the journey behind building the library and the critical problems it solves.

Azure Monitor Telemetry Client — Reinvented

You can find a fully functional sample that demonstrates how to use the library for Power Platform plugin development here: Samples on GitHub.

Things to Consider While Using the Library

Before using telemetry within a Power Platform plugin via Azure.Monitor.Telemetry, there are several important considerations that will influence how you structure the integration and manage configuration.

1. How Many Application Insights Instances Are Needed?

Decide how many instances of Application Insights the telemetry client will publish to, and where those instances will be located. There are typically two common scenarios:

Single Instance (Organization Only): All telemetry is sent to an Application Insights instance managed by the organization.
Dual Instance (Customer + Vendor): Telemetry is published to both a customer-managed instance and a developer-controlled instance. This gives development teams access to diagnostics without requiring customer-side access.

The Azure.Monitor.Telemetry library provides a TelemetryClient class that can be configured to use one or many instances of the HttpTelemetryPublisher class, each configured to publish telemetry to a different Application Insights instance.

2. Will Authentication Be Used?

Decide whether the plugin will use Entra-based authentication when sending telemetry.

Power Platform supports this through Power Platform Managed Identity (not to be confused with Azure Managed Identity), which can be used to securely authorize access to Application Insights.

The HttpTelemetryPublisher class can be initialized either without authorization or with a Bearer token obtained from Entra.

At the time of writing, Power Platform does not support assigning or switching between multiple managed identities within a single plugin. As a result, if dual-publishing is required only one can be authorized.

3. Where Will Configuration Be Stored?

Each telemetry publisher must be configured with two values:

Ingestion Endpoint (URI)
Instrumentation Key (GUID)

A recommended approach is to store the configuration as a JSON-formatted object in a single environment variable of text type.

4. What Information to Include in Each Telemetry Item?

The Application Insights data model provides many attributes that can be used to enrich telemetry data with contextual information. The library provides a TelemetryTags class to support this. Key attributes to consider:

CloudRole — A unique name of the component. Use the alias or name of your plugin, or simply “Power Platform”.
CloudRoleInstance — A unique identifier of the runtime environment. Use Environment.MachineName.
OperationId — A unique identifier for the operation. Use IExecutionContext.CorrelationId.
OperationName — A human-readable name for the operation. Use IExecutionContext.MessageName.
ParentOperationId — A unique ID of the parent operation. Use IExecutionContext.OperationId.
AuthenticatedUserId — A consistent user identifier across the Entra-connected systems. Use IPluginExecutionContext2.InitiatingUserAzureActiveDirectoryObjectId when applicable.

Careful configuration and proper enrichment ensure telemetry is actionable and easy to interpret — both for debugging and continuous observability.

Demonstration

One of the most important questions is: What does this telemetry actually look like in practice, and what insights can I get from it?

This demonstration walks through a practical setup and shows exactly what you can expect to see in Application Insights when using the Azure.Monitor.Telemetry library in a real Power Platform plugin.

Scenario Setup

The example solution consists of the following components:

A Power Platform plugin (ProxyPlugin) that is triggered by an event on a Dataverse table
The plugin publishes a message to an Azure Storage Queue
A backend service (ProxyService) picks up the message, processes it, and deletes it from the Azure Storage Queue
Both the plugin and the backend service are integrated with Application Insights using the Azure.Monitor.Telemetry library

This setup allows tracking the full flow of operations end-to-end, across both systems.

Application Map View

The Application Map in Azure Monitor is automatically built from telemetry data. It visually shows the interaction between system components.

In this setup, you see:

ProxyPlugin sends message to Azure Storage Queue
ProxyService receives message from the Azure Storage Queue
ProxyService deletes message from the Azure Storage Queue

This high-level visualization makes it easy to confirm that telemetry flows correctly and relationships between components are captured.

End-to-End Transaction View

One of the most powerful capabilities of Application Insights is distributed tracing — the ability to follow an execution path across multiple components.

In this demonstration, distributed tracing lets you follow a message as it:

Is sent by the ProxyPlugin
Is recieved up by the ProxyService
Is deleted by the ProxyService

Each of these steps is captured as a distinct telemetry item, all linked together using a shared OperationId. This correlation creates a single, coherent timeline of the entire operation, which you can visualize and inspect in Azure Monitor.

This view is essential for understanding how components interact, measuring timing between operations, diagnosing performance issues, and verifying that the system behaves as expected.

Plugin Tracing

If you open the built-in Power Platform Plugin Trace Logs, you can correlate trace messages with the telemetry seen in Azure Monitor.

This gives development teams confidence that telemetry reflects actual plugin execution and improves their ability to debug logic or timing issues.

You may match information with data available with Azure Montior.

This demonstration proves that a well-configured telemetry strategy gives you complete observability across plugin code and external services — all from within Azure Monitor.

Conclusion

Power Platform plugins are mission-critical components in modern business applications, and their behavior must be observable and measurable to ensure reliability.

Unfortunately, the built-in telemetry options provided by Microsoft fall short when it comes to flexibility, precision, and real-time diagnostics.

The Azure.Monitor.Telemetry library offers a focused and production-grade alternative that enables direct integration with Azure Monitor, supports multiple Application Insights instances, and honors the execution constraints of Power Platform.

By following the guidance in this article, you can:

Track telemetry with full control and context
Improve diagnostic capabilities and system transparency
Empower development teams with real-time insights

Observability is no longer optional — it’s a requirement. And with the right tooling, it’s also achievable within Power Platform plugins.

If you found this article useful, feel free to buy the author a cup of coffee ☕.

Azure Monitor — Telemetry Client Reinvented

Stas Sultanov — Thu, 08 May 2025 06:57:54 +0000

Recently, I was brought on board of yet another project to help build an enterprise-grade IT solution based on Microsoft Power Platform and Azure.

Like any well-architected system, the solution required robust observability — the ability to track operational behavior, diagnose issues, and maintain visibility across all components. Naturally, Azure Monitor was the go-to choice.

One of the core components of the solution was a custom plugin for Power Platform. To capture application-level telemetry from the plugin, the obvious approach was to use a telemetry client that integrates with Azure Application Insights, a feature of Azure Monitor. By default, Application Insights allows unauthenticated data ingestion — which makes it a potential target for abuse. To address this, the instance was configured to require Entra authentication for secure access.

And that’s where things began to fall apart.

After investing time trying to integrate both OpenTelemetry with the Azure Monitor Exporter and the legacy Application Insights SDK, it became clear that neither option worked reliably in plugin environments.

As a result, I made the decision to build my own telemetry client library for Application Insights.

Creating yet another telemetry library for a service already backed by a major corporation may seem unnecessary — even questionable. But in this case, there were good reasons to build it.

Plugin-Specific Constraints

Before diving deeper, it’s important to understand the unique constraints of plugin development — especially within the context of Microsoft Power Platform.

Plugins face several critical limitations:

Lightweight

Plugins can be dynamically loaded and unloaded by the host application, so DLL size and dependency count directly impact performance and reliability.
Must manage dependencies carefully
A plugin may be loaded into an application domain where other versions of the same libraries are already present. This creates a real risk of binding conflicts.

Host-specific execution rules

Power Platform handles plugins in a very specific way — caching instances and calling the Execute method multiple times. As a result, certain rules must be followed.

Target .NET Framework 4.6.2

It may not be the most modern runtime, but within the Power Platform ecosystem, it’s the only supported option — and there’s no way around it.
In short, plugin development demands tight control, a minimal footprint, and strict compatibility — none of which are priorities in the design of most modern SDKs.

Why OpenTelemetry with Exporter Is Not an Option

According to Microsoft documentation, OpenTelemetry with Azure Monitor Exporter is the recommended way to integrate with Application Insights for modern .NET applications.

However, this approach falls short — especially in plugin development, and even more so in Power Platform plugin scenarios.

File Count and Size

When using packages OpenTelemetry 1.11.2 and Azure.Monitor.OpenTelemetry.Exporter 1.3.0 in a .NET project targeting .NET Framework 4.6.2, the resulting output includes 126 files, totaling 5243 KB. That’s far from lightweight.

The Problem with Static Classes

The OpenTelemetry model relies heavily on static fields of ActivitySource class.

Due to the way Power Platform manages plugin lifecycles — reusing cached instances and calling the Execute method multiple times — this leads to repeated initialization and duplicate telemetry data being sent to Application Insights.

Maybe it is possible to implement a thread-safe singleton wrapper to control initialization. But doing so introduces complexity, and more importantly, violates Microsoft’s own plugin development recommendations for Power Platform.

Verdict

Even without considering the excessive file count and size, the use of static state in OpenTelemetry makes it fundamentally incompatible with plugin-based architectures.

For Power Platform plugins, OpenTelemetry with Azure Monitor Exporter is simply a no-go solution.

What’s Wrong with the Microsoft Application Insight SDK

Let’s take a look at the issues encountered when attempting to use the Microsoft Application Insights SDK in plugin development.

File Count and Size

When using packages Microsoft.ApplicationInsights 2.23.0 and Azure.Core 1.45.0 in a .NET project targeting .NET Framework 4.6.2, the resulting output includes 109 files, totaling 4643 KB. Not good as for me.

Target Framework

One of the first surprises is that the SDK doesn’t target .NET Framework 4.6.2, the only version supported for Power Platform plugins, and which support ends January 2027.

Instead, it targets .NET 4.5.2 and .NET 4.6 — both of which reached end-of-support in April 2022.

While this isn’t a critical blocker, it clearly signals that the SDK isn’t aligned with the needs of current, real-world development scenarios.

Flaw in Authentication

To support Entra ID authentication, the SDK offers the method TelemetryConfiguration.SetAzureTokenCredential(object tokenCredential), which expects the provided object to derive from Azure.Core.TokenCredential.

However, the SDK does not directly reference Azure.Core.dll, the library that defines TokenCredential.

Instead, it attempts to perform a runtime type check by scanning all loaded assemblies in memory and matching type names via reflection.

It’s difficult to describe how flawed this design is — and what it implies about the quality of engineering behind it.

As you can probably guess, this is where things begin to fall apart. The type-checking logic ends up grabbing the first matching assembly loaded into memory, which may not be the version your plugin references.

Yes, you could try to work around this by referencing the same version of Azure.Core.dll as the one preloaded by Power Platform.
And yes — that might work.

But I don’t need to explain how quickly — and why — this kind of “solution” will break down over time.

Verdict

While it may be possible to use the Microsoft SDK in basic scenarios that don’t require authentication, that wasn’t an option in my case.
Authentication was a requirement — and that alone made the official SDK a clear no-go.

Introducing the Azure.Monitor.Telemetry library.

Faced with the limitations of both OpenTelemetry with Exporter and the Microsoft Application Insights SDK, I made the decision to build a custom telemetry client.

The goal was simple: create a lightweight, extensible, and reliable library that integrates with Azure Application Insights — without the overhead, complexity, or incompatibilities of the official options.

What began as a solution to a specific problem has since evolved into a fully-featured, enterprise-grade library.

Result

The result is a robust telemetry client that:

Ships as just 1 file and approximately 62 KB, compared to hundreds of files and thousands of kilobytes required by existing SDKs.
Targets .NET Framework 4.6.2, .NET 8, and .NET 9.
Supports all telemetry types available in Application Insights.
Works flawlessly in constrained environments like Power Platform, where static state and runtime collisions pose real risks.
Offers minimal abstractions and zero external dependencies.
Can be used for solutions and applications of any type: plugins, standalone applications, distributed systems, and other resource-constrained environments.

Check It out
👉 Repo on GitHub
👉 Package on NuGet.org
👉 Samples on GitHub

The repo with samples contains samples for different use case scenarios like Console Application, Power Platform Plugin.

Afterword

I couldn’t find any documentation for the Application Insights API — especially the part related to publishing telemetry from the client side.

Because of this, I had to explore the source code of the official SDKs available on GitHub and even reverse engineer some parts to understand how things work.

Unfortunately, I must admit that it looks like the engineers who built the Application Insights API were not very familiar with the concept of consistency. It’s difficult to understand the reasoning behind such an inconsistent and fragmented API surface.

The Microsoft .NET client for Application Insights is a great example of how not to develop libraries. Beyond the issues I’ve described and the absence of design consistency, the implementation is massively overengineered and extremely hard to understand.

And the most surprising part? According to GitHub, this SDK has been around for almost 10 years and has over 90 contributors — yet still fails to deliver a practical, developer-friendly solution.