Forem: Mark Blandford

Threat Modelling with Threat Dragon

Mark Blandford — Sat, 16 Aug 2025 12:20:00 +0000

This post is about how to perform Threat Modelling using the STRIDE model. I'll also highlight some tips with using OWASP Threat Dragon, and some practices I choose to follow.

What is Threat Modelling

Threat modelling is a critical part of a secure development lifecycle. We use a model, such as STRIDE, to help us identify, review and mitigate potential threats and vulnerabilities within a software system (or application).

The goal is to identify and mitigate threats early, before they become problems. It is critical that everyone in the team responsible for the software application is involved, as everyone can bring a different perspective. Furthermore, it helps educate all members of the team, as application security is everyone's responsibility.

The STRIDE Threat Model

STRIDE is a threat classification model which was developed by Microsoft.

Threat	Description
Spoofing	Impersonating something or someone else.
Tampering	Modifying data or code, in transit or at rest.
Repudiation	Denying an action or transaction has taken place. Denying without accountability.
Information Disclosure	Exposing data to unauthorised parties.
Denial of Service (DoS)	Preventing legitimate people or processes from access a service or resource.
Elevation of Privilege	Gaining unauthorised access to resources or functionality.

Each element within the system can be exposed to different threats in different ways. Furthermore, they could be exposed to multiple 'implementations' of the same threat. For example Denial of Service against a process, could come from a DDos attack or from an internal, malicious actor. Each would be represented as a different threat on that process.

Quick Reference

STRIDE Threat Classification Overview

Threat	Description	Key Question
Spoofing	Impersonating others or systems.	"Can someone pretend to be someone else?"
Tampering	Modifying data or code.	"Can someone alter information?"
Repudiation	Denying actions occurred.	"Can someone deny they did something?"
Information Disclosure	Exposing sensitive data.	"Can someone see information they shouldn't?"
Denial of Service	Blocking legitimate access.	"Can someone prevent the system from working?"
Elevation of Privilege	Gaining unauthorised access.	"Can someone do things they're not allowed to?"

Threats by Threat Dragon Element

Element	Applicable Threats	Common Examples
🧑 Actor	None directly	Customers, other systems.
↩️ Data Flow	Tampering, Info Disclosure, DoS	HTTP requests, database queries, API calls.
📦 Process	All STRIDE threats	Web apps, APIs, microservices.
📁 Store	Tampering, Repudiation, Info Disclosure, DoS	Databases, file systems, browser storage.
🤝 Trust Boundary	Defines the threat context	Network perimeters, different infrastructure.

Common Mitigations

Threat Type	Typical Mitigations
Spoofing	MFA, authentication.
Tampering	Encryption, input validation.
Repudiation	Audit logs, digital signatures.
Information Disclosure	Encryption, access controls, limit data stored.
Denial of Service	Rate limiting, load balancing, monitoring.
Elevation of Privilege	Role-Based Access controls, least privilege, regular reviews.

Threat Dragon

Threat Dragon is a free, open-source threat modelling tool developed by OWASP. It enables teams to visually develop the data flows of a system, and document threats and mitigations. It supports STRIDE, which this article focuses on, and other threat models. The output of Threat Dragon is not only visual, but is in JSON, so can be easily added to any source control system.

Getting Started

Threat Dragon is available in two ways:

🌐 Web version - Simply use within the browser at https://threatdragon.com.
💻 Desktop version - Available for download from the OWASP Threat Dragon GitHub release page.

Simply create a new model and start adding elements - Threat Dragon will suggest applicable STRIDE threats for each component you add.

Tips with using Threat Dragon

Threat Dragon is a little temperamental but once you get use to it, it is a great tool.

Interface Navigation

Save frequently - Models can be lost easily and saves sometimes fail.
Click arrow heads on Data Flows to select data flows (not the line itself).
Click anywhere on Data Flows to add corners/drag points.

Threat Analysis Best Practices

Different elements have different threats - A Tampering threat on a Data Flow is different compared to a Process or Store. The mitigations will be different too.
Include all applicable threats - Even mark as N/A to show they were considered.
Mark out-of-scope elements - Document dependencies you can't control, but be aware they may change behaviour after marking as out-of-scope.
Review regularly - Threats change as systems evolve, especially when adding new integrations.

Troubleshooting

Models are JSON underneath - use JSON linting tools if Threat Dragon shows errors.
- You can manually edit JSON files for bulk changes (find/replace for renaming).

An Example

Here is a small, not very detailed example of a web application, which

Uses a 3rd party authentication provider (out of scope).
Communicates with a trusted API, which is considered a part of the software system.
This API stores data in a database.

Steps

Create the data flow / model of the application flow within your software system.
1. If it's large, consider breaking it down and create, smaller separate threat models for logical parts of the flow. Threat Dragon can have multiple models within the same 'parent' model.
Analysis each process, store & data flow following STRIDE.
1. When you have an element selected, clicking '+ New Threat' will at first default to adding the applicable threats for the element selected. However, you don't have to stick with this, feel free to add more as you see fit.

The documented threats for the example

Out-of Scope

Customer / browser - In this example, I've marked the customer as being out-of-scope as we cannot control customer behaviour or their browser. This will not always be the case however.
3rd Party Auth Provider - We assume it is secure but fundamentally it is not our responsibility to ensure it is, as it is maintained by a 3rd, trusted party.

Processes

Web App (frontend) 📦

Threat	Risk	Status	Mitigation
Spoofing	A malicious actor is able to gain access as another user of the application.	🟢 Mitigated	Tokens are only stored in secure, httpOnly, SameSite cookies so cannot be accessed by anything else. XSS & CSP is in use too.
Tampering	Modifying data or code.	🟢 Mitigated	🔒 Code reviews, signed commits, and access controls are enforced. No data is stored with the Frontend application.
Repudiation	Denying an action or transaction has taken place. Denying without accountability.	🟢 Mitigated	📜 Audit logs are maintained with user identifiers. All commits must be signed
Information Disclosure	Some, customer information is stored in the browser local storage.	🔴 Open (High)	Avoid using localStorage or sessionStorage.
Denial of Service	Preventing legitimate users from accessing a service or resource.	🟢 Mitigated	⚖️ Uses CDN caching and load balancing.
Elevation of Privilege	Gaining unauthorised access to resources or functionality.	🟢 Mitigated	🔑 Authorisation checks are carried out and there is no concept of roles within the application. Control to the infrastructure of the application and code is access controlled following least-privilege.

Backend API 📦

Threat	Risk	Status	Mitigation
Spoofing	A malicious actor is able to gain access as another user of the application.	🔴 Open (High)	CORS is used to limit access to just the Web App domain. However, no authorisation checks are carried out on the requests using the JWT, meaning anyone with a valid JWT can impersonate another customer.
Tampering	Modifying data or code.	🟢 Mitigated	🔒 Signed commits, code reviews, and access controls. No data is stored with the API.
Repudiation	Denying an action or transaction has taken place. Denying without accountability.	🟢 Mitigated	📜 Log all requests with user ID from the JWT and associated actions.
Information Disclosure	A 500 error response shows the stack trace, which exposes the language the API is written in etc.	🔴 Open (Medium)	Return generic or sanitised error responses.
Denial of Service	Preventing legitimate people or processes from access a service or resource.	🟢 Mitigated	⚖️ Rate limiting, load balancing, and request timeouts.
Elevation of Privilege	Gaining unauthorised access to resources or functionality.	🟢 Mitigated	🔑 Role-based access control and least privilege enforced at the API code & infrastructure.

Stores

📁 Internal Database

Threat	Risk	Status	Mitigation
Tampering	Queries or stored data could be manipulated.	🔴 Open (High)	Use parameterised queries, restrict database accounts, and enforce strict schema validation.
Repudiation	Denying an action or transaction has taken place. Denying without accountability.	🟢 Mitigated	📜 Database logs are maintained.
Information Disclosure	Personally identifiable information stored in plain text.	🔴 Open (High)	Encrypt sensitive fields at rest & while in use. Ensure there are strict access controls to the database.
Denial of Service	Preventing legitimate people or processes from access a service or resource.	🟢 Mitigated	⚖️ Apply query limits, use indexes, and enable monitoring and alerts.

↔️ Data Flows

POST /api/{username}/order - 📦 Web App ↔ 📦 Backend API

Threat	Status	Key Issue
Tampering	🟢 Mitigated	TLS encryption.
Information Disclosure	🔴 Open (High)	Username is in URL path.
Denial of Service	🟢 Mitigated	Rate limiting, API gateway.

Authenticate - 📦 Web App ↔ 📦 Third-Party Authentication Provider

Threat	Status	Key Issue
Tampering	🟢 Mitigated	TLS encryption.
Information Disclosure	🟢 Mitigated	TLS with no sensitive data in request.
Denial of Service	⚪ N/A	Third-party dependency risk. However, if they suffer a DoS, what impact will it have on us?

Verify JWT - 📦 Backend API ↔ 📦 Third-Party Authentication Provider

Threat	Status	Key Issue
Tampering	🔴 Open (Medium)	TLS 1.1 used (insecure)
Information Disclosure	🔴 Open (Low)	TLS 1.1 used (insecure)
Denial of Service	⚪ N/A	Third-party dependency risk. However, if they suffer a DoS, what impact will it have on us?

Queries - 📦 Backend API ↔ 📁 Internal Database

Threat	Status	Key Issue
Tampering	🔴 Open (Low)	No SSL on internal connections.
Information Disclosure	🔴 Open (Low)	Credentials / queries unencrypted.
Denial of Service	🟢 Mitigated	Resource limits & monitoring.

Conclusion

Threat modelling with STRIDE and Threat Dragon can help teams to methodically analyse software systems, to help them identify vulnerabilities before they become issues.

Key takeaways

Include everyone - Developers, architects, QAs, business stakeholders each bring unique perspectives.
Be comprehensive - Document all threats, even those marked N/A, to demonstrate they have been considered.
Review regularly - Systems evolve, and so should the threat model.

Threat Dragon may have its quirks, but it is free, accessible and fairly straight-forward to use. The JSON-based output is ideal for source control.

Remember: the goal isn't for bullet-proof security, it is about understanding and documenting the known risks.

Additional Resources

No Spikes

Mark Blandford — Tue, 21 Jan 2025 08:46:49 +0000

I thought about writing this post a while ago but it has recently come to mind once again.

A Spike is

...a timeboxed experiment meant to gather information to reduce risk and enable more accurate estimates for complex user stories.

-- Agilemania - What is an Agile Spike Story?

I'm not a fan

First off, let me be clear I'm not 100% against completing Spikes. Perhaps 99% against them though. The big issue I have is the tendency to over-rely on them for even the tiniest of uncertainty or lack of clarity or direction. I'm generally of the opinion an engineering team should be skilled and knowledgeable enough, plus have the confidence, to 'jump in' and engineer a solution. However, let's dig a little further.

The attributes of a Spike

Time-boxed ⏳

OK, so we're not commiting (if there is such as thing) to an estimate, but instead we will limit the time needed to gather the information required. How large is this 'time-box'?

A day?
A week?
A month?

If we can come up with an accurate time-box, then we have some certainty of the unknown. Can that not be factored into the 'uncertainty' when estimating the actual Story?

What happens if the Spike gives us what we're looking for?

Then what? The findings get written down, the prototype put to one-side (to decay) and the team switches context to something else. When the team then come back to start the implementation, are the findings of the Spike still accurate? Do they even make sense / do we understand them still / would the prototype still work? We probably need to spend even more time refreshing our memory.

What happens if we still don't have enough information at the end of the time-box?

Do we extend it or commit to a new Spike? - How long for? ⏱️
Do we stop and do the actual work? - So what was the point of the Spike if we are going to get going without all of the information anyway? We've just wasted the time it took to complete the Spike rather than just trying to get the work done.

Winner - Story

Experiment 🧪

This is possibly the only, possible case I can see for a Spike, but it's small. Even with using a Spike to experiment or prototype a solution, what happens when it's 'done'? What does 'Done' even look like? When is enough, enough?

Do we throw away the prototype? - If so, then we've probably wasted that time.
Do we 'productionise' the prototype in the Story? - Probably could have avoided the Spike then and just done the work. If you're already neck-deep in it, it is costly to stop and come back later too.
Do we document the findings of the experiment for later? - We've just introduced a huge context switch. Whoever picks up the actual work, which may not be the engineer who completed the Spike, may have a slight head-start but...
- They may not understand the findings (another Spike perhaps? 🙂)
- Things might have changed since. Depending on how long ago the Spike was completed, the findings may no longer be relevant.

If an experiment is to be carried out because there are two or more potential solutions, which all deliver the value in the Story, why not simply implement the most simple solution (MVP) and iterate on it?

Winner - Story (a little closer)

Gather Information 💭

I'm a strong believer in empowering Software Engineers to engineer solutions. I would expect any engineer worth their salt to be capable of gathering the information they need to complete a Story, often based on a simple, even potentially ambiguous problem statement (remember when a Story could fit on a Post-It?). However, if it is about gathering requirements, then that is not the responsibility of the engineer and the request should not even be in the team yet.

I'm struggling to think of occasions where there wouldn't at least be some information to be able to start work on a Story. After all, the ACs should be enough to get going and figure it out as you go.

Winner - Story

Reduce Risk 🛡️

OK, so we're completing a Spike because we don't know if something is technically feasible or something similar. I.e. you don't want to commit to an implementation that may not work. If your team already aims to deliver MVP and iterate on it, what is the harm in trying? You should know fairly quickly whether something is going to work, if it is great, continue. If not, stop and do something else. Doing this as a Spike, doesn't really change the 'risk', it just reduces our agility and makes us slower to implement a solution. If something is so unknown is it with the right team to do it (does the team have the necessary skills)?

Winner - Story

More accurate Estimates

Who cares? An estimate should not be a measure of time or used to measure the output or success of the team. In my opinion an estimate is made up of three components:

Development / Test Effort
Complexity
Uncertainty

I guess you might complete a Spike to try and reduce the uncertainty. Why not simply adjust your estimate of the Story instead? If you're uncertain, perhaps the estimate (in Story Points, Fibonacci Sequence) from a 3 to a 5, or perhaps a 5 to an 8. An estimate is just that and should reflect what is known at that point in time. It's why a team should ideally estimate their Stories only shortly before they plan to work on them.

Winner - Story

Complex Stories

If the story is as small as it can be to still offer value to the customer, does it matter if it's complex? Again, the level of perceived complexity should be a factor of the Story's estimate. Great engineers should thrive with overcoming complexity.

Winner - Story

Conclusion

I can probably count on one hand the number of times when I've genuinely felt a Spike was needed because the path was so unknown or brand-new then determining whether something could even ever be done was unclear. Perhaps it's just the type of work I do but it has probably been years since I've seen the value in a Spike.

If a Spike is needed to develop the 'gold-standard' of requirements or an approach to a solution, isn't that Waterfall? At the very least it reduces the agility of the team.

I have not Users

Mark Blandford — Sun, 21 Apr 2024 07:19:51 +0000

...there are only two industries which refer to their customers as users, drugs and computers.

-- Edward Tufte

I first heard this quote a number of years ago, and it has since been misquoted and misused, notably replacing 'drugs' with 'illegal drugs'. However, it still rings true to me. Why do we refer to those that utilise our software, web sites, services etc. as 'users' rather than:

Customers
Clients
Visitor
Consumer
Patron (probably not, but you get the idea)
Anything else that is specific to your application. In finance perhaps, you might use 'Adviser' for example.

I can't explain why we do.

Now consider the negative conatations of the word 'user' when in the context of:

...there are only two industries which refer to their customers as users, drugs and computers.

-- Edward Tufte

Add in the growing research and concern around the addictive nature of certain 'computers' or technology such as Social Media, the parallels are certainly there.

Time for a different approach

For a while now I have been making a concerted effort to not use the word 'users' when referring to those whom use the technology I develop. Don't get me wrong, it is a hard habit to break. I have found however, it feels like I'm becoming more familiar and aware of those that do. It's as if by referring to those 'users' by another noun, the real persona of the individual develops.

In practicing this, I've also noticed something else: when we all refer to those that use technology as 'users', it becomes difficult having conversations with each other. For example,

Dave: Hi, a user has submitted order #123. I can see from our logs this order hit your system, but we don't know what's happened. Can you find out?

Jane: Hi Dave, yep I can see there was an issue with the user's order. A user has checked-out the order on the system to investigate.

Dave: Thank you Jane, when will the user get an update?

Jane: ???

See what I mean? If everyone is talking about users, it can quickly become unclear about whom everyone is talking about! If we switch our terminology instead to 'Customer' and 'Sales Agent', it becomes far clearer.

Dave: Hi, a customer has submitted order #123. I can see from our logs this order hit your system, but we don't know what's happened. Can you find out?

Jane: Hi Dave, yep I can see there was an issue with the customer's order. A sales agent has checked-out the order on the system to investigate.

Dave: Thank you Jane, when will the customer get an update?

Jane: Not sure, I'll ping a message to the agent for an update.

Personas

When I refer to the 'clients' of an application, I'm able to better describe and empathise with their experience as well as better illustrate the purpose of the application itself. For example:

When a user is on the checkout page, they are seeing an error...

Now substitute the word "User" with "customer"...

When a customer is on the checkout page, they are seeing an error...

It gives a different meaning and importance to the sentence. By changing that one word, we immediately understand more about what is going on and what the consequences could be. For me the differences are:

A customer is someone who:

Is engaged.
Wants to 'give us money'! But we can lose money, if we provide a poor experience.
Will possibly come back if they have a great experience. Or won't if they don't.
provides an opportunity for us to build a relationship with.

A user however feels more passive and less engaged. It doesn't immediately describe what they want or their purpose.

Closing

I think if everyone in technology were to transition away to referring to the consumers of their tech, applications, services, web sites etc. to nouns more reflective of the customer's experience / persona, we will build better, more engaging and valuable tech.

The next time you go to refer to a 'user' of your web site, pause. Are they a visitor or perhaps a customer? Depending on your response, may just change how you think about and treat them and their experience.

How I practice TDD

Mark Blandford — Sat, 18 Nov 2023 09:08:17 +0000

What is TDD?

Test-driven development (TDD) is a software development process relying on software requirements being converted to test cases before software is fully developed, and tracking all software development by repeatedly testing the software against all test cases. This is as opposed to software being developed first and test cases created later.

-- Wikipedia - Test-driven development

In essence, write the tests first and then the production code.

Red/green/refactor - the TDD mantra.

-- Test-driven development: by Example - Kent Beck

Red - You write a test that doesn't work (it probably won't even compile as the Unit Under Test doesn't [yet] exist).
Green - Make the test work - You build the Unit Under Test / production code.
Refactor - clean up the test, removing duplication etc. (but please keep the test 'DAMP' / 'AHA').

Some of the feedback I hear about why Engineers don't TDD

TDD is slower

I disagree. If your team has a mentality for quality software, then you already must have a culture of automated testing. As such, why not build the tests first?

In fact, the opposite is actually true. I believe it has been proven / accepted that engineers who practice TDD, write more tests and are more productive.

When I don't write tests, development is faster

Yep, it sure would be: at first. Give it a few weeks or months and I expect you'll hit a wall, with confidence dwindling, that the software still works.

I don't like writing tests

I get it. This is where TDD has helped me. I think practising TDD, makes you better at writing tests and as such, the aversion evaporates. It's still writing code, so should be enjoyable. I think writing the tests first make the process feel less like a chore.

TDD is hard

Now we're getting somewhere. I think you're right, at least at first. In my experience, this differs between languages and competence too. For example, I began practising TDD in C#, an OO language. I found the strictness comforting and because of my level of experience at the time, it was easier.

Then, when I started developing Angular applications, I simply couldn't use TDD as I had been use to. I had to adapt. I didn't know the language & test framework well enough. I struggled with the 'laziness' of the language and the blurred lines of the unit testing suggested. Likewise, when I develop in Python, I try but I cannot do 'full' Test Driven Development but I try. I still get some benefit from the little I can do. This brings us onto the next section, the benefits I gain from TDD.

The Benefits I get from TDD

How many engineers plan the work they are about to develop? I bet most of you don't. This covers the first, three benefits I gain from TDD:

1️⃣ TDD helps me plan / acts as a To-Do list for the work ahead.
2️⃣ TDD forces me to break a task into smaller pieces (units).
3️⃣ This process immediately makes me think about how the feature is going to work and even raise any questions if the requirements aren't clear. I find this really beneficial, as it means any clarification on the requirements is completed right at the start, and not when you're already head-deep in the code.

How many times have you been working on a new feature, and got to a point where you've added a little extra¹ as you think it will be required in the future?

4️⃣ TDD stops me developing more than what has been requested. It stops me trying to predict the future.

However:

If you get today's work done today, but you do it in such a way that you can't possibly get tomorrow's work done tomorrow, then you lose.

-- Kent Beck (excerpt from Refactoring by Martin Fowler)

I bet most engineers, who don't subscribe to TDD, have gotten to where the production code is done, and they've come to write the tests and found the code isn't easily testable. This probably means your code doesn't adhere to some well regarded development patterns. This doesn't mean, make all of the methods public!

5️⃣ TDD helps me write better code. This often means it is more functional with few side-effects.
6️⃣ TDD also helps me learn and get more comfortable with the test frameworks: I actually enjoy writing tests!

Similar, you've written the production code, you're beat, you're done. The last thing you want to do is write tests. You're validated further when you tell your business stakeholders, "the work is done I just have the tests to write". They're response: "don't bother, we need that feature out".

7️⃣ I've written the tests up-front, and I get that regular feedback, as the tests go from failing to passing.
8️⃣ TDD gives me confidence that when all the tests are green 🟢, I've completed the full, working feature. There have been times where I have such confidence, I don't even need to manually test the changes: I know it works because my tests tell me so.

When to TDD

I think there are occasions when it is difficult or not possible to practice 'full' TDD all of the time:

If you don't know or cannot predict the result of a scenario, then I don't think you can TDD.
If you don't have an idea of how to get to the result / outcome, then you probably can't TDD. For example, if you need to 'trial and error' to get there. However, you maybe able to figure it out as you go (so try and start with TDD).
If you simply don't know how to write tests, then you can't practice TDD. Are you ready to develop fully featured, robust & maintainable software in that case?
If you don't have a solid understanding of the language, then TDD will be hard, but again I think you can start.

It could be argued that none of these points actually highlight a problem with Test-Driven Development. These points potentially highlight problems with the development / design approach of an engineer.

How I TDD

Let's go through two examples, for TDD with unit tests: one in Angular and another in Java.

Angular

The UI Requirement

I want to be able to change the colour scheme on my website, between three different themes: Default, Night & Sunny.

At this time, the specs of the themes are out of scope but will be controlled depending on the value of data-theme attribute on the html element.

What we need to develop for the UI feature

An Angular Service to update the data-theme attribute on the html element.
a UI element for the customer to select one of the three themes.

For the brevity of this post, we'll just address step 1, the Angular Service. This is what I have on my own website.

With these requirements, let's now write our 'to-do' list but as unit tests & test setup:

describe('ThemeService', () => {
  it('should set the html data-theme attribute to the provided theme', () => {
    const dom = {} as jest.MockedObject<Document>;

    const service = new ThemeService(dom);

    throw new Error('Not implemented');
  });
});

This test describes what we have to implement:

We need a new Angular Service.
The service has a dependency on the DOM / Document.
The service will need a public method to enable the selected theme.

Now, we know that if we even attempt to run the test, it'll fail immediately because ThemeService doesn't exist. 🔴

I always ensure too, that my new test throws a 'Not Implemented' exception. Test Frameworks generally mark tests as passed 🟢, if there is no assertion. Ensuring the test throws an exception, immediately means the test is 🔴 and I don't have a false-sense that everything is done.

💡Tip: I have added shortcuts / code snippets in my IDEs, to enable me to quickly insert a 'Not implemented' exception. I can type nie + tab and depending on the language, I get the new exception added.

Next, I actually create the Production Service:

@Injectable({
  providedIn: 'root'
})
export class ThemeService {
  constructor(@Inject(DOCUMENT) private document: Document) { }
}

At this point, I can now run the test. The spec will now compile but thanks to the throw new Error('Not implemented'), will still fail. Now back to the test.

I think I want a public method called, enableTheme(theme: string), which should take the theme and do as required:

Use querySelector() on the Document to select the html element.
Call setAttribute to set the data-theme attribute with the provided theme.

This takes us to:

Update the dom Mock, to mock the querySelector method, as we'll need to call it.
Implement the rest of the test.

it('should set the html data-theme attribute to the provided theme', () => {
  // ARRANGE
  const dom = {
    querySelector: jest.fn(),
  } as jest.MockedObject<Document>;

  const htmlStub = {
    setAttribute: jest.fn(),
  } as jest.MockedObject<HTMLHtmlElement>;

  jest.spyOn(dom, 'querySelector').mockReturnValue(htmlStub);

  // ACT
  const service = new ThemeService(dom);
  service.enableTheme('Sunny');

  // ASSERT
  expect(htmlStub.setAttribute).toHaveBeenCalledWith('data-theme', 'Sunny');
});

Now we implement the shell of the real, enableTheme(theme: string) method, again with a 'Not implemented' exception:

enableTheme(theme: string): void {
  throw new Error('Not implemented');
}

The test will now run but with the 'Not implemented' exception coming from the production code. We're getting there. Finally, we can implement the guts of the Unit Under Test.

enableTheme(theme: string): void {
  const root = this.document.querySelector('html');

  root.setAttribute('data-theme', theme);
}

Viola, we have a passing test, which means we have a fully implemented feature.

Next, we'd refactor not only the test but also the production code, safe in the knowledge we have a working test to fallback on. For example, theme would probably be better as an enum.

Java Spring Boot

The Integration Layer Requirement

I want to be able to serve the list of themes available to the UI, from our API, but where the themes are provided by a 3rd party at https://example.com/themes

The response should be a list of theme names, sorted alphabetically.

What we need to develop for the API feature

A new, Feign client, to make the get request to https://example.com/themes.
1. This client will expose a get() method.
A new Java service to call the new client.
A new end-point on our controller which calls the new service.
1. This is what our UI will interface with.

For the brevity of this post, we'll just address step 2. the Java Service.

With these requirements, let's now write our 'to-do' list but as unit tests & test setup.

This leads to the following tests:

This code is untested 😛

public class CustomThemesServiceUnitTest {
    @Test
    void getThemes_themesAvailable_listofSortedThemes() {
        throw new NotImplementedException();
    }

    @Test
    void getThemes_themesUnavailable_emptyStringArray() {
        throw new NotImplementedException();
    }
    // in reality, we would have more tests,
    // covering error responses etc.
    // because we're already thinking about these up-front
    // (and should be a part of our requirements too)
}

Working backwards, these tests describe what we need to implement to achieve our objective:

We need a new Java Service, CustomThemes.
The service will have a dependency on the soon-to-be developed, Feign client.
1. The Feign client will have a get() method we need to call to retrieve the themes.
The service will have a public, getThemes() method.
1. The method will need to ensure the list is sorted alphabetically, when there are themes available.
2. The method will need to elegantly handle when no themes are available.

Again, we know the tests will fail 🔴 immediately because of the 'Not implemented' exception.

Next, I write the shell of the new service (for brevity I'm not including Spring Boot decorators, interfaces etc.).

public class CustomThemesService {
    public String[] getThemes() {
        throw new NotImplementedException();
    }
}

We've not done much here, but it means we can continue with our tests, and they will at least compile. Now let's write the tests:

public class CustomThemesServiceUnitTest {
    @Test
    void getThemes_themesAvailable_listofSortedThemes() {
        // ARRANGE
        // we will assume the `ThemeClient` interface has already been created
        // I'm using Mockito to mock the dependencies
        ThemeClient mockThemeClient = mock(ThemeClient.class);

        // not sorted, so we can verify the list is sorted
        String[] themes = { "Sunny", "Default", "Night"};
        when(mockThemeClient.get()).thenReturn(themes);

        // ACT
        CustomThemesService cut = new CustomThemesService(mockThemeClient);

        // ACT & ASSERT
        assertEquals({ "Default", "Night", "Sunny" }, cut.getThemes());
    }

    @Test
    void getThemes_themesUnavailable_emptyStringArray() {
        // ARRANGE
        ThemeClient mockThemeClient = mock(ThemeClient.class);

        when(mockThemeClient.getThemes()).thenReturn(null);

        // ACT
        CustomThemesService cut = new CustomThemesService(mockThemeClient);

        // ACT & ASSERT
        assertEquals(0, cut.getThemes().length);
    }
}

These look like some pretty good tests to me, that satisfy the requirements. If we run them, they will fail, as we have our 'Not implemented' exception in the Unit Under Test, so let's build that now.

public class CustomThemesService {
    private final ThemeClient themeClient;

    // we will assume the `ThemeClient` interface has already been created
    CustomThemesService(ThemeClient themeClient) {
      this.themeClient = themeClient;
    }

    public String[] getThemes() {
        String[] themes = themeClient.get();

        // sort the themes
        Arrays.sort(themes);

        return themes;
    }
}

If we run our tests, now we should have one passing 🟢 and one failing 🔴. This is great and exactly what we want. Now to finish the implementation to handle when no themes are returned.

public class CustomThemesService {
    public String[] getThemes() {
        String[] themes = themeClient.get();

        if (themes == null) {
          return new String[0];
        }

        // sort the themes
        Arrays.sort(themes);

        return themes;
    }
}

And that should be it. Again, this is untested but looks about right to me. 🤞

Integration Tests

I highly recommend TDD with Integration Tests. I actually find these easier, and depending on the Unit Under Test absolutely leads to better quality. By following TDD with Integration Tests also prevents me from forgetting to implement the tests, which I think happens frequently.

For example:

You're developing a new end-point for a Java API.
1. You already use a tool such as Rest-assured for your Integration Tests.
2. You know what the new end-point is to be, what the responses are etc.
You're developing a new Angular UI component.
1. You have the designs and know what HTML elements etc. should be used.
2. You already use tools such as Playwright or Cypress for your Integration Tests.

In both of these situations, it is easy to write the Integration Tests. They won't work but when they finally do, you know you're dev complete. Furthermore, it'll make you think about the response permutations and potential edge-cases. Things, that may ordinarily get missed (or ignored later, when you're 'done') when diving head-first into the code.

Fixing defects or updating an existing feature

When you need to squash a bug or update an existing feature the process is very similar.

Run the tests before you make any changes. All of the tests should pass 🟢. (If they don't you,ve got bigger problems).
Update your existing tests, or create a new test(s) to cover the new, expected behaviour.
Run your tests. The ones you have updated, should fail 🔴.
Implement the changes required to the production code.
Run your tests. If you've fixed the issue, then you tests are green 🟢 and you're done.

I get a real buzz when it comes to squashing a bug using this approach. Creating a test that asserts the expected behaviour to fix the bug, and then seeing that go green, is great. You know you've fixed it!

Summary

Write the test shells, as a 'to-do' list, covering just the requirements.
- Remember to throw your 'Not implemented' exceptions in the tests.
Implement the skeleton of the production code / Unit Under Test.
- Remember to throw your 'Not implemented' exceptions in the Unit Under Test.
Write the body of the tests.
- The tests will still fail 🔴, but with 'Not implemented' exceptions from the Unit Under Test.
Implement the production code in the Unit Under Test.
The tests should now all pass 🟢.
Go back around, and refactor / improve what has just been implemented with the confidence that you are covered with working tests.

I would guess that the majority of Software Engineers can at least complete steps 1 & 2, on the majority of the code they write. If you can start there, then I think in no time at all, you'll be able to complete all of the steps above. At the very least, your starting to use TDD concepts and I guarantee that will lead to better quality code.

If you've practiced TDD for your unit and integration tests, then do you need any manual testing?

I always think back to a time when I built a really cool Easter Egg in a 404 error page. I was then pulled up by a more senior engineer, who (rightfully) challenged me on how I was going to test it. Needless to say, that 404 page never saw the light of day. Thank you, Richard, I'll never forget that valuable lesson! ↩

The Venn of Angular Component Testing: Benefits & summary

Mark Blandford — Sun, 29 Oct 2023 12:00:00 +0000

If you haven't read the previous parts of this series, please take a look otherwise what's to follow may not make too much sense.

The Venn Diagram of Angular Component Testing

This brings us to the heart of this series. Going back to the definition for an Integration Test from part 1 of this series:

[an Integration Test] Asserts a unit of software interacts as expected with another

After what I've previously highlighted with Sociable Tests, I think Sociable, Angular Component Tests are actually just a type of Integration Test. At the very least there is a significant overlap. Given this, we should treat and expect Sociable / Component DOM tests to behave as Integration Tests: they are more fragile & slower.

I'm sure if you've got this far though the series, then you are familiar with the Testing Pyramid?

Here is my take on that but applying a similar principal to Angular, Component Testing.

This highlights the overlap between Unit & Integration Tests when it comes to testing Angular Components and how the 'grey area' of testing is the Sociable tests. It also emphasises the ratio (not to scale) of these types of tests I think we should be aiming for.

If I could change three conventions regarding the general testing approach we take in Angular applications, it would be to have:

⏫ Have far more Solitary Unit Tests. Day-to-day I unfortunately see very few of these.

it('should test', () => {
  //arrange
  const component = new MyComponent();
  ...
  //act
  ...
  //assert
  expect(component...).toEqual(...);
})

🔼 More 'real' Integration Tests, that mimic real customer behaviour & interactions (using tools such as Playwright or Cypress).
⏬ Fewer (near zero) Sociable / Component DOM Test Tests.
1. Should push engineers to have any logic within the class rather than the template.
2. If you have to have them, then call them out. Put them in separate describe('IT Tests'){} blocks or something similar (and limit the TestBed boilerplate to that block).

I think if an applications' test suite is organised in this way, it would lead to improved development practices. Engineers will lean more towards the DRY principal and smaller, more distinct components which have a single responsibility.

Rebuttal

Given the 'Venn of Angular Component Testing' one could argue to simply just have a test suite of Sociable Tests, with no Solitary or Integration Tests. You absolutely could. I've not tried but I imagine, at scale it would become unmanageable. I would strongly expect too, that you will have a suite of Integration tests using something like Playwright or Cypress, so you'll end up duplicating your test effort.

Closing

Think about why we test interactions in the DOM? I think we do it to test how the customer is going to use our component / application. We already use tools like Playwright and Cypress for our E2E Testing¹ which are probably better suited (and potentially less fragile than Sociable Tests) for this testing.

How many times have you faced problems in the past with writing your [Sociable] unit tests to assert something in the DOM and you just cannot get it to work, no matter how many fixture.detectChanges() you add? 😄 I know I've spent too much time down that rabbit hole when I know the code 'works'. It is probably quicker² (and easier) to extend the existing E2E / Integration tests to cover the new behaviour. Plus, if there isn't any, observable change in the DOM, why bother including it in a Sociable Test at all?

Too often I see what could easily be Solitary Unit Tests written with all of the boilerplate of Sociable Tests without actually interacting with the Component Template. So the tests have none of the 'benefit' of a Sociable test but have all of the downsides of them. I've found by simply removing the ComponentFixture etc., the tests run up to 10x faster.

Final note: I was taught that if you're polluting the production code just so you can test something (think of the data-testid attribute for one, or even simply changing the scope of a method to make it 'easier' to test), then you maybe doing something wrong or unconventional.³

If you're not, then you should be. If you are performing real E2E tests against real APIs, then (👏) it shouldn't be too troublesome to reuse these tests to also execute against stubbed APIs (if you even need to bother). ↩
To write, not to run. ↩
That's a whole other conversation however. ↩

The Venn of Angular Component Testing: Clearing the Grey

Mark Blandford — Sun, 29 Oct 2023 09:24:52 +0000

If you haven't read the previous parts to this series, please take a look as this provides some important background and context to the rest of this post.

Sociable vs Solitary Unit Tests

I think it was Jay Fields that coined the term Sociable & Solitary Tests. Martin Fowler has a good article that mentions the concept too.

Solitary Tests prefer to isolate the unit under test.

Sociable Tests often rely on other units to fulfil the behaviour.

I think this more subtly describes the different approaches to 'unit testing' Angular components most Angular engineers take. Think about this too:

[A Solitary Unit Test] Never cross boundaries

-- 'Working Effectively with Unit Tests' by Jay Fields

I would consider the interface between the Component Template and the Component Class a boundary. Thus Component DOM Tests must be Sociable.

Distilling this down further, I think this leads us to:

🧩 Component Class Tests = Solitary Unit Tests

🤝 Component DOM Tests = Sociable / Integration Tests

I fall firmly into the Solitary camp of unit testing approaches: I can probably count on one hand the number of Angular, Sociable Unit Tests I have ever had to write.

My predisposition probably comes down to my development background. For example, it's simply not possible to test a WPF UI (or View) binding / interaction with the 'class' (View Model), with the unit test framework, NUnit. Furthermore, the 'View' should never have any logic anyway. I would argue, for the large part, it should be the same for Angular Templates too!

In addition, the Sociable Tests exhibit traits similar to Integration tests for example: fragility & speed. To better describe this, look at this basic example.

An Example comparing Solitary & Sociable Angular Component Tests

Take this Angular component as an example.

@Component({
  selector: 'app-root',
  template: `
    <button (click)="increment()">Increment</button>
    <p>{{ counterValue }}</p>
  `
})
export class AppComponent {
  @Input() counterValue: number = 0;
  @Output() counterIncremented = new EventEmitter<void>();

  increment() {
    this.counterValue++;
    this.counterIncremented.emit();
  }
}

And here are the tests, written in the two different approaches:

Solitary Unit Tests

  describe('AppComponent - Solitary Unit Tests', () =&gt; {
    describe('when the button is clicked', () =&gt; {
      it('should increment the counter value', () =&gt; {
        const component = new AppComponent();
        component.counterValue = 0;

        component.increment();

        expect(component.counterValue).toBe(1);
      });

      it('should emit an event', () =&gt; {
        const component = new AppComponent();

        let emitted = false;
        component.counterIncremented.subscribe(() =&gt; {
          emitted = true;
        });

        component.increment();

        expect(emitted).toBe(true);
      });
    });
  });

Sociable Tests

  describe('AppComponent - Sociable Tests', () =&gt; {
    let component: AppComponent;
    let fixture: ComponentFixture;

    beforeEach(() =&gt; {
      TestBed.configureTestingModule({
        declarations: [AppComponent],
      });

      fixture = TestBed.createComponent(AppComponent);
      component = fixture.componentInstance;

      fixture.detectChanges();
    });

    it('should display the initial counter value', () =&gt; {
      component.counterValue = 5;
      fixture.detectChanges();

      const displayedValue = fixture.nativeElement.querySelector('p').textContent;
      expect(displayedValue.trim()).toBe('5');
    });

    it('should increment the counter when the button is clicked', () =&gt; {
      const button = fixture.nativeElement.querySelector('button');
      button.click();

      fixture.detectChanges();

      const displayedValue = fixture.nativeElement.querySelector('p').textContent;
      expect(displayedValue.trim()).toBe('1');
    });
  });

Notice how the Sociable Tests are subtly different - A Sociable Test wouldn't normally test the counterIncremented event is dispatched. That would be on the responsibility of the parent 'subscriber' (which in turn shouldn't need to know under what circumstances it is emitted).¹

Where as the Solitary Unit Tests can be said to provide 100% coverage, the Sociable / Component DOM Tests can not. If you just had the Sociable Tests, you could delete the this.counterIncremented.emit(); line, which could be critical, and yet no tests would fail. 🙁

Given these examples, let's outline the key differences:

Key Differences: Solitary vs Sociable Tests

Aspect	🧩 Solitary Test	🤝 Sociable / Component DOM Test
Scope	🔬 Isolated Units	🌐 Component Interaction
Dependencies	🥸 None / Mocked	🎨 DOM & Angular Change Detection
Purpose	📐 Component Class logic	🔄 Verify the Component Class and Template work together
Coverage	💯 High Unit Coverage	〽️ Lower Coverage
Fragility	💪 Highly Resilient	💀 More Fragile
Speed (100 runs)	🚀 2.2ms Avg	🐌 26.9ms Avg
On Failure	🎯 Easy to Pinpoint	🪨 Harder

Fragility

How many times have you added a new component, only to then find some, relatively unrelated suite of tests now fail or you see console warnings? Likely this is because another component test suite, imports an ancestor of your new component. When this happens, you're probably thinking "I know but that ancestor shouldn't care about my new grandchild component". I think you're right. It feels to me like, the ancestor test suite is more like an Integration Test suite, simply because it has a hierarchy of dependencies.

Sociable Unit Tests are more susceptible to cascading failures.

-- 'Working Effectively with Unit Tests' by Jay Fields

This all comes down to using, the TestBed or notably TestBed.configureTestingModule().²

On the other hand, the Solitary Unit Tests can not suffer this same fate: they have zero knowledge of any descendants.

Speed

Remember Unit Tests are meant to be fast. They should enable an engineer to be able to run them often, ideally after any change they make. Running two, Sociable Unit Tests in an average of 26.9ms is pretty good right? But how about running tests, that give you more coverage and greater confidence but 12 times faster? Just scale that up to when you have over 2,000 tests. That's the difference between the entire test run taking 27 seconds (which is still really fast but these are very simple tests) vs 2 seconds! That's what Solitary Unit Tests give you.

If Unit Tests are meant to be fast, then I'll take running 2,000 of them in ~2 seconds please.

Furthermore, I think most engineers find it quicker and easier to write Solitary Unit Tests than they do Sociable Unit Tests (Component DOM Tests), plus maintain the component dependencies in the tests.³

Closing

In the final post in this series, I'll introduce the 'The Venn of Angular Component Testing'.

You could of course wrap the component in the test with a dummy component but you are 100% then into Integration Test territory (as far as I'm concerned). Not to mention introducing additional complexity into the tests which is just something else that can break / fail your tests. ↩
Some of this can be overcome using mocking frameworks such as ng-mocks but I struggle to then understand the value in the added boilerplate compared to simple Solitary Tests instead. ↩
How many console warnings ("component A is not a known element") do you see when you run your tests? Potentially hiding the warnings you really care about. ↩

The Venn of Angular Component Testing: The Grey Area

Mark Blandford — Sun, 29 Oct 2023 09:06:05 +0000

In my experience there is confusion between what I would consider a Unit Test and an Integration Test for an Angular Component. Drawing on my OO development experience and having practised Test-driven development for a number of years in Angular & Java (plus other languages & frameworks), I think I can help reduce the 'grey' in the landscape.

This series is limited to just testing of Angular Components. I feel the concept of Unit Testing other Angular objects, such as Services is more well defined.

To Start

A Definition of a Unit Test

Asserts a unit of software works as expected within isolation

A Definition of an Integration Test

Asserts a unit of software interacts as expected with another

What is the 'Unit' of a Unit Test?

The smallest piece of software where, for a given change to an input there is an observed (to the software / unit under test) change in output. For example a public method, which for a given input leads to a predictable, visible output (to the software / unit under test).

A Comparison between a Unit Test & Integration Test

Aspect	🧩 Unit Tests	🤝 Integration Tests
Scope	🔬 Isolated Units	🌐 Component Interactions
Dependencies	🧪 Mocked Dependencies	🛠️ Real or Mocked
Purpose	✅ Isolated Code	🔄 Component Integration
Coverage	💯 High Unit Coverage	🚀 Critical Integration
Fragility	🪨 Resilient	🍌 More Fragile
Speed	🚤 Fast	⛵ Slower
On Failure	🎯 Easy to pinpoint	👓 Potentially harder

In summary, Unit Tests target small, isolated units of code without external dependencies, while Integration Tests examine the interactions and interfaces between components or services.

How do these Relate to Angular Component Testing?

I think this where the lines blur between what [I would consider] a unit vs an integration test is. An Angular Component consists of:

An HTML template that declares what renders on the page

A TypeScript class that defines behavior [sic]

A CSS selector that defines how the component is used in a template

Optionally, CSS styles applied to the template

-- Angular Guide - Angular components overview

I ask you this question:

What is 'Isolation' or the Unit Under Test in the Context of an Angular Component?

I think if your test focuses on the class of a single Angular component in isolation, it's more likely to be a unit test.
If your test involves rendering multiple Angular components together, in a more real-world scenario, and / or interacting with the DOM¹, I think it is probably an Integration Test².

An example

For example, testing the click of a button on a component:

component.onClick(); (assuming this is what the onclick event handler calls)
- Type of Test? 🧩 Unit Test
- Why? Interacting directly with the public interface of the class / unit under test.
fixture.nativeElement.querySelector('button').click();
- Type of Test? 🤝 Integration Test
- Why? The test interacts with the DOM, so likely needs to render other dependencies and components. One could argue the DOM itself is a dependency³.

But that's not an Integration Test, it's a Component DOM Test!

You're spot on. If you read the Angular Guide for testing Components, the docs call the tests 'Component Class Tests' and 'Component DOM Tests', no where does it mention unit testing⁴. Perhaps the terminology we use day-to-day is wrong? Either way I think this leads to a grey area of what we refer to as 'unit testing' Angular Components. Moreover, here is further definition of an Angular Component:

...a component is more than just its class. A component interacts with the DOM and with other components. The class-only tests can tell you about class behavior [sic]. They cannot tell you if the component is going to render properly, respond to user input and gestures, or integrate with its parent and child components.

-- Angular Guide - Component DOM testing

So if Component DOM Tests are to address this 'limitation', that must mean the Component DOM Tests need to have dependencies and 'integrate with its parent and child components'? Sounds pretty close to the definition of an Integration Test to me.

I felt there had to be a cleaner, more succinct way to draw the line between Component Unit Tests vs Component DOM Tests vs Integration Tests. This is where I went back to my books⁵ and considered Sociable and Solitary tests.

Closing

In the next post, we'll explore more about Sociable and Solitary testing and why this concept could be a solution to the 'grey area', and how we describe and talk about Angular Component testing.

JSDOM is a 'real' DOM: ↩

jsdom is a pure-JavaScript implementation of many web standards, notably the WHATWG DOM...
The Angular docs call these Component DOM testing. I think they are just another type of Integration Test, more on that later. ↩
I suspect if you were to run Jest without JSDOM, this sort of interaction would fail. ↩
On the Basics of testing components page anyway, and I cannot find anywhere where the Angular framework / team makes the link between Component tests (class or DOM) and unit tests. However, in the documentation for ng test, Angular describes command as "Runs unit tests...". ↩
'The Art of Unit Testing' by Roy Osherove ↩

'Working Effectively with Unit Tests' by Jay Fields

'Test-Driven Development By Example' by Kent Beck