Forem: Alan Richardson

A Web Testing, Automating and Tooling Masterclass

Alan Richardson — Thu, 05 Feb 2026 08:30:00 +0000

TLDR; We can only test to the level supported by our Ability, and the degree to which we are supported by tooling to Observe, Interrogate and Manipulate the System.

Video

Watch on YouTube

00:00 Introduction
00:26 Application Under Test
02:51 Live Testing
03:41 Testing vs Automating
06:03 Testability vs Automatability
09:25 Workarounds
10:55 Technical Knowledge
15:44 Test Approach - JavaScript
17:02 Exercise - Test Approach
17:24 Automated Execution Approach
18:34 Exercise - Code Review
20:35 Exercise - Your Project
20:59 Server Side
24:20 Exercise - Test Approach - API
24:52 API Tooling
29:14 Exercise - API Testing
29:34 Automating the API
31:54 API Interacting with UI
37:09 End Notes

Introduction

A customer has reported that one of your pages doesn’t work. But your automated execution coverage tells you that it does. What went wrong?

That’s what we’re going to explore now.

difference between testing and automating,
testability vs automatability,
we’ll think about test coverage from a technology perspective,
and we’ll look at some tools that can help you test.

Here’s the Application Under Test.

https://testpages.eviltester.com/challenges/synchronization/infinite-scroll/

Infinite Scroll Challenge on the TestPages

Many sites have infinite scroll. You scroll to the bottom and it loads new items.

That functionality works. We have have automated coverage. That passes.

What’s the issue?

Oh yeah, sorry I forgot to mention. I’m going to encourage you to think. Ideally we would do this as a workshop, as hands on training, but it’s a video and text workbook.So all I can do is prompt you into thinking.

Contact me if you are interested in live training for your team or organization.

So what’s the issue?

Well, I can’t click the button. The page refreshes and scrolls too quickly and the button isn’t enabled for long enough when it is visible.

This happens with a lot of infinite scroll sites, there are footers that can’t be clicked, stuff at the bottom of the screen that you can’t see.

This did work at some point, because we tested it, but I guess someone changed the time out value and the automated execution assertions didn’t flag this as an issue.

So let’s start thinking about this from a Testing perspective.

Live

If this was a live project we could just stop here and raise a defect.

Obviously the timeout is wrong. Raise the defect. Move on to the next issue.

But wait.

We have automated coverage, and it didn’t highlight this issue.

We should think about the difference between Testing and Automating.

Testing vs Automating

Testing is what we do. As humans, we interact with the software and we observe, we build models, we learn, we compare the actual interaction with the models. We report information derived from the difference between our models and the observations.

When we investigate an issue and expand our models we interrogate the system more deeply to learn what’s going on.

Testing is the human activity of interacting, observing, thinking, learning, experimenting.
Automating, is the human activity of making the interaction and observation of the system an automated process. And that includes the comparing of observed results with the expected results.

Automating is a human activity which results in the output of an automated execution process. The only involvement with the human after the activity has been automated is to investigate the failure reports and maintain it when it fails.

So both Automating and Testing are human processes. But testing leads to more human processes, automating leads to an automated process that only involves humans when it fails.

NOTE: AI might change how I view the process of automating. Certainly I’ll need more distinctions in how I describe nuances around Automating, but for now, all the automating I do is Human Initiated, or Directed, and results in some automated execution process.

Testability vs Automatability

And let’s just quickly look at Testability and Automatability because these words are used badly when describing software.

Is this system testable?

Yes. I can access it in the browser, I can see it, I can interact with it. The button toggles too quickly but I can test the application.

But when we talk about testability we often talk about, adding ids to the elements, make it it more observable, etc.

Well the ids, are really for automatability, not testability.

In the browser, the browser tooling makes it observable because of the technology used, not the system.

Most of the time when we are talking about testability we are really talking about automatability.

This application has been built with automatability in mind.

It has ids, there are classes, the JavaScript source is visible, it is easy to change the state and configuration variables, this thing is so easy to automate and observe. But none of that was required to help me test it.

Testability is not the same as automatability. Keep that in mind as we continue through this process, particularly when we move to the server side interaction.

With the JavaScript Infinite Scroll system, the main thing that impacts my Testability, is the bug, preventing me from testing the button functionality.

Workarounds

Let’s quickly consider workarounds.

Your ability to find workarounds will impact your ability to test the system.

So your ability impacts the Testability.

Testability is as much about your ability to test the application as it is the application supporting you in testing it.

Depending on how deep you want to go. You can only test the application to the limit that you can observe, manipulate and interrogate the application.

Your ability to test the application is often impacted by the usability of the application, and that is true here. The application is not usable for one main function so it is hard to test that function.

But, we can work around that, for this application, with more Technical Knowledge.

Technical Knowledge

When working with the web, we need to understand our tool capabilities.

My tool at this point is the browser.

What can I do with it?

I can view the page source.

I need to be able to understand HTML. Some CSS. Some JavaScript.

Having the tooling ability to look at the source doesn’t help me unless I understand what I’m looking at.

So if you’re testing web applications, you probably want to understand:

How a browser works,
How HTML works,
What is CSS and how we use it,
What is JavaScript and how it works.

At the very least to have a reading ability of these technology artifacts.

So at this level, at the source, with the technical knowledge I have… I can see in the JavaScript some variables, I can see they are amendable.

But, I don’t have the tooling to amend it yet.

So, what else do I have available?

I can look at the dev tools.

I can see the DOM view shows me much the same as the source. But… it can be different.

The source is what we gave the browser to work with.
The DOM is what the browser created after interpreting and executing the source.

In the DOM I can see the JavaScript as well.

I also have the ability now, to interact and manipulate the JavaScript variables that I observed in the source.

There are a few timeout and millisecond variables there.

Perhaps the bug is the scrollAfterMillis variable?

Let me change that to 3000.

scrollAfterMillis

And… now it just takes longer to refresh, but I still can’t click the button.

Let me try the preLoadTimeout:

preLoadTimeout=2000

Now, the button stays active for 2 seconds before it loads the next set of data. That’s my workaround to make this application testable.

It also means that I can go a little deeper in the bug report and say that the root cause is the preLoadTimeout being too small a value so the user doesn’t have time to click the button.

You might be interested in this video and blog post covering more details of Chrome Dev Tools for Testing.

Test Approach

So what is my test approach for this?

load the page
scroll down
see that it refreshes and adds the data I expect
click the button
see that it doesn’t refresh when I scroll

Is that it?

Do I need to reload the page and check that it starts auto refreshing again?
How long do I need to wait after stopping it, and trying to scroll to make sure it doesn’t start auto-refreshing again?
What else?

Exercise - What is your test approach?

After this video I encourage you to think through what your Test Approach for this application would be.

Test Approach vs Automated Execution Approach

Once you’ve figure out how to test it, how would you automate it?

I’ll show you what we have here.

And this test passes.

@Test  
public void scrollToStopLoadingAndClick(){  
WebDriverWait(driver, Duration.ofSeconds(10)).until(  
        ExpectedConditions.elementToBeClickable(page.getStopLoadingButton()  
    ));  

    page.getStopLoadingButton().click();  

    new WebDriverWait(driver, Duration.ofSeconds(10)).until(  
        ExpectedConditions.textToBePresentInElement(  
            driver.findElement(By.id("statusMessage")),"Clicked")  
    );  
}

Is that good enough?

It works. It passes.

But it didn’t highlight the fact that the halting functionality is unusable for a human.

This is one of the issues when we automate something.

We automate the functionality.

We don’t automate the experience.

If you are interested in more details about WebDriver with Java then have a look at this video Masterclass on the basics of WebDriver with Java

Exercise - Critique the code

So after the video, or now, pause the video.

Critique this code:

Understand what it does
What does it not do?
- I can see it doesn’t try to re-scroll the page to make sure that clicking the button stopped the auto-scroll.
- What else does it not do?
- What conditions does it not check?
Should it match the user experience?
- How could it?
- What would you change to match the user experience?

Much automated execution coverage does not match the user experience. When it doesn’t then there is a risk that the automated execution passes, but the user experience doesn’t.

And there is a risk we don’t notice.

Testing will hopefully identify those issues. But automated execution often does not.

Exercise - Consider Your Project

Is there a risk on the projects you work with that the automated execution covers the functionality, but not the user experience of that functionality?

Server Side

OK, so that was the JavaScript version of the Infinite Scroll.

It offered us some scope for thinking like the tester.

But let’s push this a little further.

We also have the server side infinite scroll.

https://testpages.eviltester.com/challenges/synchronization/infinite-scroll/server-side/

It looks the same. It has the same bug, the button is too fast. And my automated execution passes.

But this page, when it needs new items to display it calls the server to get the information back.

Do you trust that statement?

How do you know it calls the server?

Visibly, in the browser, when I test it and observe the output. It looks the same.

We need to be able to be able to interact with the application from a technical perspective.

What if I just think it is connecting to the server, but I accidentally release the wrong version? What if I just renamed the file?

So we need to check.

We need to observe the application at multiple technical levels.

So let’s look in the dev tools again. And this time we’ll look in the network tab.

Filter by Fetch/XHR to see the requests made by JavaScript and I can observe a call to moreitems

Let’s interrogate that request

I can see a JSON response:

[
    {
        "id": 1,
        "text": "This is content item number 1. Scroll down
         to automatically load more content when the
          \"Click to Stop\" button becomes visible."
    },
    {
        "id": 2,
        "text": "This is content item number 2. Scroll down
         to automatically load more content when the
          \"Click to Stop\" button becomes visible."
    },
    ...
]

Great.

So I have tooling to increase my ability to test this.

But I need to know about:

HTTP Requests
Fetch and XML Http Request
JSON

My ability to test this application will be limited if I do not have that technical knowledge and if I do not know how to use the Dev tools network tab.

But at least I know now that it is making server requests.

I didn’t have to trust anyone. I could verify that this was true.

So now, our testing scope just expanded.

Exercise - Server Side Test Approach

Now, what do you have to test?

Is it good enough to just test the front end now but scrolling up and down and clicking the button?

After watching this video, take some time to think through what you want to test to cover the Server Side infinite scroll.

Do we also have to test that HTTP call?
How much do we have to test it?

Server Side Test Tooling

And… do you know how to do that?

How can you amend the HTTP requests?

We can do some of that from Chrome.

I could:

copy as cURL
copy as fetch

with cURL I can use the command line.

Or fetch I can amend it in the console.

I could take the cURL and paste it into an API tool.

I can do that in Bruno by creating a new request from cURL.

Or in postman I can paste in the cURL command into the new request.

I can experiment with it in these tools.

It is important to test the API on its own.

For example, when I built this, it was only when I was automating and testing the API, that I realised that I really need a limit on the count. If I let people make a request and ask for 6 million items back. That could easily bring down my server.

NOTE: you can find a list of API Tools on the API Challenges site.

Exercise - API Testing

As an exercise, think through what conditions you would want to test on the API, then you can use any of the tooling approaches mentioned to experiment.

Automating The API

I would probably want to automate the API as well.

This does remove us from the user experience because the responses from this API are normally handled by the UI, so just because we see something working in the API we can’t assume that it works with the API interacting with the UI.

I used RestAssured to automate the API.

The coverage runs at the same time as the web UI coverage.

And I have a lot more coverage at the API level, than I do at the UI level.

Think about what coverage you would add for the API.

Then try to automate it. I used Java, with RestAssured, but you could use any library or programming language you want.

Or you could even have a set of canned requests in Postman or Bruno or any of the other API tools.

For production API automating, I primarily use code and HTTP or API libraries. I don’t tend to use tools like Bruno or Postman.

But for production API Testing, I do use Bruno and other tools. Because I like the flexibility and the ability to see the requests being made.

Testing and Automating often sound like the same thing. But they have different aims and are supported by different tooling.

API Interacting with UI

And now I’m in the position where I’m testing the API in isolation of the UI.

Is that a risk?

For example, I don’t know how the front end handles a 500 error response. I can see it is the same JSON format but it is a different status code, does that make a difference?

[
    {
        "id"=0,
        "text"="For input string: '-1.02'"
    }
]

Would you test that?

Do you know how to test that?

One way to do that is to use a Proxy.

Intercept the request, amend it to be one that triggers an error and play it through to the front end.

And then I can see if the system handles error responses or not.

I would either use Zap or BurpSuite.

For this exercise I would use ZAP.

open a session using Edge
create a new context for testpages.eviltester.com
filter the history to “Show only URLs in Scope”
Set the breakpoint on all requests and responses
trigger a scroll through the UI
amend the request
see the result in the UI

I can use tooling to increase my ability to observe the system and manipulate the system at different technology levels.

NOTE: A list of recommended HTTP Proxy Tools is available on the API Challenges site

End Notes

It is often surprising how much depth we have to test and automate when testing even simple pieces of functionality.

The more that we extend our technical ability to cover the multiple levels of the application, and we learn how to use tooling to help us observe, interrogate and manipulate at those different technology levels, the more we can expand the coverage of our testing.

Automating is not the same as Testing. Both are human processes, but the output of automating is not the same as the output of testing.

But yes… we can be testing, as we are automating, we may well learn things during the process of automating the application. But we should not confuse the continued execution of the output from automating with testing.

Testing can miss things because we are human, or we may not have covered the conditions, or we may not have gone deep enough into the system.

Automated execution can miss things because we humans forgot to add the conditions, or we didn’t assert enough. But automated execution can also miss the human user experience and tell us things are working, when they clearly are not.

I hope that you do now go off and do the exercises yourself. You may not have all the skills to do this yet, you may not have tried all the tools. That just means you can revisit this exercise and application multiple times as you grow your knowledge and skill set. And repeat it when you want to evaluate or learn new tools.

That’s why I created the Test Pages, and that’s also why this is a fairly high level overview.

Remember to work through the exercises.

Exercises

The Applications Under Test

Exercise - What is your test approach for JavaScript Infinite Scroll?

think through what your Test Approach for the JavaScript Infinite Scroll application would be.
Do you need to reload the page and check that it starts auto refreshing again?
How long do you need to wait after stopping it, and trying to scroll to make sure it doesn’t start auto-refreshing again?
What else?
- what conditions would you cover?
- how would you approach the testing?

Exercise - Critique the code

Understand what it does
What does it not do? - I can see it doesn’t try to re-scroll the page to make sure that clicking the button stopped the auto-scroll. - What else does it not do? - What conditions does it not check?
Should it match the user experience? - How could it? - What would you change to match the user experience?

Critique this code:

public class InfiniteScrollTest {  

    static WebDriver driver;  
    static InfiniteScrollPage page;  

    @BeforeAll  
    static void setupWebDriver(){  
        driver = DriverFactory.getNew();  
        page = new InfiniteScrollPage(driver);  
    }  

    @BeforeEach  
    public void reload(){  
        page.open();  
    }  

    @Test  
    public void scrollToStopLoadingAndClick(){  
            ExpectedConditions.elementToBeClickable(page.getStopLoadingButton()  
        ));  

        page.getStopLoadingButton().click();  

        new WebDriverWait(driver, Duration.ofSeconds(10)).until(  
            ExpectedConditions.textToBePresentInElement(  
                driver.findElement(By.id("statusMessage")),"Clicked")  
        );  
    }  

    @AfterAll  
    public static void closeDriver(){  
        driver.close();  
    }  

}

Supporting Abstractions:

public class DriverFactory {  
    public static WebDriver getNew() {  

        WebDriver driver;  
        ChromeOptions options = new ChromeOptions();  
        options.addArguments("--disable-smooth-scrolling");   
        driver = new ChromeDriver(options);   
        return driver;  
    }  
}

public class InfiniteScrollPage {  

    private final WebDriver driver;  

    public InfiniteScrollPage(WebDriver driver) {  
        this.driver = driver;  
    }  

    public void open() {  
        String url = SiteConfig.SITE_DOMAIN +
         "/challenges/synchronization/infinite-scroll/";  
        driver.get(url);  
    }  

    public WebElement getStopLoadingButton() {  
        return driver.findElement(By.id("loadMoreBtn"));  
    }  
}

public class SiteConfig {  

    public static final String SITE_DOMAIN = "https://testpages.eviltester.com";  

}

Exercise - Consider Your Project

Is there a risk on the projects you work with, that the automated execution covers the functionality, but not the user experience of that functionality?

Because if that’s a risk, you might want to revisit your test approach and your automated execution approach.

How might your approach need to change?

Exercise - Server Side Test Approach

What do you have to test now that the Server Side calls are involved?

Is it good enough to just test the front end now but scrolling up and down and clicking the button?
Do we also have to test that HTTP call?
How much do we have to test it?

Exercise - API Testing

As an exercise, think through what conditions you would want to test on the API, then you can use any of the tooling approaches mentioned to experiment.

Suggested Tools:

Browser Dev Tools - fetch requests in console
Browser Dev Tools - generate cURL and use from CLI
Bruno
Postman

A List of API Tools is available on API Challenges site

Exercise - UI and API Interactive Testing

Use a Proxy to allow you to observe and interrogate the traffic from the Web Site to the Internal API backend.
Amend the Request to trigger an error response and see how the front end handles it.
Try testing the API from within the proxy.

A list of recommended HTTP Proxy Tools is available on the API Challenges site

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

Test Interaction with HTML form fields

Alan Richardson — Thu, 15 Jan 2026 09:00:00 +0000

TLDR; When testing web apps we need to test the interaction between the browser implemented controls and our system. We don’t test the browser.

Testing HTML Form Fields

Watch on YouTube

Testing Web Technology

I like to use the term Technical Testing.

Many people don’t. I think other people view the term as implying that some people are technical and other people are not.

I use the term to cover “Testing which is informed by Technical knowledge”.

So if you’re testing a web application, you need to understand web technology, you need technical understanding of how browsers and the web works, otherwise you can’t test it effectively.

In this video I try to explore some of the reasons why this is important.

If I’m testing a web application and it has a form, and in the form is a date field which has a date control calendar pop up which lets me set a date.

What should I test?

Should I test that…

the calendar popup works on mobile and every web browser?
the calendar control works with keyboard?
the calendar control correctly switches years and months when I click the control buttons?

Well, no. Not if the HTML looks like this:

<input id="datetime-local-input" 
 type="datetime-local" name="datetime-local">

All of the complex UI interaction is implemented by the browser.

All our application does is use an off-the-shelf browser provided control.

Do I not test the functionality?

When we are using off-the-shelf controls like this we have to test the interaction between the control and our application.

If we added JavaScript event listeners to the control to perform extra validation or functionality then… yes we have to test that.

If we use the date entered in the control by the user, then we have to test that our application can handle the data supplied by the value of that control.

If we are not using the date then we should question why we have the control on our application UI.

Learn to spot technology risks

I can see from the HTML above that there might be an accessibility risk, because I don’t see any ARIA attributes associated with that control.

So we might not have configured the control in the page properly to implement all the accessibility requirements we might want.

If we had created a custom control, or it is specialised React control, then we do want to test it because we just introduced a technology risk that might impact mobile and different browsers. We might also have functional bugs.

But if we are using built in browser controls we don’t need to test as much functionality related to the control. We need to test the functional interaction with our application and custom code.

Test the Domain Configuration

We shouldn’t spend a long time testing validation messages from custom controls, but if a control has been configured then we need to test that the configuration is correct.

<input id="password-input" type="password" name="password"
 pattern="^[A-Za-z0-9_]{4,12}$" maxlength="14" required="true">

I don’t need to test that the password control above shows dots instead of letters and hides the input. But I do need to check that the configuration we supplied for the validation pattern and length are correct. And I need to make sure that the password is actually required (because we configured that it is).

NOTE: eagle-eyed readers, who have a technical understanding of how the configuration works will probably have spotted a bug in the above configuration. Yes. We should find that bug in our testing. And we know to look for it because we understand that the control has been configured with our application domain rules, and that’s what we are testing.

Don’t Trust the Client

The greater the technical understanding we have of the web technology, the less we will trust any data that the server receives.

We can edit the DOM to bypass domain configuration.

We can use JavaScript to enter data that bypasses browser validation.

We can bypass the browser altogether and just communicate directly with the server.

A greater technical understanding allows us to understand the risks of the technology more effectively

Testing Web Apps

All of the examples above are from:

testpages.eviltester.com

I also have a guide to testing HTML elements to describe some of the risks and issues that we might have when using or configuring off-the-shelf HTML fields and elements.

Testing Input Elements - Notes and Exercises

I had to learn web technologies when testing web applications.

I have seen people functionally test the browser implemented controls because they didn’t learn web technologies.

This means they wasted time, because they didn’t understand the technology they were testing.

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

New Version of CounterString Extension

Alan Richardson — Sun, 04 Jan 2026 09:00:00 +0000

TLDR; Generate CounterStrings, random data from Regex, ranges of data, and strings made by concatenating repeated data.

I released a new version of my CounterString generation Chrome Extension with new features and a new UI.

CounterString Extension

Watch on YouTube

Source and Download Links

I have a tool page for the CounterString extension with more details.

https://www.eviltester.com/page/tools/counterstringjs/

But you can find the tool:

Overview

I first released the CounterString Extension in 2019 (I think), and at that point it was a simple input field that created a string and added it as a value to the input field.

Now it can:

Generate CounterStrings
Configurable length and delimiters
Generate Random Data From Regex
Generate Character Ranges
Repeat text, characters and regex
Add generated data as value, with input event, to an input field
Trigger Key events to ’type’ the data with configurable speed
Binary Chop Calculator for field length exploration

Some of this was a result of migrating code from my Java Test Tool Hub (which I think I worked on about 12 years ago):

https://github.com/eviltester/testtoolhub

From the Test Tool Hub I took the:

Robot - I had to find a different way of triggering events, but this allows ’typing’ the data into the field rather than just amending the value.
Increased customization of the CounterStrings
Ranges of Data
Repeated Data
Binary Chop Range Calculation

And the Data from Regex was repurposing some code from my AnyWayData.com test data generator which uses the same JavaScript library to generate data from a Regex - RandExp

CounterStrings

I wrote about CounterString Algorithms in an earlier blog post.

This extension uses both reverse generation and forward generation.

I initially learned about CounterStrings from James Bach. James has a tool called PerlClip which is a command line tool to generate data directly into the clipboard. This uses a reverse algorithm where the full string is generated and then reversed to paste into a field.

A forward algorithm allows ’typing’ the String character by character without generating the full string first.

While I don’t generate the full String first, I do precompute a Schema which I then follow to generate the String, so it takes about the same processing time, but uses less memory and allows streaming of the result.

I created a different algorithm this time, which is much simpler to understand, but probably takes longer to execute. I’ll need to update the algorithm page at some point, but you can see the algorithm implemented in the source code.

github.com/eviltester/counterstringjs/blob/master/extension/js/generateSchema.js

Other CounterString Tools

A quick hunt around the web revealed a few more CounterString tools that I don’t think I had found all of these before. So these might be interesting for anyone thinking of creating their own implementation.

PERL: https://www.satisfice.com/download/perlclip
TYPESCRIPT: https://github.com/j19sch/counterstring/
RUBY: https://github.com/jamesmartin/counterstring
JAVA: https://github.com/eviltester/testtoolhub
PYTHON: https://github.com/deefex/pyclip
RUST: https://github.com/thomaschaplin/rust-counter-strings

Another Chrome Extension:

https://github.com/Pawel-Albert/utilities-for-testing-extension
- and some associated Test Pages that I haven’t seen before
- https://pawel-albert.github.io/utilities-for-testing-extension/

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

Software Testing Podcast - AI Optimism or Pessimism - The Evil Tester Show Episode 028

Alan Richardson — Thu, 18 Dec 2025 09:00:00 +0000

Rather than retrospect on the year in general. This podcasts looks at how I’ve been approaching the learning of AI. Where the industry has gone wrong and what to look forward to with AI.

audio links and subscribe here

AI Optimism or Pessimism

I sat down to think what am I optimistic about, and what am I pessimistic about with the Software Development industry’s use of AI.

I see a lot of positive benefits for AI use in the Software Development Process.

There are many risks involved with adding AI into products and we should rightly be concerned about those and try to identify how to test AI enabled applications properly.

Many of the statements concerning how Software Testing has to evolve concern me. e.g. more technical skills, more focus on risk, point out flaws in process, identify architecture risk. I thought Software Testing already did this. So perhaps one good thing from AI is an evaluation of what part Software Testing plays in the Development Process and what are the fundamental skills and knowledge we need in order to Test software effectively.

AI Productivity

I’m incredibly optimistic about what I can personally do with AI. It’s massively increased what I can get done, especially in programming, content creation, and marketing.

But companies are cutting jobs and blaming AI. But companies have always looked for scapegoats when trimming staff AI is just the latest excuse.

If leadership truly saw their people as the reason for their growth, they’d use AI to amplify productivity and expand the team’s capabilities, not use it as a reason to shrink down.

“If companies viewed the staff as a core engine for growth, they wouldn’t want to get rid of people when AI comes in. They’d want to use AI to make the staff more effective and grow the company. Clearly they don’t.”

Many management teams see staff as a cost center, not as an engine for growth. Sales gets viewed as the engine of growth and is often paid on commission, so it seems cheap. The product is a ’thing to be sold’. And management… are always essential.

Some organizations don’t think “how do we help our people do more?” They take the position: “how can we do the same (or less) with fewer people?”

AI is the scapegoat. Not the cause.

Short-Term Pessimism

AI is overhyped. Leadership decisions are being made based on hope, not the technology’s actual capabilities.

In the short term, I’m a bit pessimistic, not about AI itself, but about how people (management) are choosing to react to it. Too many seem to want to use AI to restrict and constrain, instead of to grow.

But our work is already changing. New tools exist. We can’t afford to be complacent. We need to experiment and figure out where AI makes sense.

How Should We Use AI?

Not every product needs AI enablement and features. Too much risk, particularly if you do not know how to test the results.

I do NOT use AI for direct human-to-human communication.
- If you get an email from me, I wrote it. If I get one from you, I read it, not an AI.
I would not use AI for company procedures or performance reviews.
- Adding AI to these human, recognition-based workflows is a mistake.
I won’t use AI as a buffer between people.
- Don’t use AI to hide. Engage.

I do use AI to summarize public podcasts, blog posts, etc, Stuff I choose to consume, but not personal, direct communication.

Management is about people, and putting an AI buffer between you and your team leads to bad communication and alienation.

“Your boss should already know what you’re doing. They should be helping you explain the value that you’ve added. This is a human process of recognizing value. It’s not something that we add AI into the middle of.”

How I learned AI

When I started playing with AI, I created a podcast summarizer. My process for learning anything new is to find a project and use the new technology to build something.

I wanted to consume more podcasts but couldn’t keep up. So, I wrote a tool using:

Hugging Face libraries
Whisper AI for transcription
Different LLM models for summarization
Prompt engineering
Ollama for running models locally
OpenRouter for connecting to cloud-based models

I did everything locally as much as possible, partly to learn the limits before relying on cloud or paid models. Only after learning the limits did I see real value in some of the paid services.

Don’t believe in ‘magic’ from big models until you know what smaller, free models can actually do.

Step By Step Adoption

Step 1: Chat Interfaces

At first, I used AI the same way most people, from a chat interface, like ChatGPT or Claude. It worked but had limits. I wanted something embedded into my IDE.

Step 2: Embedding into the IDE

Now, I use a plugin called Continue, which works in both IntelliJ and Visual Studio Code.

Initially connected to Ollama (local code models)
Later I Switched to OpenRouter for bigger cloud models

So now, I can:

Jump into a chat interface via my IDE
Still use chatbots like Claude/ChatGPT for generic things
Never look at Stack Overflow anymore, everything’s in the IDE chat

Step 3: Agentic Coding

I started using agentic tools, like OpenCode, on the command line. They can scan my whole codebase, not just what’s open in the IDE.

I use it to:

Create a page object model for a URL
Write a test for that model
Generate K6 Scripts
Create swagger files
Write scripts
Write API Abstractions
… so much

Now instead of:

Writing the test
Building abstractions and models manually
Iteratively improving

I let the AI generate code, then I come in and change or adapt what I need.

“I’m using it as a tool to help me create things faster. Then I come in and use my knowledge and experience to review the code or expand it.”

Step 4: OpenSpec for Better Requirements

OpenSpec helps generate and maintain evolving and up to date requirements documentation.

Specs are continuously updated as requirements change
OpenCode has documentation to work from, in addition to my prompts

When I use OpenSpec, I’m letting the Agentic Coding Assistant write even more code, than when I use it iteratively at the CLI.

This has helped me:

write a new application faster
Keep requirements and documentation in sync
Automate coverage incrementally

I’m not sure how well it would scale to a full team, but it works well for a solo dev.

But…

AI sometimes makes strange design choices
If you don’t review and refactor, your codebase can fall apart fast
Human review is still necessary

AI saves time, but still requires fundamental core skills to evaluate their output and fix them when they go wrong.

I use AI for programming all the time, but not for actual testing.

Agentic AI in Testing

I’m beginning to look into Agentic AI tools specifically designed for testing.

I’m currently experimenting with AQE Fleet from Dragan Spiridonov

Pros and Cons

Right now, I am optimistic about AI empowering individual development team members. It makes us faster, and the tools are getting better.

But I am pessimistic about the way companies choose to adapt to AI. Focus on headcount reduction instead of increased effectiveness and growth.

What is Testing?

“I keep seeing posts that testers need to evolve into quality engineers and risk analysts and customer experience advocates because we’ll be reviewing the systems for risk more than we’re and testing them… but I thought we already did those things.”

People keep saying testers now need to be risk analysts, customer experience advocates, technical experts, etc. But that’s what good testers have always done. It almost feels like AI is forcing companies to rediscover what software testing really is, and always has been.

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

Free Technical Web Testing Course Released To Youtube

Alan Richardson — Thu, 18 Dec 2025 09:00:00 +0000

TLDR; Technical Web Testing course has been released to YouTube as a 6 hour video.

Free Technical Web Testing 101 course

Watch on YouTube

Slides etc. can be found via the course page

Practice Sites Now

The sites I use in the course are a little old.

If you want to practice with the techniques in the course then I maintain lists of sites to practice with here:

Course History

The course was created in 2013 or 2014. Originally as a free course, then I added a bunch of content and it turned into 6 hours plus 3 hours of additional webinar content.

Every course provider I hosted it on has changed terms and conditions multiple times making it painful to maintain and support.

Since the examples and videos are now showing their age, and one course provider changed their terms and conditions so much that I now no longer have access to their course platform and can’t support it.

I decided to make the course free.

The course is too big to make free on Udemy so I’ve combined all the videos into one and made them free on Youtube.

Thanks to everyone who supported the course over the years.

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

An overview of the Basic Shopping Cart Practice Web Page

Alan Richardson — Mon, 15 Dec 2025 11:19:00 +0000

TLDR; an overview of the basic shopping cart that you can use to explore, automate through the UI or REST API and perform security testing on.

The Shopping Cart Application offers a lot of possibility for testing due to the multi-state and multi-page nature. With the added bonus of a REST API which is documented using OpenAPI spec there are even more ways to interact with it.

Find the Shopping Cart Application Here

Overview Video

The video provides an overview of the shopping cart:

Watch on YouTube

Shopping Carts for Practicing

This is not the first Shopping Cart practice Application.

I know of:

SwagLabs Saucelabs demo application.
Toolshop from this has multiple versions
Restful Booker could be classed as a shopping cart
and my Basic Shopping Cart

SwagLabs is pure JavaScript so there are no HTTP calls, but you can explore and test the interaction with local browser storage, and the various users that you can login with change the display so you can use it for visual testing and admin access.

Toolshop, Restful Booker and my Shopping Cart all have APIs and the data is cleared regularly. I think Restful Booker clears data every 15 minutes. Shopping Cart clears data on a rolling basis by limiting the amount stored, so if there are a lot of users at any one time (about 200+), you might notice your old orders clearing from the backend.

When I was designing the Basic Shopping Cart I very deliberately chose to make sure that there was no possible way of having any PI data in the system. This meant that the checkout process does not ask for email addresses, credit card details, or even passwords. The users are ‘canned’ with pre-defined passwords. This means you can perform security testing without any issues.

Really you shouldn’t add any PI data to any testing practice site, but when applications use forms, the auto-complete might be switched on, and your details might be auto-populated into the form.

I do like the other Shopping Carts and have used all of them when practicing my testing.

Why A Shopping Cart?

Shopping Cart applications offer a lot of scope.

There tend to be multiple pages offering navigation scope.
Pagination if there are enough products.
API and server side interaction.
Checkout Process has state - tracked locally via cookies or browser storage, and possibly server side as well. Having state in two places offers scope for state moving out of sync and creating mismatch between server and client.
Usually a registration or login process. Which also lends itself to security testing.
Tend to be content rich.

Content rich applications are interesting. The first time we automate them we tend to hard code product names and ids. Then we realise that there is usually a CMS (Content Management System), and users can edit content, which then causes our automated execution to fail.

This might lead us to using the API and UI as a consistency Oracle for content. Does the API serve data consistently. Is the data rendered correctly. So we don’t automate or test exclusively through the UI, we use the UI and API in combination when we want to assert on data conditions.

Creation of the Shopping Cart

To build the application I started API first. The API was completely hand coded. This meant that I could test the flows and the functionality before spending any time on the UI. I was able to experiment with various authentication flows and data representations until I had one that I thought was rich enough and removed any PI risks.

Once I had the backend covered I used OpenCode to incrementally add basic REST API automated coverage, and then expanded the coverage.

Frond End Creation

I wanted the front-end to be simple JavaScript, HTMl and CSS, and avoid the use of frameworks.

My Test Applications are designed to be entry points. They will be challenging enough naturally, so I didn’t want to clutter the learning effort for people practicing by using React.

The Front End was creating by prompting through OpenCode and then I would amend and refactor where necessary. This allowed me to create the front-end quickly. But because I prompt in very small increments the code is more closely aligned with the style of coding I use, and makes it easier for me to amend in the future.

The models used were spread across free models from OpenCode Zen.

Grok Code Fast 1

Or I used Kat Coder Pro (free) on OpenRouter.

I try to use a mix of models in the CLI, and then a different set of models in the IDE. In the IDE I have Continue configured to use OpenRouter models.

Creating Content

Al the content was generated via AI tooling.

I prompted GTP-OSS-20B via OpenRouter to create 100 ’thing’ name variations. I fed it the sample ’thingy’ words and adjectives. I could have varied this more, and I could easily have used a normal combinatorial algorithm to do this but the AI interface was fast enough.

With a set of 100 names, I then asked for one line descriptions, using the same process. The are all output as JSON to make it easy for me to amend in the code.

Since this data is ‘content’ I can vary it over time. For the first release I just need 100 different products to add to the product list.

Creating Product Images

Generating the images was a little more interesting.

I am deliberately trying to keep my AI cost as low as possible to see how far I can push my use of AI with minimal cost.

My biggest cost on OpenRouter was using Nano Banana to create 2 experimental images when it was first released. The Nano Banana images cost 50c each and they didn’t end up the way I wanted. This was not an experiment I wanted to repeat with the product images. I’d need at least 100 and I might want to regenerate them if they didn’t work out.

First I tried image generation models through the local install of localai.io but I don’t think my machine is powerful enough. The process was slow and the output was basically a random png and completely unsuitable.

Rather than spend another couple of hours experimenting to improve this, I decided to try one of the image models on OpenRouter. These are easy to automate but past experience made me hesitant. However, as I was looking at the costs I noticed that the image pricing was quite cheap for one of the models, but this was only shown in the comparison page.

https://openrouter.ai/compare/openai/gpt-5-image-mini/google/gemini-3-pro-image-preview

GPT 5 Image Mini was estimated at $0.002 per K generated.

I didn’t expect to have good results with this but I tried it through a simple prompt and was surprised at the image quality.

It took me about 20 minutes to write code to call the API and experiment with prompts. This led to some good random product images and they were about $0.007 per image, so I could generate 100 images for about $0.70.

I used the product descriptions generated earlier and fed them through into OpenRouter with my script to generate the images you can see in the product pages.

Generating Image Descriptions

Rather than use Lorem Ipsum I wanted to generate fanciful and long descriptions for each of the products.

I wanted to use AI to generate a fuller paragraph style description of each product.

I found that the lava:7b model will run nicely on Ollama locally and it is very small. It has vision capabilities, so it can read images.

https://ollama.com/library/llava:7b

It only consumes about 5gb when running so it fits comfortably in my GPU memory.

I used this via api calls to describe each of the images in turn. I was very surprised at the quality of the descriptions when running locally.

This worked out well enough for my experiment and in total cost around $0.70 to generate and describe 100 images.

I thought that 100 was enough to allow pagination experiments and build on over time.

In total to do all this programmatically took about an hour to write code and 4 hours to run.

I think it would probably have taken longer for me to search the web, find a tool to do all this, if a single tool even existed at all.

Tool Vendors Take Note: it is becoming easier and easier for people to create their own tooling, cheaply and with the exact functionality they need.

Automating the Web UI

I used OpenCode to generate the Page Object Models and an initial set of coverage, I then amended and refactored to expand the coverage as I required.

This has become my normal approach to generating automated execution code now. This is also different from the way I code manually.

When coding manually:

I write the test first,
and grow the Page Object Model organically through refactoring,
this way I only create the abstraction code that I need in the test.

When coding with AI, I reverse this:

I generate the Page Object
- I give OpenCode access to the Chrome Dev Tools MCP so that it can read the DOM and it generates accurate locators. Sometimes I refine them, but they are usually good enough.
During the Page Object creation, in the same prompt I ask for
- “A set of basic tests which use the Page Object Model methods to cover the basic functionality. Do not test the Page Object, use the Page Object to avoid putting any locators for ‘driver.’ methods in the test code.”.
I then review and refactor.
Then I iteratively expand with small prompts, targeting specific coverage with each prompt.
I’ve found this works well, avoids rework and is fast.

Automating the REST API

I prompted OpenCode to read the routing code that I had written in Java for the API and create basic coverage for the each of the API end points in turn. This led to the creation of the payload objects and incrementally build up an API abstraction.

By targeting each endpoint individually I could gradually grow the code, keep control of it, make sure it was using my style of coding and automating and didn’t need a lot of refactoring later.

I then prompted to recreate the UI automated execution flows using only the API abstractions that I had just built up.

Testing

The general rule we are told is that you test first, then automate. You can see from the above description that I do not abide by this rule.

I used the automated coverage to handle the basic flows and flush out any simple errors. This creates an automated safety net that can help me when I’m fixing any issues that I find from testing. Without the basic automated coverage I might introduce more errors when fixing bugs than I remove.

Testing was performed via exploration using Chrome, heavily reliant on the dev tools and viewing requests in BurpSuite.

I then moved on to testing the REST API directly.

This helps me build up a list of issues, some of which I fix, but since it is a testing app, some of them I don’t.

The automated coverage tells me that it can work. My testing convinces me where it fails to work.

Security Testing

I coded in some security issues, so I made sure to test these.

I used Bruno as my API client to explore all these issues.

Some of them I removed because they would be annoying to the user if exploited when they were trying to test the application, so they are primarily information leakage rather than amendment. But… it might be possible if you test deep enough.

The only tool used during this process was Bruno to issue requests.

Performance Testing

I didn’t want to go live with the cart unless I could convince myself that it could handle multiple users.

I used OpenCode to convert my API automated execution scenarios into K6 scripted scenarios such that I could scale Virtual Users and hit the application functionality.

I do not recommend the Shopping Cart as a target for performance testing, that isn’t what it is designed for.

Butch Mayhew maintains a list of sites designed for practicing Performance testing against in his Awesome Sites To Test On

Two from Blazemeter and one from OctoPerf:

This took me about an hour in total, because I had to fix a couple of errors in the generated k6 script, but I think using OpenCode saved me a couple of hours since I’m not a K6 expert and have to re-learn it every time I use it.

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

Software Testing Podcast - BrowserStack Community QnA - The Evil Tester Show Episode 028

Alan Richardson — Fri, 12 Dec 2025 09:00:00 +0000

The answers given during an AMA session held on Discord on the 11th of December 2025, following a live LinkedIn video stream. The session focused on “Mastering Automatability for Test Automation”. The main theme is the concept of Automatability, which I view as the ability to automate , this personal skill is more critical than reliance on specific tools. The discussion covers various topics, including how to separate automation problems from application design issues, dealing with slow UIs and non-automation friendly third-party widgets, evaluating automation readiness, and addressing common architectural failings related to large-scale UI automation.

Links to audio version of the podcast here

Discord And LinkedIn

This was a Browserstack hosted event. The initial Q&A session started on Linkedin with a conversation between Alan Richardson and David Burns.

Recording can be found here:

LinkedIn Event

The session then moved to Discord. The Browserstack discord has many AMA and interviews so is worth signing up to have a look at.

Other AMA sessions include:

Join the BrowserStack Community on Discord and discover many more sessions, videos and conversations.

https://www.browserstack.com/community

Q&A Session Questions and Summaries

I’ve listed the questions and summary answers. Full answers can be found in the podcast, audio or video, or on the Discord AMA chat.

Q&A 1: Understanding Automatability for a First Automation Framework

Question: If I’m building my first test automation framework, what’s the one thing about automatability I should understand?

Summary of Answer:

The most important thing to understand is that automatability refers to your ability to automate. By having a strong ability to automate, you become less dependent on specific tools, making it easier to create workarounds and choose from multiple tools. Developing experience in how to automate allows you to succeed more often, and means you are not reliant on a tool interacting with your system, which makes workarounds harder. Automating is fundamentally about your understanding of what and how to automate, and practicing the application of that ability.

Q&A 2: Separating Automation Problems from Application Design Problems

Question: How do you separate automation problems from application design problems?

Summary of Answer:

If an issue causes problems when you are automating, I would call it an automation problem. While this problem might be triggered by an application design problem (such as a state-based system that is hard to track, or features that are harder to automate), the issue itself remains distinct. If the team cannot change the application design, they must figure out how to automate the application as it is. This might involve absorbing the issue, figuring out how to automate it at a different level (not end-to-end), or handling it through testing processes using observability tools like DataDog.

Q&A 3: Slow UIs and Testability/Automatability

Question: When dealing with slow UIs is the slowness a testability issue an automatability issue or both?

Summary of Answer:

Slowness is likely both, and more , because it is also a usability/user experience issue. If the slow UI impacts the user experience, it is more likely to be addressed than if it only impacts testing or automation. In cybernetics terms, testers or automators must possess the “requisite variety” to handle the variety (slowness) in the system being tested, which means knowing how to synchronize or potentially cleaning the environment to improve speed. The focus would be on the impact of the slowness, rather than the slowness itself, and whether the team or its tools can absorb the variation in response times.

Q&A 4: Handling Non-Automation Friendly Third-Party Widgets

Question: How do you handle third party widgets like payment gateways that are inherently not automation friendly?

Summary of Answer:

If a third-party widget is “not automation friendly” for one tactic (e.g., UI automation), it might become easier to automate by adopting different tactics, such as issuing HTTP requests using cookies collected from the UI. Teams may not need to automate the full flow of the widget, but instead focus on ensuring the widget is wired up correctly within their own application. This can involve only testing partway through the flow, or using a mock or stub in the environment so that the full widget flow doesn’t need to be tested constantly.

Q&A 5: Evaluating Automation Readiness and Consultancy Frameworks

Question: How do you evaluate an application’s automation readiness during consulting? Do you follow a framework?

Summary of Answer:

I do not use a formal consulting frameworks. The closest methodology used is the meta model from Neuro-Linguistic Programming, which involves asking questions to build a model of the client’s environment and processes, comparing it to the reality they face.

learn more about NLP and Testing in this paper I wrote.

Regarding automation readiness, an application is considered ready to automate “as soon as someone wants to automate it”. Readiness is judged by whether the client is prepared to commit to whatever it takes to automate the application at that specific point in time to achieve their desired outcomes, regardless of the application’s current state.

Q&A 6: Architectural Patterns Failing in Large-Scale UI Automation

Question: What architectural patterns do you see repeatedly failing when it comes to large scale UI automation?

Summary of Answer:

Recurring issues often stem from the team lacking the ability to automate and consequently blaming the tool for problems, rather than creating necessary workarounds. Common process anti-patterns, not strictly architectural patterns, include deploying differently into test and production environments (not using the same install process).

A major failure point is test data maintenance , especially trying to use production data or any data that the team does not control. Automating against specific data conditions without control over that data causes random test failures. This can be worked around by hardcoding tests against data conditions instead of specific data, and dynamically selecting the required data during execution.

Q&A 7: Prioritizing Testability and Automation in Sprint Planning

Question: If testability improves debugging and automation improves scale, how do we prioritize them during sprint planning?

Summary of Answer:

Prioritization can be based on what the team wants to achieve (the outcomes) by the end of the sprint, specifically focusing on the expected coverage from testing and automation. It is beneficial to plan for features that need extensive testing to be delivered early in the sprint. Ideally, testing and automating occur in parallel, and teams automate at lower levels (like unit level) to reduce the necessary coverage at the higher UI level. Issues often arise when teams are divided into isolated roles, creating process problems that hinder effective interaction and prioritization.

Q&A 8: Playwright and the Illusion of Reduced Automatability Design Needs

Question: Do modern frameworks like Playwright reduce the need for high automatability design or is that an illusion?

Summary of Answer:

It is an illusion. Frameworks like Playwright are designed to absorb application variability through features like retry mechanisms (for synchronization) and locator strategies (like visible text), which reduces the need for constant notification when minor changes occur. This absorption capability makes Playwright effective for agent-based automation where the goal is checking an end-to-end path and the final result.

However, this absorption can hide issues that a team might want exposed. Even when using Playwright, developers must still understand how to automate and structure their code using abstraction layers (like page objects, domain objects) to ensure long-term maintainability and efficiency.

Q&A 9: Explaining Automatability as an Investment to Leadership

Question: How do I explain to leadership that improving automatability is an investment not a delay?

Summary of Answer:

The explanation depends on what is being improved. If improving automatability means increasing the team’s ability to automate, it can be presented as an investment in staff training. If it involves adding technical aids (like IDs in the UI), leadership might perceive it as a delay because they may not value UI execution coverage or may already be confident in unit-level automation. To convince leadership, the team could demonstrate the return on investment by showing the alternative world. This involves comparing the current reality to a scenario where improved automatability allows the team to do beneficial things they otherwise couldn’t, thereby highlighting the value gained.

Q&A 10: AI Agents Dealing with Dynamic Elements

Question: We are exploring AI agents for our teams and I want to know how does the AI agent deal with dynamic elements like rotating banner banners, third party widgets or AB tests?

Summary of Answer:

How the agent deals with dynamic elements depends on how it works (e.g., building high-level BDD scripts or generating code). Agents often operate on first principles. If an agent uses a BDD approach, it works from a runtime specification and handles dynamic elements because it works from scratch for each execution, constantly aiming to fulfill the objective. For example, if an unexpected pop-up appears, the agent clears it and continues.

If the agent writes code, it uses what is often called “autohealing”. This process automatically amends the script based on the current application state, prioritizing the achievement of the objective regardless of whether the change is “right”.

Q&A 11: Early Signals of Flaky Features

Question: What early signals tell you that a feature will become flaky once automated?

Summary of Answer:

Early signals involve understanding the synchronization points of the page. A feature is likely to be flaky if the application is populated or amended over time by JavaScript and the automation tool is not synchronizing properly on the DOM buildup. If a page is being constantly updated in the background without clear visual indicators (like spinners, which are easy to synchronize on), flakiness is more likely.

The signal is the update process itself, particularly when it is non-deterministic (e.g., how totals are updated in a shopping cart). The core question is whether synchronization is required to prevent flakiness, and if it is difficult to synchronize, that is a strong signal. If necessary, automatability might be enhanced by adding an extra flag to the DOM to signal when the update is complete.

Q&A 12: Layers to Focus on in Microservices for Automatability

Question: In microservices setups which layers should teams focus on first to increase overall automatability?

Summary of Answer:

The foundational layer to focus on is the human understanding of the architecture and the requirements for automation. In microservices specifically, teams would typically focus on the interface layer and their ability to automate it while keeping the interface standard.

If microservices are communicating via HTTP messages compliant with a version standard, automation can be relatively easy. If interfaces are internal and change randomly, issues may arise, requiring attention to managing event-based queues if applicable. Strategies include using versioned interfaces or having a process to update automated coverage and abstraction layers when microservice interfaces change. It is crucial to avoid replicating interface objects (like payload objects) directly into test code, as this can prevent tests from spotting issues when new or removed fields occur in the application interface.

Q&A 13: Budget-Limited Automatability Fixes for Fastest ROI

Question: For a team with limited budget which problems around automatability should we fix first to get the fastest return on investment?

Summary of Answer:

The fastest return on investment comes from enhancing the team’s ability to automate. This improvement allows teams to develop workarounds, find alternative solutions, and identify when to use different techniques. It is not about purchasing multiple or expensive tools. Instead, investment could be placed in training, practicing, exploring the capabilities of existing tools, and eliminating fundamental issues like test flakiness by fixing the root causes.

Q&A 14: Collaborating Earlier to Avoid Automatability Rework

Question: How can developers and testers collaborate earlier to avoid expensive rework on automatability issues?

Summary of Answer:

Collaboration is achieved by removing the barriers that cause people to be isolated into silos, such as separate programming, testing, or test automation teams. The core issue is often that a “development team” is defined only as a programming team, instead of encompassing responsibility for design, programming, product suitability, testing, and production.

Practical steps include involving the programming team in the automated execution maintenance. When programmers contribute, they often ensure data IDs are present, which removes many hard-to-automate issues typically found during end-to-end testing. Sharing the responsibility for maintenance ensures people understand and resolve related issues earlier.

Q&A 15: Automatability in Continuous Delivery and Trunk-Based Development

Question: How should teams think about automatability when shifting towards continuous delivery and trunk-based development?

Summary of Answer:

This environment requires high automated execution coverage that runs quickly, often with features being merged multiple times a day. Automatability is achieved by ensuring the person or pair responsible for creating a feature is also responsible for adding automated execution coverage (unit tests). These tests demonstrate that the feature has been tested and provide future checks against accidental changes impacting the functionality.

Teams could focus on structuring unit tests at the domain level (e.g., focusing on users or orders) rather than strictly class level. This approach results in a degree of internal end-to-end flow tests without needing extensive external system testing. Furthermore, application architecture can be designed so that interfaces (like HTTP interfaces) can be tested primarily at the domain level, reducing the need for numerous actual HTTP calls.

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

Convert Chrome Dev Tools Recording to WebDriver Java Code Using Free OpenCode AI Agent

Alan Richardson — Mon, 24 Nov 2025 10:19:00 +0000

TLDR; Record and Playback is a notoriously bad way to automate. But what if we record, then have AI write code from the recording? Answer: it is a little better.

I performed an experiment to try tand demonstrate how to replicate the AI-powered test automation features commonly found in commercial SaaS testing tools using open source solutions. The process involved recording user interactions in an application with Chrome DevTools Recorder, exporting the recording as JSON, and then using an AI tool to convert the JSON into JUnit tests with WebDriver and page objects.

Overview Video

Watch on YouTube

In this video I:

record some web actions using Chrome Dev Tools recorder
fail to play them back because record and playback is notoriously bad
feed the recorded JSON file into OpenCode with a prompt
check the results of the AI process writing Automated Execution using WebDriver and Java

We do get working automated coverage out the back. I would need to edit it a little, and if I wanted to do this longer term I would need to amend the prompt to meet my coding preferences.

process

I used the triangle application on Test Pages:

https://testpages.eviltester.com/apps/triangle/

Test Pages has a deliberately hard to automate side bar. The easy automation is through the top level menu.

But I recorded the Triangle application by:

clicking on the side menu bar to open the Triangle application
and the instructions
reading the instructions
clicking back on the triangle application
interacting with the app to generate invalid and valid triangles

When I tried to playback the recording in the Chrome Dev Tools, it failed.

Most of us experience failure with record and playback tooling.

AI tooling

I have OpenCode, configured to use Chrome Dev Tools MCP, and working with Kat Code Pro Free in OpenRouter, but could just as easily have used any of the free coding LLMS on Open Code Zen or with Ollama locally.

I used a basic prompt

The process took about 5 minutes to run.

It opened the web page a few times,
created some Page Objects.
Found that the menu interaction was hard.
Decided to jump straight to the triangle page and automate that way
It did replicate the functional interaction included in the recording
it did generate assertions for the results
I could have used the test code generate with some small amendments.

Lessons Learned

SAAS tools which take this approach will use Agents, which are essentially longer prompts. I should have added into the prompt more conditions:

“given the json attached, convert the json into a set of junit tests using webdriver and page objects to recreate the flows recorded by the user. The json file is a recording made in Chrome Dev Tools recorder. In the test code, do not write any findElement code, always abstract the location and interaction into a page object. Use existing page objects where possible and try to re-use existing methods. Make sure that any page object methods you use are used in a test so we have coverage of the interactions.”

Over time, if I wanted to continue to automate using this process, I would refine the prompt further until I got the results I wanted, and possibly encode this prompt as an agent.

Would I do this?

Unlikely that I would do this in the real world.

This seems like the replication of a record and playback approach.

I do create Page Objects and have created initial execution coverage using AI.

I have some empathy for AI tooling scanning applications and building flows through the application to automatically interact and generate a wide variety of data, to increase the scope of coverage across a few simple flows. I can see potential in augmenting a team to allow the team to go off and do more detailed stuff. Provided the AI tooling doesn’t get noisy and keep distracting the team.

For any functionality as simple as I experimented with here, I hope that when people evaluate SAAS tools, that they also compare the paid tools to the capabilities of Open Source tools.

Nothing wrong with paid tools. They are incredibly valuable when they make hard things simple, and add value when they make hard things easier.

But, when the activity is simple and the functionality differences between Open Source and paid are small, I hope companies consider “what if we upskilled our staff a little such that we could use the Open Source tools”.

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

What tool to choose for Web UI Test Automation?

Alan Richardson — Thu, 25 Sep 2025 09:00:00 +0000

TLDR; I tried to objectively assess the web automation tool market. And as a new development team I’d probably choose Playwright, but as someone experienced in automating the web I’d still choose WebDriver.

Reviewing Automation Tools

I reviewed the market for Open Source and free Web Automation Tools so that I could more objectively answer questions like “What tool should we use for Web Test Automation”.

I tried to approach this like any development team would:

web searches for tools
read forums and comments
read the documentation

But I also tried to project scalability. If the team starts to create a lot of Web UI coverage and wants to have many different versions of browsers available then which tools also let them use cloud providers as the execution agent.

This was a thought experiment so I didn’t prototype execution with each tool I have used both tools so for the purpose of this write up I skipped this step.

Web Searches

There are a lot of spam articles on “17 Best Web Automation Tools for 2025”. You could create random variations of this title and probably find an article using your generated title on the web.

What you find are random lists which lump paid no-code tools with unsupported niche code libraries mixed with commercial AI based tools. Also automation tools (Webdriver, Playwright) would be mixed with execution and design frameworks (Robot, Cucumber).

An impossible combination to search through and most of those pages offered me no value.

So the first thing a team has to decide are filtering criteria:

paid or free and open source?
code based or no-code based?
AI or non-AI?

This will depend on the team. Development teams with coding skills are probably best served by looking for free code based tools. They will have the skill sets to take advantage of the flexibility provided.

Test teams with no coding skills will have a harder time choosing tools because there is so much competition for paid, no-code, AI tooling.

I’m focused on the free and code based tooling because I code, and I work with, Development teams with coding skills.

I also recommend that teams who are automating without these skills, try to develop these skills as it can help avoid tool lock-in and ultimately opens up more approaches for automating.

Community Sentiment Analysis

If I didn’t have much experience with the tools I’d check sentiment analysis and see what people were recommending on Community sites like Reddit.

Looking at the community comments Playwright is promoted more than Selenium WebDriver.

WebDriver seems to require a PR boost because it is seen as outdated. This is unfortunate because it isn’t.

At this point I’d have a smaller list of tools:

Selenium WebDriver
Playwright
WebdriverIO
Cypress
Puppeteer

And as I start looking through them I realise that if I want the option to use multiple languages I really only have two options: Selenium WebDriver and Playwright. All the other tools require automating in TypeScript.

Multi-language support is only offered by Selenium WebDriver and Playwright, all other tools are aimed at JavaScript/Typescript. This is probably a natural side-effect that ‘web applications’ have a front end that is a mix of HTML, CSS and JavaScript so the obvious choice to automate the front end is using the language space of the web: which is TypeScript and JavaScript. But, many teams are polyglot with a mix of skill sets. The core development skill for a large application is going to center on the main code base, which is likely the back-end, and that might well be written in TypeScript, but could equally be Java or Python.

A library with support for multiple languages is likely to the be choice for most polyglot environments.

My two main choices for Web Automated Execution driven by coding would be Selenium WebDriver and Playwright.

The community comments might lead me to think that WebDriver is flaky and hard to use. But I’ve never found WebDriver hard to use, or to be the the root cause of flakiness in my execution. I’ve never needed self-healing locators or auto-waits for stability.

Some comments against WebDriver: “you need to have the right driver installed for the browser” were removed a long time ago, first by the Selenium Manager library and now this is built in to WebDriver so it is very easy to get started with WebDriver since it downloads drivers and browsers when required. Selenium Manager

If I read community comments about WebDriver driver and browser management I might be concerned, but I’d then see in the Playwright documentation that I have to manage the browsers for playwright using maven or npm commands see browers docs. The documentation for Playwright actually makes this seem harder than it needs to be because I’m pretty sure I’ve never done this with Playwright and it downloaded what it needed automatically.

Scalability Support - Online hosts for Web Automation Tools

Reviewing the online Web Automation SAAS tools can give a good indication of which libraries and tools are popular and scalable.

If you start using a tool heavily then you might want to expand the range of browsers that you automate against, this can be time consuming to setup locally so Online hosts might help by pointing the test code at a cloud instance for a specific browser.

BrowserStack
- https://www.browserstack.com/
- Selenium, Playwright, Cypress, Puppeteer, WebDriverIO
LambdaTest
- https://www.lambdatest.com/
- Selenium, Playwright, Cypress, Puppeteer, WebDriverIO
SauceLabs
- https://www.lambdatest.com/
- Selenium, Playwright, Cypress, WebDriverIO
TestingBot
- https://testingbot.com/
- Selenium, Cypress, Puppeteer, Playwright, WebDriverIO

Reading the documentation for these tools I’d learn of other tools which are mentioned when looking through docs of online hosts e.g. TestCafe, Nightwatch. But again these are not polyglot so wouldn’t really expand my options.

At this point I’d also see that with my short-list of tools I have options around scalability which don’t require me setting up my own multi browser environments.

Playwright vs Selenium WebDriver

I narrowed my choice down to Playwright vs Selenium due to multi-language support.

So now I have to go through the thought process of comparing the tools.

Perceived Execution Speed and Coverage

Playwright has the advantage that it is newer and only targets a subset of browsers and browser communication protocols, it has no support for legacy browsers, it has never had to concern itself with backwards compatibility.

If you need to use legacy browsers then you’ll probably use WebDriver.

Both WebDriver and Playwright are converging on the WebDriver BiDi W3C standard, so when this happens a choice between WebDriver and Playwright is likely to come down to the programmer oriented API.

Documentation

Comparing the Playwright and WebDriver documentation and we can see the advantage that a tool with Big Company support has.

Playwright is provided by Microsoft and has people paid to work on the documentation and examples. WebDriver is community driven and so naturally the Playwright documentation is currently more attractive than WebDriver.

For an experienced team this isn’t really an issue because the WebDriver API is very well documented at the code level and both projects have example code on Github.

For a team just starting out the documentation page and community comments could well swing people over to Playwright immediately but I would hope this wouldn’t be a deciding factor for experienced teams.

Auto Waits and Retries

Playwright is probably also viewed as ’easier’ because of the auto-waits. I have some experience with automating so this actually makes me nervous, I’ve seen auto-waiting and auto-synchronisation cover up application bugs.

The auto-wait conditions are documented on the Playwright site under actionability Here I can see the automatic synchronisation on check, click, etc. I checked the Playwright documentation and see that I can configure the auto-wait action timeouts so I can take control over that behaviour when necessary.

Playwright has auto-retrying assertions expect(locator).toBeEditable() which seem analogous to the WebDriverWait

API Coverage

The addition of API coverage from a single tool might push teams towards Playwright.

I personally prefer tools and libraries to be focused on ‘one thing’ and do it well

API coverage can easily be achieved on projects using WebDriver by adding specialised libraries for the coverage required e.g. HTTP libraries for API testing and mocking libraries for APIs.

The dynamic mocking that Playwright supports where network traffic from the current page are mocked is harder to achieve with WebDriver and requires Network access. But also, is that the ‘right’ place to test that functionality? I’d expect to see that covered pre-deployment at the code level rather than running integration level and would expect that to be covered by testing-library.com level component tests with a mock API.

Future Project Scope

For me, this API Coverage might act as a Risk that the scope of Playwright expands beyond what I would require.

The scope of Playwright might be trying to evolve to cover deployed application execution and component development testing (like Cypress or Testing Library).

I can also see that there are differences in functionality between the language variants with the Node.js version offering a web server for local execution. This also makes me concerned about future viability of some of the languages. Would Java always be a first class citizen in the Playwright world?

Since the feature set between Playwright and WebDriver isn’t too different, I probably wouldn’t worry too much about this because most of my library usage would be behind a set of abstraction layers and I’d be able to migrate the implementation of those layers to other tools if necessary.

What to choose?

If I was new to automating I’d probably choose Playwright.

I’d be nervous about the differences in language support, but it probably wouldn’t be enough to stop me. If I was automating in TypeScript it probably wouldn’t concern me because I suspect TypeScript is the primary supported language for Playwright. For .Net I probably wouldn’t be concerned because .Net is Microsoft and they’ll keep supporting it.

Both tools are very good, they are maintained, they are both fast, they are both converging on BiDi. Playwright has extra features, many of which can be added to WebDriver projects using other libraries.

But this is a hard choice. And is probably determined by the feature set that you think you need for automating.

A Minority View

I feel that I’m in a Minority at this point. For production web automating I’d still choose Selenium WebDriver with Java or Kotlin.

I have a strong preference for single-focus libraries that do not try to extend their functional remit too far so that helps sway me towards WebDriver so I personally would still choose WebDriver for production work, but I acknowledge that I am biased based on my previous exposure to WebDriver.

I’m pretty sure that when the WebDriver BiDi standard is moved out of draft and supported fully by all browsers that the differences between WebDriver and Playwright will be minimal, but WebDriver may have lost a lot of the installed user-base by that point as people have moved towards Playwright.

The weight of public opinion at the moment is pushing towards Playwright which I think is also a pity because the standards that make Playwright possible were created and pushed by the WebDriver project team.

There is obviously more money behind Playwright, but it’s free and Big Companies often choose not to support tools for business reasons. This risk feels unlikely to happen but it’s in the back of my mind.

I think WebDriver remains an essential part of the Web Automation eco-system and it is unfortunate that the Big Browser Companies like Microsoft, Google, Apple are not listed on the Selenium WebDriver sponsors page. Despite this the WebDriver team manage to continue to push for open standards that make automating the web easier and better, and maintain tooling across languages and browsers that other tools do not support.

Multiple Language Support Tools Summary

Selenium WebDriver

https://www.selenium.dev/documentation/webdriver/
Programming languages supported: Java, Python, CSharp (.Net), Ruby, JavaScript, Kotlin(Java)
Browsers: Chrome, Edge, Firefox, IE, Safari
Functionality is aimed at Browser Automation.

Playwright

https://playwright.dev/java/
Programming languages supported: Java, Python, JavaScript/Typescript (Node.js), .Net
Documentation has some odd choices
Functionality covers Browser, API Automation, API Mocking, Assertions, auto-waits, videos, screenshots, traces

TypeScript/JavaScript Language Tools Summary

There are a lot of tools in this category which I didn’t really look at e.g. Nightwatch, TestCafe, and a host of others. In some ways I made life easy by only considering a polyglot team. If you specialise in Typescript and JavaScript then you have a lot of options to evaluate.

WebdriverIO

https://webdriver.io/
a JavaScript based abstraction on top of WebDriver protocols and Appium and Puppeteer (https://pptr.dev/)
Programming languages supported: Node.js (JavaScript/Typescript)

Cypress

https://www.cypress.io/
Chrome, Edge, Firefox, WebKit (experimental)
Programming languages supported: Node.js (JavaScript/Typescript)

Puppeteer

https://pptr.dev/
Chrome Dev Team maintained JavaScript library that is an abstraction on top of the WebDriver Bidi protocol, works with Chrome, Chromium and Firefox
Not recommended for normal Automated Coverage use cases, but if you want something lightweight that only targets a specific browser set, in JavaScript then have a look.
Programming languages supported: Node.js (JavaScript/Typescript)
Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

Software Testing Podcast - Episode 026 - Portfolio Projects - The Evil Tester Show

Alan Richardson — Wed, 17 Sep 2025 09:00:00 +0000

Should you have an online portfolio showcasing your skills and abilities to help get a job?

It really depends on the recruitment process. But… if I’m recruiting, and you have a profile then I will have looked at it. So it better be good.

Welcome to The Evil Tester Show! Covering the common question of “Should we have portfolio projects?” Alan shares his experience as both a creator and reviewer of portfolios, covering everything from the dos and don’ts of GitHub repos, to which skills actually benefit from a showcase, and how a well-crafted readme can make a difference. Whether you’re considering building a portfolio, looking to upgrade your current one, or wondering where to focus your efforts, this episode has practical advice to help you stand out as a top 1% applicants and avoid creating a portfolio that could hurt more than help.

audio and subscribe links here

Episode Summary

A practical look at online portfolios for developers and testers. Covering both trying to find your next role and also looking to switch careers. Most people don’t have public portfolios so that means you can really stand out if you build a good one.

We explore what actually matters in a portfolio: the difference between learning repos, personal projects, and true showcase projects designed for job hunting. We break down the content that impresses, and the stuff that doesn’t. We emphasize the need for simplicity and quality. Expect to find tips about managing forks on GitHub, writing an effective README, documenting your learning process, and promoting your work across LinkedIn and blogs to maximize visibility.

Tailor your projects to communicate exactly what you want and make your portfolio a focused sales tool rather than a catch-all for everything you’ve ever coded. Plus, advice on how to keep your repo clean, how to handle learning projects when switching roles or languages, and why you should never be afraid to show that you’re still learning.

Most recruiters won’t look at your portfolio, but when they do, you’ll want to be in the top 1%. If you do decide to build a portfolio, make sure it represents your best work.

Outline

00:00 Portfolio Value

The episode starts by discussing the real-world value of online portfolios. Most recruiters don’t look at them, but if the right person does, yours better be impressive. The host shares personal experiences with portfolios and offers perspective on how useful these tools can really be.

02:59 Stand Out Skills

We dig into what actually makes you, and your portfolio, stand out. Typical skills tested during interviews like defect writing, test cases, or strategies do not need to be shown off in a repo. Instead, focus on things that make you unique, such as personal opinions, in-depth case studies, or examples of thought leadership. This is often best done through blogs or articles, linked to your GitHub on your LinkedIn profile.

09:19 Project Types

A breakdown of project types:

Learning Projects: Messy, experimental, used to learn new tools, languages, or skills. It’s okay if they’re not polished, but document what you learned.
Personal Projects: Hobbies or interests that show personality and initiative. These don’t need to be perfect but should reflect genuine enthusiasm and curiosity.
Portfolio Projects (Showcase Projects): Your best, most complete work. These should be polished, well-documented, and focused on the skills you’re trying to sell.

Making these distinctions helps recruiters judge your work appropriately. Always clarify the purpose in your README files.

12:27 Showcase Projects

Showcase projects are your sales pitch—they need to be as close to perfect as possible. The host shares advice on what to include in a GitHub repo:

Use your profile README to introduce yourself professionally.
Only include original repos, not forks, to keep things clean.
Don’t commit IDE or compiled files; use .gitignore.
Use static analysis tools and add unit tests to show you care about quality.
Readme documentation should cover project intent, installation instructions, known limitations, and choices made during development.
Whenever possible, leverage GitHub Actions to show your code works.

Recency and maintenance matter when looking for work. Keep your portfolio projects updated and relevant.

19:39 Promoting Yourself

Once you’ve created a portfolio project, get the word out. Add it to your LinkedIn profile, feature it as a project, and share articles about your work. Updating and promoting your portfolio regularly keeps it visible and demonstrates your ongoing growth.

21:44 Final Advice

Most professionals don’t bother with portfolios, a missed opportunity for many, but if you go the extra mile to create one you can stand out. Make sure it’s good, you don’t want to stand out because it’s bad. Even if recruiters don’t look at it most of the time, the right one will, and that’s when it really helps. Sharing your learning and even your mistakes is crucial for personal development, especially as you gain more experience.

Key Takeaways

Most developers and testers don’t have a portfolio, so building one sets you apart when it’s high quality.
A strong online portfolio acts as a sales tool, focusing on your best, most relevant skills for the job you want.
Differentiate between learning projects, personal hobby repos, and polished, focused showcase repositories.
Keeping documentation clear, and up-to-date is just as important as good code in your portfolio.
Promoting your portfolio on platforms like LinkedIn and your blog can drive the right attention.

Quotes

“Should you have an online portfolio showcasing your skills and abilities and will that help you get a job? Well, I got an online portfolio, I’ve had one for years. But I also know that people don’t look at it.”

“If you’ve got a portfolio and it’s bad, it might put interviewers off. If you’ve got a learning project and you’ve been copying someone on YouTube… When people look at that, they’re gonna go, ‘This person doesn’t know what they’re doing.’”

“What you want to showcase is something that makes you different, a reason why someone would pick you up as opposed to someone else.”

“Portfolio project says, ‘I know how to do this, therefore judge me on the work.’ A learning project is, ‘I learned from these other people.’”

Join our Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

Software Testing Podcast - Episode 025 - The Respect Episode - The Evil Tester Show

Alan Richardson — Sun, 07 Sep 2025 09:00:00 +0000

In this episode we discuss respect in the world of software development, with a particular focus on the relationship between testers and programmers. The question isn’t just, “Do testers get enough respect?” but also, “Are you getting the respect you deserve, and do you deserve it?” The episode challenges individuals to evaluate their own skills, contribution, and attitudes.

Ever feel like you’re not getting the respect that you deserve in your job? This episode dives deep into the Respect in tech, especially focusing on software testing versus programming. We look at why some roles seem to earn more respect, what that means for workplace culture, and how you can change things for yourself and your team. Respect isn’t just about manners or titles - it’s about how the system works and how we show up in our roles.

If you’ve worked in agile projects, you might have heard, “Everyone is a developer.” But some roles seem to get more recognition than others. Is this because of how we define our jobs, or is it just baked into the way our workplaces run? This episode is a call to action, urging everyone to look at respect both at a personal, process and craft level.

We’re breaking down the difference between self-respect, respect for others, and respect built into your team’s process. You’ll see why just doing your job isn’t enough. You have to own your craft, communicate what you do, and make your contributions visible to earn genuine respect. By the end of this episode, you’ll have practical steps to make respect part of your daily work, whether you’re writing code, testing, building products, or managing.

Episode Summary

We shared insights from working on agile teams, where everyone is supposed to be seen as ‘developers’. But in practice, hierarchies pop up, with programmers often getting more respect than testers or product roles. The host argues that respect is a complex mix: it comes from individual skills, how well you integrate with the process, and how visible and valuable your output is.

If you don’t see value in your role, improvement is near impossible. Respect starts from within, and only then can you genuinely offer respect to your peers. This isn’t just a feel-good statement, the episode offers actionable advice: make your work visible, advocate for process improvement, pair up on tasks, and always keep learning.

The podcast also explores respect at multiple levels: the craft itself, the skills you bring, and respect for the person. The episode describes veneration as a kind of misplaced awe for work you don’t understand, and why that’s not real respect.

With the rise of AI, roles like programming might become less mysterious, changing how people view and respect them.

Finally, there’s a real-world look at dysfunctional organizations. Respect can be drowned out by fear, power plays, and closed processes. Still, the host insists you can make a difference at your touch-points: knowledge sharing, pairing, and fostering visibility.

Build self-respect, respect your craft, and model peer-to-peer respect every day.

Outline

00:00 – Respect Dilemma

The episode kicks off questioning whether testers get the respect they’re due, or whether programmers get more because their role is seen as more valuable or mysterious. The host points out that not getting respect might mean you have to level up your skills or make your work clearer to others. It’s not just about demanding respect; it’s about doing work that merits it.

02:41 – Human Level Respect

Respect isn’t just about roles, it’s about understanding and appreciating what each person brings to the team. The host shares that, on agile teams, the idea that “everyone is a developer” quickly breaks down when programmers get more recognition than testing or product roles. Respect should happen between people, not only between roles.

06:31 – Self-Respect First

If you don’t respect your own work, you can’t expect others to. The episode unpacks the concept of self-respect, challenging listeners to evaluate their skills, growth, and own contributions. There’s advice on how to build true self-respect: keep learning, share your skills, and don’t settle for routines that don’t work and don’t challenge you. Recognizing the value of what you do is the starting point for earning respect.

10:17 – Respect Cycle

You respect your craft, your peers respect you, and you respect them back. This creates a healthy team dynamic, with everyone functioning as equals, rather than building hierarchies or silos. The host explains how roles should be seen as complementary, with each one necessary for the team’s success, feeding into improvement and renewal.

15:37 – Knowledge Sharing

Knowledge sharing is called out as one of the most powerful drivers of respect in teams. The host stresses that sharing what you know builds trust and exposes the complexity behind roles that might otherwise seem simple. It also invites collaboration, helping people step into each other’s shoes. Escape rooms and team pairings are light-hearted but meaningful ways to see one another’s strengths.

18:53 – Respectful Organizations

Not all organizations foster respect. Some rely more on fear and command-and-control than genuine appreciation. The host dives into what a respectful culture looks like: leaders who model right behavior, open processes, and team members who feel safe to raise issues. Even in less ideal places, you can foster respect in your sphere by building solid relationships and pushing for honesty in process issues.

21:26 – Final Thoughts

The episode closes with the core advice: build self-respect, model it, and offer respect to others as peers. Respect isn’t about elevating someone else above yourself, it’s about recognizing everyone’s value, including your own. When you do this, you help shape a culture and a process where respect is sticky, even if leadership is lacking.

Key Takeaways

Respect in software teams isn’t automatic. It’s influenced by systems, visibility, and personal ownership.
Self-respect is the foundation for earning and giving respect to others in the workplace.
Knowledge sharing and pairing are powerful tools for building respect among team members.
Hierarchies based on fear and power undermine genuine respect and teamwork.
AI may change how people view and respect software roles, making process visibility and contribution even more critical.

Quotes

“You might not be getting respect because you don’t deserve it. So we need to look at that. We need to make sure that we are capable of doing the role well and selling it properly so that we get respect.”

“Respect happens at a human level. When we see the value that each individual brings and we honor and respect that in our environment.”

“You have to start from a point of ‘I have a craft that is worthy of respect, I have a set of skills that are worthy of respect and I know why they are worthy’.”

Join The Evil Tester Patreon from as little as $1 a month for early access to videos, ad-free videos, free e-books and courses, and lots of exclusive content.

Using Chrome Dev Tools AI Assistance to Automate UI from Javascript Console

Alan Richardson — Tue, 26 Aug 2025 17:01:49 +0000

Automating from Console with AI Assistance

TLDR; Can we use the Chrome Dev Tools AI Assistance to generate code that automates an application from the Javascript console? Answer: Yes we can. Prompt to generate code, iterative amendment of prompts required.

Using AI Assistance to Generate Automated Execution Code

This video shows how I used the Chrome Dev Tools AI Assistance to generate code to automate a simple application from the Javascrip console:

Automating From Console

For years, I have written code to automate and amend applications from the Javascript console. This is actually how I learned Javascript.

I created a free course on Test Automation U that covers the basic Javascript skills around this topic.

testautomationu.applitools.com/automating-in-the-browser-using-javascript/

And a support page with more resources

And a free ebook on the topic

AI Assistance

Chrome Dev tools now has an AI Assitance tab with a chat interface designed to help you debug CSS and DOM issues.

developer.chrome.com/docs/devtools/ai-assistance

There are many use cases listed on the official site:

Fix styling issues
Fix layout issues
Explain and investigate Network calls
Explain Javascript source files
Improve page performance

And if it can do all that, could it augment my ability to automate applications from the console?

I decided to find out.

7CharVal

The 7CharVal application is a simple application I created with a few bugs in it.

it takes an input
you click a button
the app outputs if the input was a "Valid Value" or "Invalid Value"
to be valid the input should be 7 chars

You can find the application here:

testpages.eviltester.com/styled/apps/7charval/simple7charvalidation.html

The application is surprisingly useful for practicing testing. On Patreon I even released a 200+ page ebook of testing exercises and notes, all built around this tiny application.

Automating 7CharVal from console

I can automate the 7CharVal application from the console:

let validChars = 
    "*0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
let errors = [];

// loop over the ascii chars
for(x=0;x<256;x++){

    let aChar=String.fromCharCode(x);
    let expectValid = false;
    let resultExpected = "Invalid Value";

    // decide if the char should be valid or invalid
    if(validChars.includes(aChar)){
        expectValid=true;
        resultExpected = "Valid Value";
    }

    const checkButton = document.querySelector('input[name="validate"]')

    for(position=0;position<7;position++){

        // create the input string
        let seedString = "0000000";
        let positions = seedString.split("");
        positions[position]=aChar;
        let inputString = positions.join("");

        // input the string
        document.querySelector(
                    'input[name="characters"]').value=inputString;

        // trigger the process
        checkButton.click();

        // get the output
        let result=document.querySelector(
                    'input[name="validation_message"]').value;

        // document the result
        let reportLine = 
            `Input: "${inputString}" Dec: ${x} `+
            `Position:${position} Expected: "${resultExpected}" `+
            `Output: "${result}"`;

        // add an alert in the reportif it is not what we expected
        let resultPrefix = "";
        if(result!==resultExpected){
            resultPrefix = "**ERROR** ";
            reportLine = `${resultPrefix} ` + reportLine;
            // separate errors out into their own report
            errors.push(reportLine);
        }
        // document all the results
        console.log(reportLine);
    }
}
// report on the errors
console.table(errors);

This code:

iterates over 256 ascii chars
and adds the char into each of the 7 positions of the input string
and checks if the generated string is correctly marked as Valid or Invalid

It took me about 20 minutes to write by hand.

Using AI Assistance to write Code

During the iteration of prompts I decided to take a slightly different approach to automating the application and generated a wider range of inputs rather than picking a single character in a single position. This makes isolating single character issues more difficult but can provide a wide set of data inputs to flush out combination issues. e.g. strings like 'Fe7Nsbb', 'znOzzQg', '7O2i29a', 'wqOdesY', 'ghRGLCO', 'lLrwH*j'

The full write up of this experiment is on my AI Experiments repo:

I iterated gradually through a series of prompts. e.g.

Create code to run in the Javascript console to enter the string "Aabbccd" into the characters field. Then press the "Check Input" button. Then check that the validation_message field shows the value "Valid Value".

The full set of prompts can be seen in the experiment.

I tried to avoid relying on the memory context of the AI tooling and kept iterating on the prompt to produce a single output code block.

And my final prompt ended up as:

Create code to generate the list of testStrings. The test strings should be 7 characters long. Generate enough test strings such that each of the valid characters should be used at least once in a generated test string. The valid characters are A-Z, a-z, 0-9 and *. Then create code to run in the Javascript console to enter each of the generated test strings into the input field with name "characters". Then press the input element of type button with value "Check Input" button. Then check that the validation_message field shows the value "Valid Value". Add any inputs that pass and generate validation_message "Valid Values" to an array called validInputs. Add any of the input values that fail to generate validation_message "Valid Values" into an array called invalidInputs and throw an exception.

The generated code is below. Copy and paste it into the Javascript console when on the 7charval page and you will automate the application.

const characters = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789*";
const stringLength = 7;
const requiredCharacters = new Set(characters);
const testStrings = [];
const generatedCharacters = new Set();
let attempts = 0;
const maxAttempts = 10000; // Limit attempts to avoid infinite loops in case coverage is hard

// Generate strings until all characters are covered or max attempts reached
while (generatedCharacters.size < requiredCharacters.size && attempts < maxAttempts) {
    let randomString = '';
    let stringContainsNewCharacter = false;
    for (let j = 0; j < stringLength; j++) {
        const randomIndex = Math.floor(Math.random() * characters.length);
        const char = characters[randomIndex];
        randomString += char;
        if (!generatedCharacters.has(char)) {
            generatedCharacters.add(char);
            stringContainsNewCharacter = true;
        }
    }
     if (stringContainsNewCharacter || testStrings.length < requiredCharacters.size / stringLength + 5) {
        testStrings.push(randomString);
    }
    attempts++;
}

console.log(`Generated ${testStrings.length} test strings covering all required characters (attempted ${attempts} times).`);

const validInputs = [];
const invalidInputs = [];
const characterInput = document.querySelector('input[name="characters"]');
const checkButton = document.querySelector('input[type="button"][value="Check Input"]');
const validationMessage = document.querySelector('input[name="validation_message"]');

if (!characterInput) {
    console.error("Character input with name 'characters' not found.");
} else if (!checkButton) {
    console.error("Check Input button not found.");
} else if (!validationMessage) {
    console.error("Validation message element not found.");
} else {
    console.log("Starting validation tests...");
    for (const str of testStrings) {
        characterInput.value = str;
        checkButton.click();
        // In some cases, you might need a small delay here for asynchronous updates:
        // await new Promise(resolve => setTimeout(resolve, 50));
        if (validationMessage.value === "Valid Value") {
            validInputs.push(str);
            console.log(`Input "${str}" passed validation.`);
        } else {
            invalidInputs.push(str);
            console.warn(`Input "${str}" failed validation. Received: "${validationMessage.value}"`);
        }
    }
    console.log('Validation tests complete.');
    console.log("Valid inputs:", validInputs);
    console.log("Invalid inputs:", invalidInputs);


    if (invalidInputs.length > 0) {
        throw new Error("Validation failed for some inputs. See invalidInputs array for details.");
    }
}

Notes

The AI tooling was automating the 7charval application during the prompt and summarizing the results of the execution through the LLM.

I found that the generated code shown to me, was different from the code used by the AI tooling during its execution. The AI tooling liked to put the results in an object but didn't do that in the code provided to me. There is a risk that its code did something different, but that could be solved by code reviews.

I found the constant reviewing of the code tiresome because each generated block of code was 'new' and I didn't have diffing tools in the chat window to see what was different. I could have pasted it into a diffing tool to aid my review but I gave it a cursory review and became more intersted in the output of the execution.

The AI Tooling would gradually add extra chars into the final code output making it impossible to copy and paste into the console without first editing the code. It looked like it was trying to add markdown code formatting into the code formatting.

This was a minor annoyance that I could work around.

Conclusion

This feature would have helped me learn Javascript, had it been around when I started automating from the console.

I will certainly use this in the future to help me generate initial code that I can edit and refine myself.

I recommend experimenting with the AI Assistance as a code generator for the use case of automating applications from within the browser itself.