Forem: Minh Le (Henry)

How I Built a Playwright Pytest Automation Framework with an AI Assistant

Minh Le (Henry) — Thu, 02 Apr 2026 03:55:14 +0000

Why use an AI assistant

"AI won’t replace humans. But humans who use AI will replace those who don’t." - Sam Altman

Test automation is a perfect domain for AI augmentation. Much of the early work—boilerplate code, fixtures, selectors, test skeletons—follows predictable patterns. AI assistants can generate these patterns quickly, suggest test flows, and help debug issues, especially when given clear context.

GitHub Copilot’s ecosystem is built for this. With Instruction files, you can define your architecture and coding rules. With Skills, you can give Copilot new capabilities. With Plugins, you can extend Copilot into domain‑specific workflows like Playwright automation.

The engineer still owns the architecture, validations, and edge cases. The AI accelerates the repetitive parts.

How I used the AI assistant

Working with an AI assistant is like onboarding a new team member. Even the smartest assistant doesn’t automatically understand your project or expectations. You have to teach it how you work.

Step 1 — Define the rules with Instruction Files
I created Instruction files that described:

Architecture and folder structure
Naming conventions
Preferred libraries
Locator strategy
Coding style
Test design principles
Quality standards These Instruction files became the “brain” the assistant used to generate consistent code.

Step 2 — Equip the assistant with Skills and Plugins
To make the AI genuinely useful for Playwright automation, I added:

Playwright CLI Skills: These give Copilot real browser access, DOM exploration, and the ability to inspect selectors.
Testing Automation Plugin: This plugin adds Playwright‑specific workflows, structured test generation, and UI exploration capabilities.
QA Sub‑Agent: A specialized reasoning model tuned for QA tasks.

With these components, I effectively built a custom AI QA automation stack:
• QA Sub‑Agent → QA‑focused reasoning
• Testing Automation Plugin → Playwright engine
• Playwright CLI Skills → Browser tools
• Instruction Files → My rules and architecture
At this point, the assistant behaved like a junior QA engineer who already understood my framework.

============================
Example test generation
Here is the kind of prompt I gave the AI assistant:

Use the instruction files and explore the website https://www.target.com/, then generate the test for the workflow below

Search for the product "AIWA ARC Noise Cancelling Over Ear Wireless Headphones"

Add to cart (via Choose Options dialog)

Verify cart

The assistant explored the site, followed the instructions, and produced a test using the Page Object Model, Playwright’s recommended selectors, and Pytest fixtures. It wasn’t a final product, I still reviewed and refined it, but it gave me a strong first draft in minutes instead of hours.

What worked well

The biggest benefit was speed. I could move from setup to first test execution much faster because the AI assistant handled a lot of the repetitive scaffolding. Another advantage was consistency, since I could ask it to follow the same naming patterns, locator strategy, and fixture style across files, which matches Copilot best practices for maintaining a consistent codebase.

What to watch out for

Speed
The biggest win was how quickly I moved from an empty folder to a running test suite. The assistant handled repetitive scaffolding so I could focus on architecture and quality.

Consistency
Because I defined my conventions upfront, the assistant generated code that matched my structure, naming patterns, and locator strategy. This aligns with Copilot’s best practices for maintaining a consistent codebase.

Reduced Cognitive Load
Instead of switching constantly between writing fixtures, designing POMs, and drafting tests, I could offload the mechanical parts and stay focused on higher‑level decisions.

What to Watch Out For

AI‑generated code still requires human review. Common pitfalls include:

Brittle or overly specific selectors
Missing edge cases
Incorrect assumptions about application behavior
Over‑generalized test flows when prompts are vague The more specific the prompt, the better the output. Treat the AI as a fast junior engineer: helpful, but not a substitute for expertise.

Conclusion

Using GitHub Copilot’s Instruction files, Skills, and Plugins to build a Playwright + Pytest automation framework is not about replacing engineers—it’s about amplifying them. By combining AI‑generated scaffolding with human review, Playwright best practices, and thoughtful test design, you can dramatically accelerate framework setup without sacrificing quality.
For teams looking to scale automation quickly, this hybrid workflow turns what used to be a slow, manual process into a streamlined, efficient one. It’s a modern, practical approach to building reliable UI automation at speed.

Simplify API Testing with Postbot: Postman's AI Assistant

Minh Le (Henry) — Sun, 05 May 2024 10:22:07 +0000

AI continues to pave the way for the future, offering numerous tools to enhance productivity. As a tester, you may have admired the AI tools developers use, such as ChatGPT and GitHub Copilot, and wished for a similar assistant in your software testing process. Postman has heard you and introduced Postbot, an AI assistant that simplifies API workflows in Postman.

Here's how Postbot can streamline your work in Postman:

1. Automate API Testing: Postbot can add new tests for your API requests and responses, update existing tests, or extract specific fields from responses.
2. Generate API Documentation: Postbot can automatically create comprehensive documentation for your API requests, including request parameters, descriptions, and response examples.
3. Debug Requests: Encountering unexpected errors in your API requests? Postbot can help you identify the root cause and offer solutions.
4. Visualize Response Data: Postbot enables you to display API response data in formats such as tables, charts, or graphs, enhancing cognitive understanding and presentation.
5. Access Help and Resources: Trained on Postman's Learning Center, Postbot can guide you through challenges, offering step-by-step assistance and linking you to detailed resources.

Let's take a closer look at Postbot's capabilities: 👩‍💻👩‍💻👩‍💻

To begin, you can ask Postbot to add tests for your API response simply by selecting the "Test for response" option. Postbot will then generate happy path tests for you automatically.

Additionally, you can customize prompts using natural language to tailor your tests.

The innovation doesn't end there. Postman's Visualizer tool, powered by Postbot, lets you transform request responses into visually engaging representations such as graphs, tables, or charts, enhancing clarity and comprehension.

Happy testing! 🚀

Optimizing Performance Testing with Docker: K6, InfluxDB, and Grafana Integration

Minh Le (Henry) — Wed, 25 Oct 2023 12:18:51 +0000

In today's fast-paced software development landscape, performance testing is essential to ensure the reliability and scalability of applications. Docker has emerged as a valuable tool for simplifying the setup and management of performance testing environments. In this article, we'll explore how to leverage Docker to create a powerful performance-testing stack by integrating K6, InfluxDB, and Grafana.

Unpacking K6 👩‍💻

An open-source load testing tool that makes performance testing easy and productive for engineering teams. It's free, developer-centric, and extensible.
Ability to write performance tests as code.
Ability to run in hybrid mode to check for both Frontend & Back end performance test

Key features of K6 🤔

Load testing
Browser testing. Through the k6 browser, you can run browser-based performance testing and catch issues related to browsers only which can be skipped entirely from the protocol level.
Chaos and resilience testing. You can use k6 to simulate traffic as part of your chaos experiments, trigger them from your k6 tests, or inject different types of faults in Kubernetes with xk6-disruptor.
Performance and synthetic monitoring

A Hands-On Approach 🚀

We set up the infrastructure to run the performance test with Grafana K6. Then, we send the test results to InfluxDB and visualize them to the Grafana Dashboard as a Time Series Analysis report.

Prerequisites:

Make sure that Docker Desktop is installed on your machine.

Steps

Clone the Repository:

https://github.com/lengochoangminh/hands-on/tree/main/k6

Navigate to the folder K6
Then launch the infrastructure:

docker-compose up -d influxdb grafana

and execute the test script:

docker-compose run k6 run /src/test_scripts/dummy-json/GetSingleProduct.js --insecure-skip-tls-verify

Finally, let's open your web browser and navigate to Grafana Dashboard to monitor live test results in Grafana Dashboard at http://localhost:3000/d/k6/k6-load-testing-results

Simplifying Mobile App Testing with Appium 2.0 and Device Farms

Minh Le (Henry) — Mon, 25 Sep 2023 13:03:19 +0000

Setting up a mobile device farm for testing applications can be a daunting task, but recent advancements in tools like Appium 2.0 have made this process much more approachable and efficient. One such tool that stands out is the Appium Device Farm, known for its user-friendly interface and powerful capabilities. Let's walk through the steps to set up and streamline your mobile testing using this incredible tool.

Prerequisite:
Before we get started, ensure you have Appium version 2.0.X installed on your system. This is the backbone of our setup, allowing us to manage and interact with the devices effectively.

Installing Appium Plugins
Let's start by installing essential Appium plugins using the following commands:

appium plugin install --source=npm appium-device-farm
appium plugin install --source=npm appium-dashboard

Tracking iOS Device Events
For tracking iOS device events like device plugging and unplugging, we'll use the 'go-ios' module. Install it using npm:

npm install -g go-ios

To ensure seamless integration, set the 'GO_IOS' environment variable. Open your .zshrc or equivalent file and add the following line:

export GO_IOS="/usr/local/lib/node_modules/go-ios/dist/go-ios-darwin-amd64_darwin_amd64/ios"

Completing the Installation
With the plugins and necessary modules installed, we're almost ready to start using the setup.

Starting the Appium Server
Connect your physical devices to the server or utilize simulators. Start the Appium server using the following command:

appium server -ka 800 --use-plugins=device-farm,appium-dashboard -pa /wd/hub --plugin-device-farm-platform=both

This command kickstarts the server and sets it up to use the plugins for managing the device farm effectively.

Accessing the Dashboard
Voilà! You're now all set. Access the dashboard to monitor device availability at http://localhost:4723/device-farm/.

Running a Basic Demo Test
Let's run a simple demo test to ensure everything is working smoothly. Below is a basic demo test using Appium to interact with an iOS application on an iPhone 14 simulator.

public class IOSBasicDemo {

    protected static IOSDriver driver = null;
    public WebDriverWait wait;

    @BeforeMethod
    public void setup() throws MalformedURLException {

        XCUITestOptions options = new XCUITestOptions();
        options.setDeviceName("iPhone 14");
        options.setPlatformVersion("16.4");

        String appPath = System.getProperty("os.name").equals("Windows 10") ? "\\apps\\TheApp.app" : "/apps/TheApp.app";
        options.setApp(System.getProperty("user.dir") + String.format(appPath, ""));

        driver = new IOSDriver(new URL("http://127.0.0.1:4723/wd/hub"), options);
        wait = new WebDriverWait(driver, Duration.ofSeconds(10));
    }

    @Test()
    public void TestScenario() throws InterruptedException {
        wait.until(ExpectedConditions.visibilityOfElementLocated(new AppiumBy.ByAccessibilityId("Photo Demo"))).click();
        Thread.sleep(3000);
    }

    @AfterMethod
    public void teardown() {
        driver.quit();
    }
}

Exploring the Appium Device Farm Dashboard
The Appium Device Farm dashboard offers a comprehensive view of your testing environment. From execution video capture to device logs and debug logs for troubleshooting, it provides crucial insights to enhance your testing experience.

A huge shoutout to the contributors at the Appium Device Farm GitHub repository for their dedication and efforts, in making mobile testing more accessible and efficient!

Happy testing with Appium Device Farm! 🚀