Forem: Kaustav Singha Roy

Codeless Test Automation

Kaustav Singha Roy — Thu, 21 Mar 2024 14:06:42 +0000

Over time what used to be manual testing in software industry has evolved into automated testing which involves writing code to execute repetitive testing steps and generate test outputs automatically. Typically, most projects today would have a hybrid approach having a mix of both automated and manual testing. In addition to requiring specialized programming skills, automated testing also require time and effort which can sometimes exceed the benefits of automation.
This problem brings into picture codeless test automation, which enables non-technical testers to participate actively in the QA process. In addition, there is a jump in productivity owing to lack of coding effort which has a huge financial benefit as well.

This is an analysis of various aspects of codeless test automation and some of the products available today.
Why to Use Codeless Test Automation
The answer to this should be simple and intuitive. There is usually a simple graphical user interfaces (GUI) to design and automate tests. The testers would drag and drop components and the coding takes place dynamically in the background without the tester having to provide any input. There is no need for dedicated automation engineers and testers can focus more on test design and analysis and not on the coding.
In some cases non-technical resources like business analysts could be involved in the functional testing phase. They may have good subject matter expertise but not advanced technical skills to write automated tests effortlessly. Codeless test automation is easy for them to adapt to as well. Even for experienced automated testers this is undoubtedly a much faster process to create tests.
In addition to that, there is a reduction in maintenance headache often as there is no automated testing codebase that needs frequent update.

Having market standard products brings in another important benefit. Normally they are well researched and exhaustive and increases the usual test coverage which may not factor in all aspects. Codeless test automation tools often provide a wide range of pre-built test libraries and integrations with various technologies and platforms. Testers can leverage these resources to create comprehensive test suites, covering different aspects of the application, such as UI interactions, data validation, and API testing. This helps identify potential issues across multiple layers of the software stack which may take significant effort and man power to accomplish otherwise.

For anyone attempting to try this, these steps can be followed:

➕ Platform Selection : Research and choose a codeless test automation tool or platform that suits your project requirements. There are several popular options available in the market, such as TestCraft, TestProject, Katalon Studio Tricentis Tosca etc.
➕ Visual Scripting: Utilize the graphical user interface (GUI) provided by the codeless automation platforms to design test cases. There could be drag and drop components, predefined actions, visual workflows that create test scenarios.
Arrange the test steps in the desired sequence and configure inputs, test conditions and desired output.
➕ Test Data Configuration: Set up the necessary test data required for executing the test cases. Define input values, mock data or connect to external data sources. Some codeless test automation tools provide built-in data management capabilities to streamline this process.
➕ Execute Test Cases: Execute the designed test cases using the codeless test automation platform.
➕ Analyze Test Results: Once the test execution is complete, review the test results and analyze any failures or discrepancies.
Codeless test automation platforms provide comprehensive test reports and logs, highlighting the status of each test case and any encountered issues. This also reduces the manual effort required otherwise.
Challenges
Obviously like anything else, this also comes with its own challenges which needs to be kept in mind.
➕ Limited Customization: Ultimately codeless aspect is achieved by standardising test flows and proving pre-built tools to achieve standard automation. This also means they may have limitations when it comes to customizing complex test scenarios. Advanced test cases requiring intricate logic or conditional branching may still require some level of coding or the assistance of a technical developer.
➕ Learning Curve: Although codeless test automation aims to bridge the gap between technical and non-technical testers, there is still a learning curve associated with mastering these tools. Testers need to familiarize themselves with the specific interface, workflows, and concepts of the chosen codeless automation platform. So the actual benefit may be realized only after a period of time.
➕ Along with that the usual challenges of integrating any software product remain here too. Organizations need to evaluate codeless test automation solutions that align with their development ecosystem and ensure there is no conflict with existing architecture. Not all products will work everywhere.
Lastly some of the existing products in the areas include TestCraft, TestProject, Katalon Studio and Tricentis Tosca. Almost all of them typically provide a visual drag and drop interface along with test recorder and object spy for effortless test creation. They provide almost all of the benefits explained above. In addition, some of them provide cloud based test execution and integrates with CI/CD tools as well enabling a smooth DevOps workflow supporting market leading products in these spaces.

An exciting and highly productive aspect to think about for any project team having time and budget constraints. However, it’s important to consider the specific requirements and constraints of the project when evaluating these products all of which can help in reducing coding overhead for test automation.

A Study in Favour of Shift-Left Testing

Kaustav Singha Roy — Sat, 16 Mar 2024 19:50:17 +0000

The technology industry is evolving rapidly, necessitating agility and efficiency in development processes. Agile and Kanban practices have become instrumental in maintaining quality while accelerating delivery. A key paradigm shift in testing and QA approaches is the adoption of “Shift-Left” testing. This article explores the significance of Shift-Left testing, its divergence from traditional methods, the associated benefits and common applications.

Difference with Traditional Testing: In traditional testing, the testing phase is positioned towards the “Right” of the software development life cycle (SDLC). This means that requirement understanding, test case creation, execution, and defect detection occur after development is complete. This approach poses risks, such as delays and missed project timelines due to late bug detection and increased rework. To address these issues, the Shift-Left approach moves many testing steps to earlier phases of the project. It promotes collaboration between developers and testers, fostering shared responsibility for quality.

How Shift-Left Helps:

Early Bug Detection: By testing early, Shift-Left testing catches defects in their early stages, reducing the risk of critical issues going unnoticed. This minimizes the cost and effort required for defect fixing, avoiding setbacks.

Improved Communication and Collaboration: Continuous feedback from testers enhances code quality and increases efficiency. Testers gain a clear understanding of requirements from the beginning, resulting in better collaboration.

Faster Time to Market: Shift-Left testing accelerates the development cycle by reducing rework, optimizing test cycles, and streamlining defect resolution. This enables organizations to gain a competitive edge by delivering software solutions faster.

Test Driven Development (TDD): The Shift-Left approach aligns well with Agile continuous testing and TDD methodologies. Features are tested from the initial draft code and continuously throughout the development process. By the time formal testing begins, a significant portion of the testing steps has already been completed, reducing inefficiencies.

In a Shift-Left approach, testing activities are not delayed until a dedicated testing phase. Instead, they occur concurrently with development. The following steps exemplify this approach:

Requirements understanding from testers’ perspective

Test planning

Test design

Test environment setup

Code quality reviews

Partial test execution

Early defect identification and management

Types of Testing In-Scope

While not all testing types can be executed before development is complete, the following can be shifted at least partially to the left.

Unit Testing: In Scope. Developers conduct unit testing during development once logical functionalities are complete.

Integration Testing: In Scope. Testers can test the integration between various modules early and continuously.

Regression Testing: In Scope. If regression test cases are automated, they can be executed during development, providing early feedback. Manual testing may have limited success in shifting regression testing to the left.

Performance Testing: Partially in-scope. Proper performance testing requires complete deployment, making early results less accurate and beneficial.

System Testing: Partially in-scope. By definition, system testing involves testing the software as a whole and is typically executed after development completion. However, preparatory steps, such as test creation, can occur earlier.

Security Testing: Partially in-scope. Similar to performance testing, comprehensive security testing is advisable after development completion.

Implementing the Shift-Left approach requires a cultural shift and additional training for testers. Despite the challenges, it offers significant advantages to Agile teams and organizations requiring rapid development and testing. While the concept of Shift-Left testing has existed for some time, it has gained additional traction in recent years, helping organizations stay competitive in the ever-evolving software development landscape.

Test Automation — Playwright vs Puppeteer

Kaustav Singha Roy — Tue, 23 May 2023 18:50:17 +0000

In this article, I will compare Playwright and Puppeteer, two most prominent web automation tools around, highlighting their similarities, differences, strengths, and weaknesses. The goal is to equip you with the essential insights to make an informed decision on which tool best aligns with your project requirements and development needs.

Overview of Playwright and Puppeteer

Playwright is an open-source Node.js library developed by Microsoft that enables developers to automate browser actions and interactions for testing and scraping purposes. It supports multiple browser engines, including Chromium, Firefox and WebKit, allowing for seamless cross-browser testing. Playwright provides an easy-to-use API and a robust set of features, making it a go-to choice for many developers.

On the other hand, Puppeteer is a Node.js library created by the Chrome team at Google. Like Playwright, Puppeteer is designed for automating browser actions and interactions, but it primarily focuses on the Chromium browser engine. Puppeteer boasts a user-friendly API and a strong feature set, making it a popular choice for web automation and testing tasks since the time it came out in 2017.

Similarities

Playwright is written and maintained by people who had earlier worked in Puppeteer and then moved to Microsoft so there are obvious similarities.

Both tools provide a comprehensive API for automating browser interactions, such as clicking buttons, filling out forms, and navigating pages. There is also headless browser support, allowing developers to run tests and automate tasks without the need for a visible browser window. Both tools can capture screenshots of web pages or generate PDF files, making it easy to save visual representations of a site’s content. In both Puppeteer and Playwright, developers can intercept and modify network requests and responses, providing greater control over the testing environment. Both allow execution of JavaScript within the context of the web page as well, enabling developers to interact with and modify the DOM as needed.

Differences Between Playwright and Puppeteer

Despite their obvious similarities and similar goals, Playwright and Puppeteer have some key differences that set them apart:

Browser Support: One of the most significant differences between the two tools is their browser support. Playwright provides out-of-the-box support for Chromium, Firefox, and WebKit, facilitating cross-browser testing. In contrast, Puppeteer primarily focuses on the Chromium browser engine, with limited support for Firefox and no native support for WebKit.

Selector Engine: Playwright offers a more advanced selector engine compared to Puppeteer. It supports a variety of selector types, including CSS, XPath, and text-based selectors, while Puppeteer mainly relies on CSS and XPath selectors. This can make it easier to target specific elements on a web page with Playwright.

Multiple Contexts and Pages: Playwright allows developers to work with multiple browser contexts and pages simultaneously, enabling more complex testing scenarios. Puppeteer, on the other hand, has limited support for managing multiple contexts and pages, making it more challenging to handle intricate test cases.

Auto-Waiting Mechanism: Playwright features an advanced auto-waiting mechanism that automatically waits for elements to become available before performing actions. This can help to reduce the need for manual timeouts and improve the stability of tests. Puppeteer lacks this advanced auto-waiting functionality, requiring developers to rely more on manual timeouts and workarounds.

Accessibility Testing: Playwright has built-in support for accessibility testing, allowing developers to verify that their web applications meet accessibility standards. Puppeteer does not have native support for accessibility testing, requiring additional tools or libraries to perform such tests.

When to select Playwright:

➕ You need to test across multiple browsers, as Playwright offers out-of-the-box support for Chromium, Firefox, and WebKit.
➕ You have complex testing scenarios that involve multiple browser contexts or pages, which Playwright handles more effectively.
➕ You prefer advanced element selection capabilities, as Playwright provides a versatile selector engine.
➕ You value stability and automatic waiting mechanisms, which are built into Playwright and help reduce flakiness in tests.
➕ Accessibility testing is a priority, as Playwright offers built-in support for this feature.

When to Choose Puppeteer:

Puppeteer might be the better solution if:

➕ You mainly target the Chromium browser engine and don’t require extensive cross-browser testing.
➕ You or your team have prior experience with Puppeteer or an existing code base that leverages it, making it more efficient to continue using the tool.
➕ You have simpler test cases that don’t require managing multiple browser contexts or pages, which Puppeteer handles with some limitations.
➕ Your project relies on Chrome-specific features or APIs, as Puppeteer’s close integration with the Chromium browser engine ensures better compatibility and support.

How to Decide Between TDD vs BDD vs ATDD vs DDD

Kaustav Singha Roy — Sat, 01 Apr 2023 19:19:32 +0000

Many of us struggle to understand the different philosophies of development. There are quite a few ways to develop software in the market today. As a testing professional this is important for me so this is my take from my experience on this topic.

Test Driven Development (TDD)

This is perhaps the oldest of the method here and was developed in the late 1990s.

Unit Testing is the driving force here. Rather than writing the code and then testing, first it is tested. This will obviously fail and then code will be written and tested again. This process will keep repeating till the test is successful.

Test cases are typically automated in same language as the implementation language.

The steps involve –

· Writing unit test cases in collaboration between developers and testers

· Executing test cases

· Coding to address issues

· Executing test cases again and so on..

· Testers can do integration tests once Unit test is successful.

Behavior Driven Development (BDD)

This method started getting popular in the early 2000s and acceptance testing is the focus here. This is driven from the purpose of focussing on the expected behavior so that both technical and non-technical people are on the same ground. Typically implemented using many available tools in the market like Cucumber, Quantum, SpecFlow, JBehave etc.

Typically the test cases would be written in a language called Gherkin which follows a Given, When, Then format of specifying anything. Tests are written at scenario level and clearly maintained in feature files.

The end goal is to have a test case in natural language yet in a standard format. This can be understood and implemented by everyone working in the team and ensures end goal is met.

The steps involve -

· Identify the scenario for BDD

· Write the feature file in Given, When, Then format

· Run the feature file in appropriate language

· Check the test results of the run and fix the errors.

· Redo the process

Acceptance Test Driven Development (ATDD)

This also came into focus in the early 2000s. Acceptance testing is again the focus here as the name suggests but the purpose is clear capture of the requirements. This is also usually written in Gherkin and typically the tools used in BDD can be used here also. This brings customers to the testing table even before development has begun.

There are few situations where ATDD can do wonders. As the end goal is very clear to everyone, estimate are more accurate and collaboration improves in the team. It can help when technical debt is high in the project.

ATDD can be effective in reducing rework also due to clarity and save time. It reduces many problems Agile teams typically face.

The steps involve –

· Train the stakeholders on objectives and expectations

· Write the acceptance test cases

· Implement iterative learning

Domain Driven Development (DDD)

Understand the big picture and end goal. Once that is clear, decompose it into smaller deliveries and follow usual SDLC for the smaller pieces. This is the gist of domain driven development which came after all the other methods. This is not replacing any of the above methods but working in tandem with them to improve the process.
Summary

So do these methods exist separately? Is it a choice to pick one of them or all together?

It depends on the project but for a reasonably big one, all of them can be used together. In fact for best results, it may be good to combine some of them together and not just pick one.

Start with the big picture by doing DDD
Once decomposed into smaller pieces, go with BDD or ATDD or both as required
Once further decomposed into unit level, developers need to work with TDD

There may be training period for the team to get accustomed to this but once there, the development process can improve and reduce issues as well.