Forem: DavidTzemach

Managing a Test Manager’s Most Annoying Issues

DavidTzemach — Fri, 30 Sep 2022 10:15:04 +0000

A test manager should perform in multiple dimensions on a daily basis, utilizing various professional and interpersonal skills. Even if the manager is not their team lead, they must provide accurate test estimations, be fully aware of the functions and requirements under development, define the scope of testing, apply appropriate testing measurements, plan, deploy, and manage test coverage, and even encourage and motivate testers.

There are numerous areas that can cause problems with all of these career functions. Here are the most common (and irritating) things I hear as a test manager, as well as the strategies I’ve developed to deal with them.

Simply automate all of the tests

Due to some poorly written articles about the benefits of test automation, stakeholders believe that automation will solve all problems and find all bugs in the code.

The solution: The only solution is education in this case. Inform the stakeholders of the amount of work required to maintain the automated tests, evaluate the results, and develop new ones for new functions.

Here’s a comprehensive end to end Testing tutorial that covers what E2E Testing is, its importance, benefits, and how to perform it with real-time examples.

Just test the essential functions

After changing a small but vital piece of code, the testers are instructed to test “only the main functions, just to be on the safe side.” This request is usually related to understandable time and effort constraints, but it’s still annoying.

The solution: Analyzing the changes with the developers in charge of the function is usually beneficial. I use white-box methods and build the test scripts based on the results with the existing regression test cases. A good Pareto analysis that identifies which 20% of the application is used 80% of the time can also help. If absolutely required, define a testing limit after consulting with project stakeholders about its scope and potential consequences. For this, I frequently use risk assessment matrices.

These requirements are overly complicated

The requirements are too complex to validate and check. The feature should be compatible with the legacy functions, and the cutting-edge solution should cover all bases and corners.

The solution: In these cases, I adopt my analytics skills and map out the new functions using good old-fashioned fieldwork drawing diagrams and gathering information from developers, other business analysts, and architects.

It was working on my local machine

After discovering a serious, potentially critical defect, you are met with the following response from the developers: “But it was working on local machine”

The solution is to write a detailed description of the bug and then delegate responsibility to the developers. Don’t fall into the codependency trap. In the long run, try to emphasize the importance of running unit tests before handing over the item to be tested to the QA.

Take a look at how Appium automation works and see how to perform Appium testing of your mobile applications.

I didn’t have time to go over the results

During the hands-on or other meetings, you discover that the involved parties have not read the test report whatsoever. It’s especially irritating if you spent a significant amount of time preparing it. It’s pointless to blame them; they simply receive far too much information on a daily basis!

The solution: Assume that stakeholders will not read your reports, and prepare for meetings with remarks highlighting the most significant things and measurements.

Shift it from legacy to new

Every company has a good old module or function running in an out-of-date framework that is no longer supported. It must be rewritten in a new (probably fancy) framework, but it should function normally.

The solution: Begin with a detailed mapping of the given function, communicating with the individuals who developed it if possible. On the implementation itself, I’ve always tried to collaborate closely with the developers. This rewrite is also an excellent opportunity to implement some redesign, such as deprecating unused data tables or UI elements.

No test management tool is required

A proper test management solution is not seen as necessary by the stakeholders who provide the funding for the project. No matter how many hours I spent fabricating a test report, management was only genuinely interested in the major defects. And I had to do it manually because we didn’t have a professional test management tool.

The solution is to tie the acquisition of such a tool to the start of a new major project, emphasizing the benefits of using one — preferably with case studies. This approach will make obtaining resources much easier.

Prepare the new testing squad in a few weeks

Leadership wants you to quickly fill the test team for a new project even though you’re short on experienced testers, and they require that everyone be knowledgeable about the product even though there isn’t any available training.

The answer: Spend time with management outlining the present employment market as a solution. While waiting, make an effort to be inventive in your search for new coworkers by combining new advertisements, social media, and instructing interns.
Referral schemes are also quite helpful, and it’s no accident that major corporations like Microsoft utilize them as their primary method of hiring new employees.

Check,mobile emulators online from LambdaTest allows you to seamlessly test your mobile applications, websites,and web apps on mobile browsers and mobile devices.

The testing environment will be accurate enough to represent actual consumer conditions

Even though the test environment is supposed to be an actual representation of the production environment, this is frequently not the case. It can be costly and labor-intensive to set up a QA environment for them.

The answer: I have discovered that it is vitally essential to map the differences between the two ecosystems thoroughly, along with the hazards related to each change. Meaning that If there are an ’n’ number of discrepancies between the two environments, there will be 2n faults among the results and findings. This means that even if the majority of discrepancies are false alarms, you should dedicate a significant amount of time to debugging.

Agile Release Planning Explained

DavidTzemach — Fri, 30 Sep 2022 08:47:34 +0000

The Agile manifesto describes the core values and principles that, at least in theory, should guide the whole process of Agile development. One of the core aspects of this manifesto is the continuous value that organizations should provide to their customers. Principles like:

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

“Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”

“Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.”

Now, in theory, it may seem to be a great way to create happy customers by providing them with a production-ready product at the end of every sprint. However, providing a working product per sprint is just a narrow way to look at it. As you already know, the project may contain more oversized Product Backlog Items (PBIs) such as Epics and features that will take several sprints (e.g., four to twelve sprints) before they can be released.

Learn what Selenium is and its benefits for automated cross browser testing. See how Selenium works and get ready to use it for testing.

What Is Release Planning?

Release planning is a dedicated meeting for organizations in Agile projects to create a high-level plan for multiple sprints ahead (e.g., four to twelve sprints) and not for a single sprint as it is usually done in the engineering teams. Release planning is vital as it allows all significant stakeholders to assess critical aspects of the project (e.g., risks, resources, tools, etc.), set the initial expectations about which features will be included in each release, and set the goals which the customer/business wants to achieve. In addition to the above, the release planning session allows the three main pillars of the project (i.e., customer, business, and engineering) to gain vital information that serves as a base to track progress within the project timeframe.

A very common approach to determine the amount of time it will take to deliver a specific number of features within a release is to use a simple formula: the number of features within a release / expected velocity = The number of sprints needed to complete the release scope.

Long-Term Planning? In Agile?

In Agile software development, we want to avoid the unrealistic demand to plan the whole release upfront. This is just not the way we do things in Agile. There is no need to invest in large plans that will most likely be a complete waste of time. Now that has been said; we need to remember that we cannot run an Agile software development project without any upfront planning; we must have the minimum understanding of what the customer is expecting to receive and how we will deliver it.

The Prerequisites to Release Planning

The project release planning is a crucial meeting that involves many stakeholders, and we do not want to waste their time. A good approach is to use ‘Entry’ criteria for the meeting that specify simple prerequisites such as:

There is a clear definition of the market objective.
The high-level vision of the product.
A prioritized product backlog including the relevant features.
The average velocity of the Agile teams that will do the work.
List of risks, dependencies, and any gaps that can affect the project.
List of goals and expectations.
Available resources that will take part in the project.
A clear definition of the project scope.

Are you using Playwright for automation testing? Run your Playwright test scripts instantly on 50+ browser/OS combinations using the LambdaTest cloud. Sign up for free!

Goals of Release Planning

Now that we know what release planning is all about, let’s continue and review some other goals that the business should aim to achieve when conducting a release planning session:

To identify any missing information that can influence the release.
To determine the release testing strategy.
To identify any technical limitation that must be addressed before the beginning of the first sprint.
Coordinate with development teams to synchronize their plans.
To get high-level estimations with story points.
Risk identification.
To identify project dependencies (external providers, other systems, etc.).
Synchronize all participants with critical epics and features that will be part of the project.
To agree on critical dates and milestones of the release.

The Agenda of Release Planning

The agenda of release planning and how to execute it can differ between organizations; there is no “one” way to do it that guarantees it will achieve its goals. However, there is one general (and very simple) process that you can use to run the meeting that can be good enough in the beginning and then improved. Let’s examine its different phases:

Phase 1: Presenting the Meeting agenda

In the first phase, the Product Owner (Or any other stakeholder) will review the meeting agenda, including its goals and the expected outcomes at the end of the session. To ensure that all the participants are familiar, it can be a good idea to perform a short cycle of introductions when there are new/external stakeholders.

Phase 2: Reviewing the release scope

In the second phase, the Product Owner (PO) should explicitly review the project scope and features that should be part of the release. In addition, this is usually when the PO provides a high-level description of the project roadmap (Including its key milestones), business urgency to deliver, and the product vision.

Phase 3: Reviewing the current project status

If it’s not the first release planning session, then previous work is already done. A good practice is to provide “high-level” information regarding the current progress and what has been achieved until now. In addition, the review usually contains the following data:

Feedback from various stakeholders on market conditions.
Expert opinions related to technical challenges.
Velocity from previous releases.
Technical Debt that must be addressed within the release timeframe.

A comprehensive end to end Testing tutorial that covers what E2E Testing is, its importance, benefits, and how to perform it with real-time examples.

Phase 4: Identifying risks and concerns

In this part of the meeting, engineering teams have the chance to present any risk, concerns, gaps, or any other issues that can influence the project release scope. Once all items are identified, there should be a mitigation plan and an owner to ensure it gets done.

Phase 5: Reviewing, updating, and defining critical artifacts of the process

This part of the meeting should be dedicated to reviewing, updating, and defining critical artifacts of the project (e.g., DoR, DoD, Available Resources, etc.).

Phase 6: Preparing a high-level plan

It is now time to start working on the release scope in detail. Once there is an agreement on features and stories that will be part of the release, it is time for the development teams and their Product Owners to start planning. Let’s look at some of the everyday activities related to this phase:

Teams should work with their Product Owner to discuss any non-functional stories they will need to deliver the release scope.
Teams will start determining their overall development and test strategy to meet the release scope.
Product owners will prioritize the team backlogs to meet the release scope and significant milestones.
Discussions are had between different teams to handle technical/project dependencies.

Phase 7: Confidence and commitment

This is the time for the teams to share their overall plan and to approve their commitments.

Phase 8: Meeting retrospective

A valuable practice in this meeting is to take the last 15 minutes and conduct a retrospective. The retrospective is concise and usually focused on a single question “How can we improve the next release planning session?”

Agile Test Planning - The Levels of Precision

DavidTzemach — Thu, 29 Sep 2022 11:53:36 +0000

A frequent misperception about Agile development is that it requires less planning or no planning; this misperception is mainly related to one of the core items in the Agile manifesto stating that we need to “Respond to change over following a plan.” While this sounds good in theory, it can’t hold water in complex software development projects (we can use it as a mindset and try reducing planning but cannot ignore it).

Agile teams tend to plan all the time in the different phases of the project; yes, it’s different from other software development methodologies where the planning is mainly done at the beginning of the project, while in Agile development, it’s done in smaller chunks, with focusing on what is needed to complete a feature, story or task.

In addition, the whole planning process is done with a clear goal to achieve more than just the planning plan; we use the planning sessions to generate fast feedback to learn and adapt, to make future planning sessions more effective and less time-consuming. These Agile planning efforts and strategies are relevant to all Agile activities and therefore used in test planning; we plan our tests in small chunks and short feedback loops. As the team plans to test, the focus is on what they need to know now.

A worth mentioning point for test planning is the size and complexity of the project. In small, experienced teams, planning for testing can be simple. They have the tools and knowledge to consider the type of testing, their automation requirements, and how much effort the team will take to complete the testing. On the other hand, test planning can get complicated in large organisations where each project involves multiple teams working on the same product. However, the basic principles apply regardless of the project complexity and size.

In Agile projects, we don’t know all the details about each Epic and Feature at the beginning of the sprint. Some may be predictable and straightforward because we’ve done similar work before, allowing us to understand the complexity, effort, and testing needed. Others may be more complicated, or they may be so new and unknown to the team that it’s challenging to come up with the information to handle the testing challenge. In Agile Development, we should use a different mindset from the one that assumes that we will have all the requirements upfront and where planning is based on that information.

Using different Agile frameworks, we can do exactly this by using the principles of iterative development and simplicity; we look at levels of planning and ask what information the team needs to know at each level. Different organizations will have different contexts, so you need to differentiate between what is essential to your team and organization and what is not.

We discuss this approach within the boundaries of sprints used in the scrum framework, but we can also apply the concepts to flow-based methods such as Kanban. Team using flow-based methods may have the possibility to release a working increment after the completion of each story, or they may accumulate the stories until a feature is complete and then released to production.

In the Scrum framework, most likely the most commonly used Agile framework in the industry, the organization, and its teams will have four primary levels of details that they will have to consider when planning while at the same time keeping the whole product in mind. I use the following terms for these different levels of precision for test planning:

Epic: One or more teams working on a single project releasing at specific intervals or on defined dates. An Epic is a collection of many features.

Feature: A feature is a collection of user stories representing the business capability of a piece of functionality that is useful to the business; A feature completion time can require multiple iterations to complete. A feature delivery may involve many teams working on different parts.

User Story: A small testable chunk of functionality created by the team (Non-Functional requirements) or by the product owner (Business Requirements). Regardless of the type of the story, it should be completed within a single sprint (in my teams, I set a limitation of 2–3 days). A story is a group of many tasks.

Task: A piece of work that’s part of a story, defined only by the development teams and taking less than one day to complete.

Let’s explore how you may adapt your test planning to each level, as well as discuss the level of documentation, level of risks, and artifacts that might be expected at each of the four levels I shared above. For connivances, let’s assume we have a three-month product release (a yearly quarter) with two-week iterations using cross-organization. During this time, the teams are expected to deliver multiple features that each team is responsible for delivering in the release cycles.

At the Epic level, teams should already have a good idea of the product vision and its goals to answer the questions of “Are We Building the Right Thing?” The high-level test approach should cover the main testing aspects of the product because this is the time when you are planning what will be in the release scope. Due to the importance of planning at this level, I always ensure that my teams will have a simple checklist of questions that they can follow to ensure they run an adequate test planning session, for example;

What are the test environments you will need?
Are there any new technologies that the teams will have to learn before starting the project?
Are there any new tools that we should purchase?
What types of test types you will use.
What is the planned coverage (Automated Vs. Manual)
What will be tested during the different phases of the release pipeline?

These activities may involve the help of more departments and groups outside your delivery team, such as marketing, Support, operations, and more. For example, you may need the system team to run endurance and security tests that the team will not cover during development or ask the finance teams for a budget to purchase new hardware/software that the teams will use during the testing process.

Epic

A product delivery cycle of a Single-Team
When your team is working on a feature/story, they are responsible for completing all testing activities (Design, Documentation, Execution, and Analysis). The team may still need external help, such as security experts, but the team is accountable. When you have a single responsible team, the meaning is that the project is usually tiny and is limited to only a few iterations.

A product delivery cycle of multiple-Teams

Having multiple teams work on the same project can (and usually will) add complexity. Therefore, an overall test approach should consider dependencies between code parts developed by different teams, testing only their components or features associated with the project. As you may guess, test planning at this level is crucial for achieving good results once the teams will start to develop and test the features and stories associated with this project.

One practice I usually do in my projects is to create project “maps” that help all stakeholders in the project; these maps are typically created in a “Mind map” tool which helps visualise the different aspects involved in the testing effort. Also, when creating these “High-Level” maps, I involve vital stakeholders who understand the big picture and how this release integrates into the system.

Below are a few examples of the lists you can use at the Epic level:

Map of owners and responsibilities per area of testing.
Map of all dependencies outside the product teams or between features.
Map for all risks related to each testing effort and mitigation plan.
Map of Test environments and Tools that the team will have to use in the project.

Creating these maps will contain all the teams’ information to work on their features. It continually allows them to look at the big picture and aim to find possible integration issues early and before the testing effort t is started. This will increase communication and reduce the risks.

Consider the investment return for all the time you can save by helping your teams start working on their features while knowing what challenges they may have and consider it part of their test planning efforts. For example, teams with many dependencies with other teams will know that they will need to work together on their code and test the integration between the different components.

If it’s possible and you have enough information with the right group of people, you may think about creating a product release test plan before the start of delivery cycles; I recommended keeping it as lean as possible and straightforward as you can. Focus only on the high-level aspects of the test plan (the teams will do the detailed test planning). Consider who the audience is and what is essential to know before creating their test strategy and detailed test plans. Also, at the Epic level, test planning should include specific topics I already described above, such as: Determining the test strategy and identifying testing risks.

Before moving forward to the feature level, the last thing worth mentioning is that teams may not have enough information to create a solid test plan. Therefore they may decide to defer test planning until after any necessary investigation (primarily by using spike solutions) and more about complex features.

Learn what Selenium is and its benefits for automated cross browser testing. See how Selenium works and get ready to use it for testing. Read more.

Feature level

Features are work units driven from their Epic representing some business-value capability broken up into user stories. We need to make sure that these stories (functional and sometimes non-functional stories) Are testable so that the team involves in the project gets into a rhythm of coding and testing each story until the complete feature is “done.” Remember, a feature is a set of stories, and sometimes there can be many of them. Each delivery team concentrates on a particular feature without always knowing what other teams are doing; this uncertainty increases when multiple teams are involved (I used to work on a project with more than 50 teams working in parallel). There may be dependencies Between the teams.

If we take Scrum, for example, each scrum team works with its product owner (PO) to determine how much work the team can complete during the delivery cycle base on their capabilities, capacity, etc. in a scrum, the team will hold a planning session so the team can understand the goals they need to achieve and working with the PO to understand what are the stories they need to deliver base on their priority (set by the PO). And what about the testing activities? This is where testers usually add their value by asking clarifying questions that may help the team determine the size of the feature or story to understand the actual effort involved in delivering it.

Testers can also help the team consider the impact on other system components and issues while integrating the code with other parts of the system. Also, they promote a testing mindset to what security or performance concerns there might be. As I mentioned at the Epic level, feature-level testing involves different assumptions and risks that need to be monitored and documented in a dedicated repository (Wiki, mind maps, Confluence, SPO, etc.) visible to engineering and business stakeholders relevant to the project. Watch how the teams use this information, and ensure you keep improving it using retrospectives and other dedicated meetings to ensure you are using the correct format. If there are communication gaps, look for the best fit for your situation.

Sometimes, Mangers or clients will demand a formal test document outlining what your team will test in the delivery cycle. Before you create a formal document, I recommended that you write it in a way that allows other non-technical stakeholders to understand it and keep it as simple as possible (One or two pages without getting into details of scope unless explicitly requested). You must think about who will use it and why they want it. In my projects, if this document is not asked for, I still try to ensure that any testing risks will be written and shared with anyone interested (my preferred option is to create a mind map capturing those risks and any other information I think is essential).

When it is time to break the feature into stories, I work with my team’s product owner to create high-level acceptance tests, using specific examples of expected behaviors and off-course misbehaviors for the feature. Following this approach, you can help your team focus on the proper scope of work and keep the business value as high as possible. Try to use a mind map as much as possible (it will be worth it, this I can promise you) for creating a testing strategy for the feature with your whole team before they will, break it into smaller parts that will be executed as part of the iteration’s stories. It is one way to start a discussion and allow all team members to contribute and reveal hidden gaps or potential issues before they arise.

As you probably know, planning testing activities for extensive features containing multiple user stories that cross iterations can be complicated. To help the team, I instructed them to add a dedicated “Features testing” story for these. The story will contain tasks related to testing activities, such as “Creating automation Strategy,” “Set Test Environments,” “Explore the feature,” “Define testing efforts,” and more. This type of story ensures that the feature testing is done and calculated as a team effort in a given iteration.

Are you using Playwright for automation testing? Run your Playwright test scripts instantly on 50+ browser/OS combinations using the LambdaTest cloud. Sign up for free!

Story Level

Once we set the test plan at the feature level, the team will start working on the story level, moving from the “Macro” to the “Micro.” At this level of precision, we need (per story) to set high-level acceptance tests: an example of expected requirements and at least one example of misbehavior that defines the story’s scope and the necessary tests the team will have to run to validate it. The planning for the story levels happens during story reediness sessions (sometimes called refrainment or backlog grooming), which help the team create tests for the story and can be expended during the iteration planning sessions.

So if want to simplify my suggested process, we will use the reediness sessions to start writing test ideas based on the information and knowledge you already have at the feature level, continue during iteration planning, and then expand it to other tests that will guarantee that the required functionality works as expected.

A comprehensive end to end Testing tutorial that covers what E2E Testing is, its importance, benefits, and how to perform it with real-time examples.

Task Level

When thinking about testing activities at the task level, there are fewer planning and execution-related activities, such as creating test data, setting up environments, writing unit tests, and more. Some teams estimate tasks during iteration planning, but I find this a waste of time as the estimation should be only on the story level. However, this can be useful for teams that are new to Agile. When the iteration starts and tests are executed, make sure to call attention to it during daily standups by reporting progress or asking for help if the testing task takes significantly longer than anticipated.

Agile Development Best Practices for Determining What to Automate

DavidTzemach — Tue, 27 Sep 2022 12:34:36 +0000

Agile teams will not prosper if they don’t use frameworks to automate various development processes, including tests, builds, test environments, etc. Understanding that the primary goal of all agile teams is to successfully deliver real value to the customer ,which can be interpreted as providing functional software after the sprint, is necessary to comprehend why a lack of automation results in failure. In this article, I will address the central questions of “What can and what we shouldn’t automate in agile projects”.

What can we automate in an agile environment?

Most types of testing benefit from automation, starting from basic unit tests through system tests. But, as you know, unit tests don’t supply enough coverage to catch system regression failures. Running a set of manual tests before every check-in, which could be dozens of times a day, isn’t practical. When developers can’t run tests by pushing a button, they will not be motivated.

Let’s talk about the kinds of tests that are most suitable for automation. Automation starts with an automated framework that allows developers to check their code often and receive quick feedback about its impact. So let’s start with this first.

Check out the most wanted tools for automation testing that have climbed the top of the ladder so far. Let’s take a look.

Continues Integration (CI) system

This is where we create the most important, albeit logical and straightforward, ground-rule for automation. Due to the nature of agile software development, which is faster than any other approach, agile teams should focus on automating any repetitive or tedious work involved in developing software.

And no candidate is better for this than automating build creation as part of an agile development process. Due to the fast nature of agile development, the team should create numerous builds per day, especially to test newly added code.

CI systems are crucial in an agile environment. Continuous integration and build processes are the two systems that give the most significant ROI of any automation effort:

CI allows for immediate feedback at the unit-test level (if you have the relevant unit tests to support it).
It reduces many of the risks involved in adding new code without testing it.
It allows the team to create and deploy numerous (stable) builds and provides for multiple check-ins per day.
It improves communication because team members can receive a notification once the build is ready without checking the status.
CI systems speed up testing time by reducing the number of errors at the unit level before these errors become apparent in advanced phases of the testing process.

Based on the above, agile teams must implement continuous integration and build the framework as soon as possible. Although it requires continual maintenance, it’s the only option for agile teams to succeed and reduce technical debt in large complex projects.

Based on these simple facts, you may see that agile teams must implement continuous integration and build the framework as soon as possible. Although it requires continual maintenance, it’s the only option for Agile teams to succeed and reduce technical debt in large complex projects.

Development and Test Environments

Agile teams need to test and develop in a fast-changing environment; as a result, there is less room for the creation and maintenance of work environments. Agile teams can use the automated deployment of their environments without multiple hours of manual work. In addition, the team can use automation to handle many other areas related to their work environment:

Creation and cleaning of the testing data and configuration.
Setup of specific topologies and architectures.
Simulating a particular scenario for reproducing a bug.

Here are 30 Top Automation Testing Tools you can use in 2022.

Testing of the User Interface (UI)

The agile development process embraces the approach that the team must deliver an incremental working functionality at the end of each iteration; as a result, the team will usually execute basic automated regression tests at the GUI level.

As I mentioned earlier, I’m a great believer in automated testing. Still, in some cases, we need to consider whether we want to use it, especially when we want to test the user interface of an application whose GUI changes.

To overcome the challenges of GUI testing, there is of great importance in selecting the most suitable tool for the job, one that’s easy to maintain and flexible enough to absorb changes. This is probably the most critical key to successful GUI automation.

Testing all layers of the application

I’m a great believer in automated solutions that can reduce manual testing efforts to the bare minimum necessary. It starts at the first layer of the application by running unit tests that we all agree are crucial in reducing problems that won’t become more significant problems later when found in that layer of testing.

Next, we have the second layer of component tests. Programmers test components as a whole module by using a set of inputs and outputs that the module produces. The third and, for me, the most crucial part of the testing strategy is integration tests, where modules are tested together as one suite. And if that is not enough, why not test the whole system by running the fourth layer of system tests, which test the entire application as an entire system.

Perform browser automation testing platform on the most powerful cloud infrastructure. Leverage LambdaTest automation testing for faster, reliable and scalable experience on cloud.

Performance, Load, and Stress tests

Suppose you’ve ever been involved in the testing process that included one of the testing mentioned above types. In that case, you probably know that it’s almost impossible and undoubtedly ineffective to use manual testing methods as the preferred way to run them. Furthermore, there is a wide range of tests that you cannot run without automation tools. In addition, using manual tests will not provide the accurate test results we can achieve by using dedicated automation tools that can simulate the exact scenario without any human interference that may affect the testing process and, therefore, the results.

What shouldn’t we automate..?

If you are familiar with how I approach testing, you are probably already aware of my strong support for automation frameworks that enable the team to develop a more effective, efficient, and dependable coding and testing process. However, some parts of the testing procedure still require the intelligence, common sense, and eyesight of humans.

Usability Testing

Usability testing is very different from other test types that determine the quality of the software. It cannot be automated because it requires someone to work with the software to determine how it was experienced and where the gaps are in the user experience.

GUI Testing (Is it worth the ROI?)

GUI testing is one of the most challenging areas to automate. I’ve seen just too many organizations that invested thousands of human hours in automating the GUI of their products but, in the end, found it was a waste of time that didn’t provide the expected ROI. Some GUI tests can be used to ensure there are no unexpected changes in the GUI. Still, you should ask yourself whether it’s worth the costs and investment instead of improving other quality issues that will provide a better ROI and reduce the risks in that area.

Tests that are not worth the investment

I used to be involved in an automation project where the test team automated a thousand provided tests (at least on paper). So what is wrong here? They automated almost all the available test scenarios without really thinking about the ROI.
The team invested weeks in automating tests marked as low risk, tests that would never fail, and tests whose failure had a meager chance of impacting the software. The entire automation process was based on the spirit of “let’s automate everything” instead of asking the simple question of the ROI that this automation project provides.

In some cases, some tests are written without real thought as to whether they are essential or not. Once the automation project starts, the team will automate these tests just because they never want to rerun them (because they know there is a 0% chance that it will make a difference).

Tests that need to be executed only once

The main goal of automated testing is to allow the team to focus on essential things in the software development lifecycle. Automating test scenarios that will run only once are not worth the team’s time to invest in the design, creation, and execution of these tests.

Exploratory Testing

In my opinion, exploratory testing is the best and most efficient method that agile teams use in any testing process. Exploratory testing can be used for learning purposes (you learn more about the software when testing it) or to provide a fast way to evaluate the overall quality of the software. However, when it comes to real testing effort, a skilled tester must design and execute tests.

Exploratory testing should be done by humans and not by automated scripts because automated scripts will not let the tester take in new information he generated from the exploratory session and use it to improve future testing and development processes.

In addition, although exploratory testing is a great testing approach, there is a real need for automated tests that will allow the team to focus on their exploratory sessions without worrying about any regressions that automated tests should cover.

Agile Development — Are we building the right thing?

DavidTzemach — Tue, 27 Sep 2022 10:33:55 +0000

I think that I can say for a fact that most Agile teams (and especially their testers) feel there is never enough time to test. But for me, testing the software is only one part of the equation. I often saw how organizations waste time developing software that doesn’t meet their customers’ highest-priority needs or products whose cost of investment exceeds their value to the company.

I worked with some great development teams who have mastered the most advanced technologies and practices, allowing them to deliver robust products throughout the years. Most of the customer issues found and reported in this project turned out to be missed or missing requirements caused by a lack of communication/set of expectations.

Also, now you can perform browser test automation on the most powerful cloud infrastructure then leverage LambdaTest.

Always ask “Why”

In my experience, one of the most critical ways that Agile teams and their testers can contribute to their customer is by making sure they focus on the business purpose of each new feature they need to deliver. When we can ask why customers request the feature and the problem they are trying to resolve, we’re more likely to build the right thing. During feature review, the team (Engineering with Product) will discuss different aspects such as feature requirements, goals, acceptance tests, and how to measure success. With this discussion, the team can create a roadmap and set business and technical milestones.

A constructive discussion about “why” we are building this product or why our customers need it, involving both product and engineering team members, frequently leads to the realization that the customer isn’t asking for a product they actually need or that what they are asking for won’t solve the problem. This is an essential conclusion as, in the end, if the customer doesn’t get the right solution, the business may suffer.

This is because the ROI and delivered functionality are vital components of the product’s business value. This is why it is so important to focus on the “Why” before diving into creating the functionality. So, when working on a new business requirement asked by the field, keep thinking about building the “right” thing and any other questions that will help you keep product quality high.

Practices for Customer Engagement

Understanding the “Why” can be difficult in most cases as the customer is not always available for the team. So, how can we overcome this barrier and answer the “Why”? There are many effective practices to help the business and delivery teams figure out the right things to build.

I proposed some tools for examples and requirements in my book “Agile Quality: A Practical Approach,” including mind maps, checklists, flow diagrams, and more. Here, I want to focus on Impact mapping as an additional practice that I found valuable for understanding customer needs and using them to increase the value we deliver and create tests that guarantee the quality of the product.

Leverage LambdaTest automated testing for a faster, reliable, and scalable experience on cloud.

Impact mapping

Impact Mapping is a strategic planning technique (introduced by Gojko Adzic in 2012.) I use it in many organizations during product development to visualize the impacts that various people (in the method they will be addressed as ‘Actors’) will have in contributing and helping us achieve a common business goal.

A goal might be something simple such as implementing a specific tool that will promote discussions, something you want to achieve as a team (for example, better RCA and retrospective processes ) you want to promote, or it might be something big you want to achieve such as implementing a new development framework that will take weeks and even months.

Impact Mapping has been adapted from other planning and brainstorming tools such as Flow-Diagrams, Mind mapping, etc. (all covered in my Book “Agile Quality: A Practical Approach”). Impact Mapping is a light, simple, and structured way to make sure the team maintains their focus on the purpose of what to build, and it helps identify the most valuable features to build first.

The focus shifts from “What are we going to do?” (as I usually see in organizations at the beginning of a new project) to “Why are we building this, what is the problem it will resolve, what is our goal, who can help us promoting the project and what risks can getting in our way” For, me the most significant thing about it is that it increases and encourages every participant in the room to focus on the goal and not waste time on bureaucracy and other political issues that may influence the decisions made at the end.

At the end of the impact map process, you should have a diagram that visualizes the process outcome, including:

The goal the team wants to achieve or problem statement.
High-level prioritization of project scope that could be delivered.
What impacts (if any) the development process and how.
List of people in the business who help or hinder our ability to achieve our goal.
List of potential impacts and risks that may affect the project.

User Story that translates deliverables into specific features to be implemented.
An impact map is a visual mind-map structure used for answering four fundamental questions:

Q1: Why? — The central node that starts the whole process is used to answer “Why are we doing this” and “What is our Goal?” Your goals should define the problem or requirement to be solved and are usually presented by someone who represents the business (Product Owner, project manager, etc.).

For Example:

Increase gross margin.
Reduce field tickets.
Increase the number of monthly Active Users by x until the end of the year

Q2: Who? — This node is used to describe the people (Individuals, Roles, and Key stakeholders) who can make an impact (who can help us, and who is getting in our way?”) on the outcome. And if we simplify, we need to map the people who can bring the team closer to achieving the goal or prevent us from reaching it.

For Example:

Support team
Existing customers
New customers
Customer success agents
Marketing department
Finance department

Do you know which are the most wanted automated testing tools that have climbed the top of the ladder so far? Let’s take a look.

Q3: How? — During this stage, the team will identify what they want out of each actor to help promote the team from achieving their goals. So, a good practice to generate this list is to ask questions like “How could our actors’ behavior change to help us achieve our goal?” or “Which behavior is most likely to get us to our goal?” Or “Which behavior is most likely to add a negative impact on our ability to reach our goal?

Let’s say that you want to increase awareness around your company website. An ideal impact would be getting users (Identified in the previous node) to spread the word about the site to their friends using a subscription program.

Q4: What? The fourth and last step of the impact mapping process identifies the deliverables (Tools, Features, Processes, and Business activities) that actors will use to achieve the desired outcome. More importantly, they represent what the team can do to help them.

For example, let’s say that you’re managing a technical blog that helps users increase their knowledge. Your goal is to boost traffic from new users. The impact you want is to get users to access your app more frequently.

So, one of the ‘Features’ that can help you achieve this goal could include an E-mail campaign or push notifications to potential readers. It will allow them to remain updated with new content uploaded to the blog and give them a reason to return to the blog.

Who should attend this session?

I have a rule for my teams that each meeting should include all stakeholders to promote the meeting goal. So, as long as you keep it in mind, for this session, I would usually expect to see the “Product Owner,” “Project Technical Owner,” facilitator, and project sponsors (Business and Technical).