Background: A leading e-commerce platform regularly introduces new features and updates to stay competitive in the market. However, the rapid pace of development often led to regression issues, impacting user experience and revenue. As it was a short-term project, the test automation effort was not done.
Strategy: The testing team adopted an agile approach to regression testing, aligning testing cycles with development sprints. As usual, the sprint lasts for two weeks. Test planning was done at the Sprint planning stage itself. The effort required for testing the stories and bug verifications was decided and the QA workload for a particular sprint was identified. A test plan was created outlining the testing activities to be performed during the particular sprint. A high-level test strategy and approach for testing was developed by identifying testing objectives, scope, and business priorities.
Prioritize Test Cases: The team prioritized test cases based on the risk associated with changes and maintained a comprehensive regression test suite that had some repetitive tests. Identify and prioritize test cases based on their criticality to the application's functionality and the likelihood of regression. Focus on testing high-impact areas and critical business processes first. With this approach, we completed the testing of risky areas first. It helped identify the critical bugs first.
Manual Execution Strategies: Optimize manual test execution by employing efficient testing techniques such as pairwise testing. Leverage experience-based testing to uncover potential regression issues intuitively.
Pairwise testing empowered with experience-based testing was one of the best strategies we used. Because we had to use many data inputs to test the e-Commerce system. This methodology helps reduce the time and required resources for testing.
Test data management was done carefully to avoid the drawbacks of pairwise testing. The team prepared and maintained relevant test data sets required for regression testing. Ensure data integrity and consistency across test cycles to replicate real-world scenarios accurately. Because identifying the right set of inputs is a challenging job here. If this part was not done well, then the required test coverage won't done. However, it managed well with e-commerce experts who have industry experience in testing vast e-commerce systems. Apart from using this methodology for functional testing, the team heavily used this mechanism in cross-browser and cross-device testing.
The benefits of all pairs testing are many. One of the main benefits of pairwise testing is that it can help identify defects that may not be discovered through other testing methods. For example, consider a system that allows users to customize the search filter based on size and color. Without all pairs testing, attempting to test every potential combination of these variables would pose a considerable challenge. However, with pairwise testing, testers can swiftly and effectively generate test cases covering all feasible combinations of size and color, including other search filters. This ensures that the software operates as intended, minimizing the likelihood of users encountering unexpected issues or errors.
Challenges and Solutions: As you know, challenges are part of this job and they're unavoidable. There were instances during some sprints where unexpected stories and bugs arose. In such cases, we had to change our plans. As usual, such situations caused frustration and tension within the team, and I won't deny that reality. It made them crazy. Consequently, the team had to prioritize and address the new stories and bugs over the previously planned ones. Hence, the team had to override the planned stories and bugs with new ones. As a result, some stories were dragged to the next sprints.
Avoiding unplanned stuff for the sprint was not unavoidable.
Therefore, to overcome this challenge, fostering open communication and collaboration among cross-functional teams, including developers, testers, business analysts, and stakeholders was helpful for us. Share insights, feedback, and observations to facilitate a collective understanding of regression risks and mitigation strategies. The impact associated with the bugs and new features were identified and impact analysis for testing was done accordingly. Keeping proactive communication helped the team to avoid misunderstanding the releases and requirements.
The delays in the development tasks were another challenge. Because within two weeks, the QAs also need to finish their jobs. Therefore, releasing the features and bugs at the eleventh hour gave a huge QA workload to do within a short time. The finding of bugs towards the end of the sprint added significant pressure to everyone involved, requiring additional testing cycles.
You know uncertainty is everywhere. Therefore, as Quality Assurance professionals, making timely and informed decisions is crucial. If you recall the Money Heist series, the professor exemplified this concept effectively, demonstrating the importance of remaining calm and making decisions under pressure. I think similar credit should go to QAs. :D
Results: The agile regression testing approach significantly reduced the time and effort required for regression testing, enabling faster feedback loops and accelerated time-to-market for new features. Moreover, it enhanced software quality by catching regression issues early in the development cycle, leading to fewer post-release defects and improved customer satisfaction.
How's your experience in Regression testing with the agile approach?
Let's discuss your strategies, the challenges you faced, and the solutions you take.
Thanks for your time.
@Vishaka Warnakulasooriya In my experience, Regression testing in Agile is essential for maintaining product quality amidst rapid changes. We integrate Regression testing seamlessly into Agile processes, leveraging automation and continuous testing. Collaborating closely with teams, we prioritize tests based on risk and business value. Automation streamlines repetitive tasks, allowing more focus on complex scenarios. Continuous improvement through feedback loops ensures our Regression testing practices evolve with each iteration, contributing to project success.😊