For an agile project manager, regression testing often becomes a pain point. Two major challenges have been presented in this type of test:
Increased time and efforts. Regression testing can take a lot of time, especially in bigger projects. Sometimes, teams have to spend a full sprint on regression, which is hardly acceptable. In addition, the increase in time and efforts means the cost has increased.
Lack of attention/concentration. Repeat performing the same test cases for a long time results in lack of concentration. This can cause bugs in the production, which also increases the overall project cost.
In order to solve these problems, experienced vendors of software testing services and QA follow certain balanced strategies. Let’s explore some of them.
Regression testing in Agile: How to optimize?
There are three most common methods of optimizing regression testing in Agile Development:
- Test Priority (Risk-based and collaborative approaches)
- Two-level approach
- Automation approach
Each of them has its own advantages and disadvantages.
Following this approach, a testing team breaks regression testing into two cycles:
After this approach, a test team breaks the regression test in two cycles:
1. Repetition Regression The team perform iteration regression at the end of each sprint. Iteration Regression is focused specifically on features and changes made in the recurrence and areas of the application that could be affected.
2. Complete Regression. Test engineers carried full regression before releases and project milestones in order to ensure that the application works as per planned.
However, this approach is not as easy as it looks like. To be successful, it requires an uninterrupted communication within the project team. Communication with the developers, the test team will be able to get a broader view of what was done in repetition and will make a well-liked choice about the essential type of regression testing.
Another thought is that the time has passed since the last full regression.
It may be useful to set the schedule for regular full regression and execute it despite changes made in a particular repetition. This will help the team avoid tiresome full regression after every sprint and will keep the application relatively bug-free all the time.
Test prioritization: a Risk-based approach
To apply a risk-based approach, the testing team selects test cases covering the application areas affected by recent changes in the project and ranks them according to priority, which reduces regression time and efforts.
As the project proceeds further, the risk-based approach can be applied to the Agile Regression Testing Suite. The test team divides this test suit into three categories:
High priority. These test cases make about 10% of all regression test cases. They are critical functions of the software, the areas that are highly visible to the users, the defect-prone areas and areas that go through many changes.
Medium priority. These regression test cases (about 30%) describe exceptional situations (negative test cases, boundary value test cases, etc.). This priority applies to test cases that frequently detect bugs in previous product releases.
Test cases of high and medium priority are always included in the new agile regression test suite.
Low priority. These test cases (remaining 60%) cover the rest of functionality. In regression testing, testing engineers use them before a major release to ensure full test coverage.
Thus, allotting priorities helps in reducing time and effort on regression testing in Agile, with no harm to the quality of the product. However, there may be some problems with this approach, because the testing team is limited to their own tasks and can’t see the whole picture. Fortunately, there is another strategy that is totally related to Agile specifications.
Test prioritization: Collaborative approach
This approach allows team members to contribute their knowledge and unique viewpoint to regression testing. During a sprint, the team acts as a regression board covering every piece of work done by each team member. Members of a team can make minor changes based on their knowledge on the board.
For example, a developer understands that the recently added feature can affect user profile, so he adds this information to the board. When it’s time to run a regression test, the team consults the board and assesses time investment. Using this approach, the project team applies their unique perspective and knowledge to regression testing, which makes this type of test more efficient and less time-consuming.
Although acceptable and convenient, priority approach should be cautious, because they can become the cause of technical debt, a trap undermining overall project implementation.
Warning: Technical debt
Technical debt in agile development demonstrates the necessary efforts to eliminate all types of unfinished work heaped up during a sprint/sprints/project lifecycle. Typically, technical debt appears when the team decides to “fix it sometime later”. And in some cases it makes sense.
For example, if the team postpone security testing to some sprints when the system is more stable, then it goes into technical debt. But this is a reasonable strategic step set by intelligent time management.
Problems arise when technical debt is out of control. Disciplined Agile Delivery (DAD) framework provides a wide range of steps to effectively manage technical debt. Surprisingly, continuous tight regression testing is one of them. It brings us back to drawing board.
So it is possible to optimize the Agile Regression Test as well as to control the technical debt? The answer is in the Regression Test Automation, the third optimization strategy.
Automatic regression testing in an Agile environment is the best way to deal with the most common challenges this type of test presents. Automatic regression testing decreases the test time from day to hours, and spares/reserves a testing engineer the tedious task the same test cases repeatedly. Still, automation of regression testing in Agile should be treated with caution.
The best thing to do when planning a repayment automation is to wait for the right time. When the project has already reached some level of maturity, it is better to start an automatic regression testing, and major changes have already been done. In the early stages, most projects make changes in the UI, product flow, or logic in each iteration, which increases testing efforts (both manual and coded).
Apart from this, the automated regression test suite, although comprehensive, may not always be valid. In Agile, automated regression testing demands that the testing suite completely reflects changes and updates in the application functionality. It is reasonable to review it after each major release and discard the obsolete test cases which no longer contribute to product quality assurance.
Nevertheless, with the proper autotest suite, there’s a manual testing to stay. Manual regression testing that thoroughly detected bugs in previous iterations. In addition, experts recommend confirmation of automatic regression results with quick manual smoke tests to identify false positives/negatives.
Focus on communication
A testing team should ensure smooth communication with the other project team members to optimize the Agile Regression Test:
- BAs: To monitor changes in requirements and to assess them with a testing perspective.
- Developers learn that they learn the changes made during a particular repetition. This will help in balancing efforts to assure acceptable quality without undertaking excessive testing.
- PMs to update them on the current situation of testing and to avoid time crunches at the end of the project recurrence.
A common ground is there for all the four (or three) strategies for regression testing optimization.
Regression testing in Agile is like a bitter pill: is unpleasant but it’s necessary to assure the quality of the product. Fortunately, Agile testing teams have a group of strategies which they can apply to optimize the regression testing. The most popular are:
- Two-level approach
- Test prioritization
- Regression testing automation
The teams usually select the approach that best fits in terms of scale, deadline, number of releases planned, etc. Still, there is a common ground in three approaches – an uninterrupted communication.