Automated Regression Testing Challenges in Agile Environment

Abstract
Recently, when I wanted to start my new Automated Testing Project with
four resources, I thought of applying any one of the Agile
methodologies. But I was not able to continue because, a series of
questions were raised inside my mind. The questions are like “Is it
possible to use Agile methodologies in Automated Testing?”, “Can I use
traditional tools”, “Should I have to go for open-source tools”, “What
are the challenges I have to face if I am implementing automation in
Agile Environment”. In this article let us analyze some of challenges
we face while implementing Automation with Agile methodologies.
Automated testing in the Agile environment stands a risk of becoming
chaotic, unstructured and uncontrolled.

Agile Projects present their own challenges to the Automation team;
Unclear project scope, Multiple iterations, Minimal documentation,
early and frequent Automation needs and active stakeholder involvement
all demand lot of challenges from the Automation Team. Some of these challenges are:

Challenge 1: Requirement Phase
Test Automation developer captures requirements in the form of “user
stories”, which are brief descriptions of customer-relevant
functionality.

Each requirement has to be prioritized as follows:

High: These are mission critical requirements that absolutely have to be done in the first release
Medium: These are requirements that are important but can be worked around until implemented.

Low: These are requirements that are nice-to-have but not critical to the operation of the software.

Once priories are established, the release “iterations” are planned.
Normally, each Agile release iteration takes between 1 to 3 months to
deliver. Customers/software folks take liberty to make too many changes
to the requirements. Sometimes, these changes are so volatile that the
iterations are bumped off. These changes are greater challenges in
implementing Agile Automation testing process.

Challenge 2: Selecting the Right Tools

Traditional, test-last tools with record-and-playback-features force
teams to wait until after the software is done. More over, traditional
test automation tools don’t work for an Agile context because they
solve traditional problems, and those are different from the challenges
facing Agile Automation teams. Automation in the early stages of an
agile project is usually very tough, but as the system grows and
evolves, some aspects settle and it becomes appropriate to deploy
automation. So the choice of testing tools becomes critical for reaping
the efficiency and quality benefits of agile.

Challenge 3: Script Development Phase

The Automation testers, developers, business analysts and project
stakeholders all contribute to kick-off meetings where “user-stories”
are selected to next sprint. Once the “user-stories” are selected for
the sprint, they are used as the basis for a set of tests.

As functionality grows with each iteration, regression testing must
be performed to ensure that existing functionality has not been
impacted by the introduction of new functionality in each iteration
cycle. The scale of the regression testing grows with each sprint and
ensures that this remains a manageable task the test team use the test
automation for the regression suite.

Challenge 4: Resource Management

The Agile approach requires a mixture of testing skills, that is,
test resource will be required to define unclear scenarios and test
cases, conduct manual testing alongside developers, write automated
regression tests and execute the automated regression packages. As the
project progresses, specialist skills will also be required to cover
further test areas that might include integration and performance
testing. There should be an appropriate mix of domain specialist who
plan and gather requirements. The challenging part in the Resource
management is to find out test resources with multiple skills and

allocate them.

Challenge 5: Communication

Good communication must exist among Automation testing team,
developers, business analysts and stake holders. There must be highly
collaborative interaction between client and the delivery teams. More
client involvement implies more suggestions or changes from the client.
It implies more bandwidth for communication. The key challenge is that
the process should be able to capture and effectively implement all the
changes and data integrity needs to be retained. In traditional
testing, developers and testers are like oil and water, but in agile
environment, the challenging task is that they both must work together
to achieve the target.

Challenge 6: Daily Scrum Meeting

Daily Scrum Meeting is one of the key activities in Agile Process.
Teams do meet for 15 minutes stand up sessions. What is the
effectiveness of these meetings? How far these meetings help Automation
practice Developers?

Challenge 7: Release Phase

The aim of Agile project is to deliver a basic working product as
quickly as possible and then to go through a process of continual
improvement. This means that there is no single release phase for a
product. The challenging part lies in integration testing and
acceptance testing of the product.

If we can meet these challenges in a well optimized manner, then
Automated Regression Testing in Agile environment is an excellent
opportunity for QA to take leadership of the agile processes. It is
better placed to bridge the gap between users and developers,
understand both what is required, how it can be achieved and how it can
be assured prior to deployment. Automation practice should have a
vested interest in both the how and the result, as well as continuing
to assure that the whole evolving system meets business objectives and
is fit for purpose.