Project Management, Software QA

Session Based Testing: A feasible solution for performing exploratory testing

Exploratory testing is a type of manual testing method where software is tested without a set of tests defined in advance. During an exploratory test session, you are not restricted to a script or a set of predetermined steps. When a QA engineer just goes Ad-hoc into exploratory testing without following any structure, it is very difficult to quantify what work was done, what parts of the software were tested and how much time was actually spent testing.


Session Based Testing (SBT) is a framework that supports exploratory testing. From the semi-structured interviews that I performed I came to know that QA engineers where performing a lot of exploratory testing during the sprint but where not using SBT. This was also evident from the survey performed where about 80% of all the respondents including QA engineers and the team members working with QA engineers answered “No” to the question: Do you use Session based testing?
SBT framework helps bridle the randomness around exploratory testing without compromising on the methodology involved in exploratory testing. It also overcomes major issues involved with exploratory testing like a lack of structure that provides no visibility on the progress made with exploratory testing and test coverage achieved with it (Saxena, 2012).

Basic Elements of Session Based Testing


is the goal or agenda associated with the test session.


is an un-interpreted period of time spent often time boxed to less than 2 hours.

Session Report

is a record of the time boxed test session containing the charter, area covered, how testing was conducted, list of bugs, risks and issues found, documentation of data files the tester used or created during the session, percentage of time the tester spent on the session vs. investigating new issues, session start time and duration etc. the session report can contain more information if required and can be customized to the needs of the project.


is a short discussion between the Product Owner and the QA engineer about the session report.

Parsing Results

in exploratory testing using SBT is performed with a standardized session report. This report can be parsed with a software tool like Microsoft Excel to store results as aggregated data for reporting and metrics.


with the help of SBT, QA engineers can plan test sessions to fit the needs of the project. Charters can be added or dropped based on tests executed and changing requirements. (Saxena, 2012)

Representation of Session Report
Representation of Session Report

Proposed Usage of Session Based Testing (SBT)

SBT is a feasible solution for performing exploratory testing. Managers in the company like to see metrics because they like to see progress in the project. Using SBT, QA engineers would be able to provide this metric on testing efforts to the managers.There are several tools in the market that can assist SBT. I have also tried to perform a proof of concept with an open source tool called “sessionweb”. But the most feasible tools for a #.Net company could be “Microsoft Test Manager” or “Bonfire” also known as “Jira-Capture” which is an add-on for “Jira”. Both the tools were already available in the company I work. “Jira” is already used as a project tracker in the company and all the QA engineers have Microsoft Visual Studio installed in their standalone machines with which “Microsoft Test Manager” is pre-installed. Seeing the benefits of structured exploratory testing with the usage of “Session Based Testing” it is proposed as a feasible solution.

Project Management

From Traditional QA Engineer to Agile test consultant

Quality Assurance in Traditional Software Development

Quality is an inherent element in agile development and it is commonly told that agile professionals are quality infected or test infected (Ambler, 2005). In traditional development processes, QA had never worked closely with software developers and was also not involved in the project in the early phases. They often played the so-called “gatekeepers” role telling project managers “Sorry, the requirements are frozen; we can add the feature in the next release.” (Lisa Crispin, 2009)

QA never controlled how the source code was written nor checked if the developers tested their code, neither collaborated with them in testing the code. QA carried out testing only post development and had the power to stop software releases from going forward if problems were identified. They were also involved in a lot of documentation work like creating test plans, bug tracking and reporting, change request, etc.

Traditional development cycles were long and the development teams focused on making sure all the specified requirements were delivered in the final product by the release date. In my experience, most of the time release dates were not met and were postponed to a future date. The development team was usually made of experts, and most of the time the team had no idea on what features were going to go live in the next release.

It was the QA’s job to read the requirements and create their test plans for the feature which was going to be released. Often the test plans were created from requirements parallel to the development of the product. QA engineers often had to wait for work as testing normally took place at the end of the project (Lisa Crispin, 2009).

Transition of Quality Assurance into Agile Software Development

The transition to short iterations of agile development produced an initial shock wave. This was also felt in the quality assurance department. Most of the QA engineers now also called as Agile Testers/Consultant whatever…. had questions in their mind and some of them were: how can I define requirements and then test and deliver the production department tested code in 2 weeks? How can I test parallel to development and that too within 2 weeks?

QA engineers also had to learn that the quality criteria would be defined by the customers or the Product Owners. They also had to be prepared for the challenge of testing a product based on acceptance criteria without any test plans. QA engineers were now part of the development team where the role of QA transformed from merely a tester or a bug finder in traditional software development into a consultant’s role who had to educate the development team in how they could improve the quality standards of the product with combined effort. QA engineers also had to learn how to code automatic tests as it became a necessity to maintain speed and make timely software releases. QA engineers took part in all Scrum meetings and perform manual acceptance testing tasks when coding was completed.

Initiation of QA activities in Traditional Development & Agile Development

Challenges in Agile Quality Assurance

Agile being iterative and incremental meant that the QA engineers had to test each incremented piece of code as soon as it was completed. A Scrum iteration is about 2 weeks long where the whole team collaborates to build and test a little piece of code or feature and makes sure it works. But what I found unusual from my interaction with agile consultants and developers was the level of “done” did not apply to the testing tasks. Normally developers should never get ahead of the testers/QA, because a story or feature is not “done” until it has been tested. Thus the collaboration between developers and testers was weak when it came to manual testing tasks; testers were solely responsible for manually testing tasks at the end of the user story. QA engineers were also held responsible for final acceptance tests on a shippable story on the staging environment before it was released live.

Although being agile has a lot of key benefits, adapting to this new process can be a challenging task. More involvement of customers and stakeholders in agile development has introduced a unique approach to meet requirements and specifications. Being agile also means that this methodology is flexible to change and more effective for development. Yet, agile methodologies have been under critic and are facing challenges to get an overall acceptance over traditional software development.In the next post i will try to point out some challenges QA engineers/ Agile Tester  are facing in agile development based on inputs from  semi- structured interviews, personal work experience and a survey.