Exploratory testing is a type of manual testing method where software is tested without a set of tests defined in advance. During an exploratory test session, you are not restricted to a script or a set of predetermined steps. When a QA engineer just goes Ad-hoc into exploratory testing without following any structure, it is very difficult to quantify what work was done, what parts of the software were tested and how much time was actually spent testing.
Session Based Testing (SBT) is a framework that supports exploratory testing. From the semi-structured interviews that I performed I came to know that QA engineers where performing a lot of exploratory testing during the sprint but where not using SBT. This was also evident from the survey performed where about 80% of all the respondents including QA engineers and the team members working with QA engineers answered “No” to the question: Do you use Session based testing?
SBT framework helps bridle the randomness around exploratory testing without compromising on the methodology involved in exploratory testing. It also overcomes major issues involved with exploratory testing like a lack of structure that provides no visibility on the progress made with exploratory testing and test coverage achieved with it (Saxena, 2012).
Basic Elements of Session Based Testing
is the goal or agenda associated with the test session.
is an un-interpreted period of time spent often time boxed to less than 2 hours.
is a record of the time boxed test session containing the charter, area covered, how testing was conducted, list of bugs, risks and issues found, documentation of data files the tester used or created during the session, percentage of time the tester spent on the session vs. investigating new issues, session start time and duration etc. the session report can contain more information if required and can be customized to the needs of the project.
is a short discussion between the Product Owner and the QA engineer about the session report.
in exploratory testing using SBT is performed with a standardized session report. This report can be parsed with a software tool like Microsoft Excel to store results as aggregated data for reporting and metrics.
with the help of SBT, QA engineers can plan test sessions to fit the needs of the project. Charters can be added or dropped based on tests executed and changing requirements. (Saxena, 2012)
Proposed Usage of Session Based Testing (SBT)
SBT is a feasible solution for performing exploratory testing. Managers in the company like to see metrics because they like to see progress in the project. Using SBT, QA engineers would be able to provide this metric on testing efforts to the managers.There are several tools in the market that can assist SBT. I have also tried to perform a proof of concept with an open source tool called “sessionweb”. But the most feasible tools for a #.Net company could be “Microsoft Test Manager” or “Bonfire” also known as “Jira-Capture” which is an add-on for “Jira”. Both the tools were already available in the company I work. “Jira” is already used as a project tracker in the company and all the QA engineers have Microsoft Visual Studio installed in their standalone machines with which “Microsoft Test Manager” is pre-installed. Seeing the benefits of structured exploratory testing with the usage of “Session Based Testing” it is proposed as a feasible solution.