From Oppenheimer to Barbie: Move from chaos to the perfect world by prioritizing test scope with RBT

One of the main entry points into the technical QA field is through exploratory testing. In this procedure, a QA professional is introduced to a product or project they've never interacted with before and asked to explore it to identify defects and report them to developers.

This method gives QA the freedom to explore the system without strictly following a predefined set of test cases. They utilize their experience, intuition, and knowledge to interact with the product or project, aiming to uncover possible defects, unexpected behaviors, and areas for improvement.

In reality, there is often confusion at this stage regarding terminology. What I've described above is not actually exploratory testing, but rather ad-hoc testing. The key distinction is that ad-hoc testing doesn't require previous experience or strategic planning. Conversely, exploratory testing demands prior experience and understanding of test strategies, such as heuristics, mind maps, checklists, and similar techniques, to effectively explore the product.

According to James Bach:

“Exploratory software testing is a powerful but widely misunderstood approach. In certain situations, it can be far more productive than scripted testing. All testers practice some form of exploratory testing unless they simply do not create tests at all. Yet, few of us have studied this approach extensively, and it's not highly respected in our field. This attitude is starting to change as companies seek increasingly agile and cost-effective methods of software development.”

I won't delve deeply into exploratory testing in this article (perhaps in a future one), but if you want a comprehensive understanding, I recommend two key resources:

• The article "Exploratory Testing Explained" by James Bach (quoted above);
• The book "Explore it! Reduce Risk and Increase Confidence with Exploratory Testing" by Elisabeth Hendrickson.

Particularly, the book provides detailed explanations on how to explore effectively, practical sessions, quality factors, essential elements, and when to stop testing.

Scope Prioritization

Although these readings deeply cover exploratory testing, there remains one topic not mentioned explicitly — scope prioritization.

After these readings, you may now understand how to conduct exploratory testing more efficiently. However, how should you prioritize the scope identified during exploratory testing? Without this prioritization, all gathered information could easily become chaotic.

With this chaos in mind, I recalled another resource I encountered early in my career, known as Risk-Based Testing (RBT).

Risk-Based Testing (RBT) is a software testing approach focused on allocating testing resources to system areas presenting the highest risk. Its primary goal is identifying, prioritizing, and addressing software risks, ensuring tests target critical and error-prone components.

Instead of uniformly applying testing across the entire system, RBT optimizes the testing process to maximize the discovery of significant defects while minimizing risks associated with software usage. This becomes particularly critical when testing resources like time and budget are limited.

You can create your test cases based on these efforts, structured as follows:

Testing efforts identified through exploratory testing are prioritized according to their impact multiplied by occurrence frequency (shown visually from red to green, highest number to lowest).

Once all testing efforts are mapped in this traceability matrix, your role as QA is ensuring coverage starts from behaviors with the highest impact-to-occurrence ratio, progressing down to the lowest and least frequent risks.

Therefore, your testing focus should consistently follow this prioritized order: 25, 20, 20, 16, 15, 15, 12, 12, 10, 10, 9, 8, 8, 6, 6, 5, 5, 4, 4, 3, 3, 2, 2, 2, 1.

But how do you identify which system part corresponds to 25, 20, 16, 15, and so forth? This is where your field experience comes in. If you're unsure or lack experience, you can rely on market benchmarks, customer-defined criticality criteria, or validate these priorities with your team.

In other words, the vast project flow observed during exploratory testing, initially appearing chaotic and lacking scope prioritization, starts transforming into an organized and structured process.

Even better, you're not limited to using RBT exclusively in test execution. RBT can also be effectively applied in prioritizing business rules, acceptance criteria, prototype designs, requirements documentation, and more.

Interestingly, as you engage more frequently in exploratory testing, prioritizing scope becomes second nature, forming an effective exploratory testing framework based on experience combined with scope prioritization.

Together, exploratory testing and risk-based testing strengthen software quality, streamline the testing process, and significantly increase the likelihood of discovering and addressing critical defects before they reach end users. These approaches not only enhance defect detection but also foster a proactive quality mindset, enabling closer collaboration between development and testing teams to mitigate risks from the earliest stages of the software lifecycle.

The combination of exploratory testing and risk-based testing plays a crucial role in ensuring the quality of products and projects. Exploratory testing allows testers to interact with software more intuitively and flexibly, grounded in testing techniques. Meanwhile, risk-based testing directs testing efforts towards areas with the highest impact and risk.

Ultimately, this strategic combination contributes to delivering a more reliable, functional, and secure product or project.