Product validation before test case sprawl
Test cases multiply when nobody is sure what serving the user means. I validate the product against user and business outcomes first, and document second.
Test case sprawl is often an anxiety response. A feature lands in the sprint, the user journey is still fuzzy and the pressure to look thorough is real, so the tester opens the test management tool and starts typing. By the end of the sprint there are dozens of new cases, every field has its boundary values, and nobody can say whether the feature does what its user needed.
I have worked with test packs in TestRail, Zephyr and HP ALM, and as Test Practice Manager I wrote the test-case guidelines for a practice. I have nothing against test cases. My objection is to writing them first. A case documents a check you have already decided is worth repeating, and that decision depends on knowing what the product is for.
What product validation means in testing
Testing textbooks separate verification from validation. Verification asks whether the product was built as specified. Validation asks whether what was built serves the person it was built for and the business that paid for it. Most test cases are verification: given this input, expect that output. They are only as good as the specification behind them, and a specification can be met in full by a feature nobody can use.
Product validation, as I use the term, means asking the second question first. Before I document anything, I want to know what success looks like for the user and for the business, and whether the build in front of me delivers it.
Four questions to ask before writing test cases
- Who is the user? A role in a context: a dispatcher at a shared desktop, a driver who has just parked and wants to walk away. “The user” with no description usually means the team has pictured itself.
- What are they trying to complete? The task or decision that brought them here, in their words. Pay and leave. Approve the request before the supplier’s deadline.
- What would make them abandon or complain? Being asked for the same detail twice, a wait with no feedback, a total they cannot reconcile, an error that does not say what to do next.
- What does the business need to know before it ships? That the money is right, that the audit trail exists, that Support can see what the customer saw.
The answers come from Product, from Support, from the design and sometimes from production data. When nobody can answer the second or the fourth, that is the first finding, and it is worth more than any case I could write that day. It usually means the acceptance criteria describe a mechanism where they should describe an outcome.
From validation to selective test coverage
With those answers I walk the journey end to end on the real build, on the surface the user will have, with data that resembles theirs. No scripts yet. I keep short notes against each step: what I tried, what happened, what I could not check and why.
The walk produces three kinds of output. Defects, where the build does not do what was agreed. Questions for Product, where it does what was agreed and the result still fails the user. And gaps, where nobody specified the behaviour at all, which become stories. A case-first approach never finds the second kind, because a case can only fail against an expectation someone wrote down.
Only then do I expand into written cases, and only where repeatability pays rent.
| Write a test case when | Keep a session note when |
|---|---|
| The check belongs to the core that is re-run every release | It verifies a one-off change that will not be revisited |
| Someone else has to run it: a colleague, a UAT participant, an auditor | Only the person who explored it will ever need it |
| The setup is too intricate to hold in memory, such as pricing rules or a permissions matrix | The steps are obvious to anyone who knows the product |
| The result is needed as evidence for compliance or sign-off | An automated check already asserts the same thing |
| The behaviour is stable | The behaviour is still changing from sprint to sprint |
On most features the left column is short. It lines up with the core journeys in a regression strategy, and that is no accident. A case is a promise to run something again, and the regression core is the list of things worth running again.
What test case sprawl costs
Every case is a maintenance commitment. The product changes, the case does not, and within a few releases part of the pack describes behaviour that no longer exists. Someone then has to choose between updating cases nobody values and running cases that mislead. Usually neither happens, and the pack turns into a number on a dashboard.
The number is the worse cost. A large pass count looks like evidence, and it hides whether the journeys that matter were exercised by a person at all. It is the manual version of the failure I describe in When more automation hides a broken user journey.
Flip the order, validate first and document second, and the pack stays small enough to maintain. The time goes into friction Product recognises as its own problem.
The objection: fewer test cases looks like less testing
I hear this from managers and from testers themselves, and it deserves a straight answer. Case counts are easy to report, and some organisations have measured QA effort that way for years.
I change the unit. Coverage is reported per journey: which were exercised, on which surfaces, with what result, and what was left out. Session notes with screenshots are evidence. So is a short case for each check that needs repeating. Where a regulator, a client or an auditor needs scripted proof of specific behaviour, those checks get full cases, and that is exactly where repeatability pays.
There is a failure on the other side too. “We explored it”, with no notes, is no better than a hundred unread cases. Validation without a record cannot be reviewed, repeated or handed over. The discipline is the same either way: write down what you learned, at the level of detail the next reader needs.
Validation does not sit on a Gantt chart as a phase before testing. It is the lens every other QA activity looks through: which cases to write, what to automate, what belongs in the regression core, what to say in the readiness note. A team that asks the four questions first ends up with fewer cases and can say what each one is for.
Questions I get asked
What is product validation in software testing?
Product validation checks whether what was built serves the user and the business outcome it was meant to deliver. Verification checks whether it was built as specified. A feature can pass every specified check and still fail validation, for example when a user cannot complete the task without help. Validation needs someone to walk the real journey, with realistic data, and judge the result.
How many test cases should a feature have?
There is no useful number. Count the checks that will be run again, run by someone else or needed as evidence, and write cases for those. One-off checks on a change are better kept as session notes against the journey. A small pack that is current and tied to named journeys gives better evidence than a large one nobody maintains.
How do you reduce test case maintenance?
Stop writing cases for checks that will not be repeated, and retire cases that describe behaviour the product no longer has. Keep full cases for the regression core, for anything another person must run and for compliance evidence. For everything else, record what was explored and what was found in short notes. Fewer cases, each with a reason to exist, are cheaper to keep true.
Filed under Product validation and acceptance criteria. Terms: Product validation.
