A Quality Maturity Matrix leadership actually uses
Most QA maturity models become wallpaper. How I build a small matrix with Engineering and Product so each cell points at a planning decision and nobody chases a score.
Most maturity models become wallpaper. Someone runs an assessment with dozens of rows and five levels, a radar chart goes into a steering pack, the programme gets a score to one decimal place, and nothing is planned differently the following week. Leadership rarely ignores quality on purpose. They ignore dashboards that measure activity when they need to see where the risk sits.
I introduced a Quality Maturity Matrix as Test Practice Manager on a global facilities programme. A matrix like that earns attention only when each cell answers a decision: where are we fragile for users, and what would it cost to fix that before the next release?
What a quality maturity matrix is
A quality maturity matrix is a grid. Each row is a dimension of how the team produces quality. Each column is a level, described as behaviour you could observe in the backlog, the latest readiness note or a UAT session. Each row carries a marker for where the team is today, with the evidence for putting it there.
It is a planning tool. A formal maturity assessment asks how an organisation compares with a reference model. The matrix asks something narrower that a Head of Engineering can act on: which part of how we work is most likely to hurt users next, and is it worth a sprint to change? It certifies nothing.
Which dimensions and levels to include
I keep it small: five to seven dimensions, chosen from what the teams already argue about. Requirements clarity, defect hygiene, release evidence, environment stability and stakeholder alignment are nearly always there, with mobile or integration coverage where the product has those surfaces. A dimension that has never caused an argument or an incident does not need a row.
Each dimension gets three levels. Ten feels rigorous and behaves like noise: nobody can tell a four from a five, and the session becomes a negotiation over numbers. Three levels force a plain statement: this hurts us, this works with effort, this is dependable. I colour them red, amber and green, the vocabulary planning meetings already use. An illustrative matrix for a web and mobile product might read like this.
| Dimension | Fragile (red) | Workable (amber) | Dependable (green) |
|---|---|---|---|
| Requirements clarity | Stories reach the sprint without acceptance criteria | Criteria exist, written late and mostly by QA | Product agrees criteria in user language before work starts |
| Defect hygiene | Backlog full of duplicates and tickets nobody trusts | Swept before big releases, unowned in between | Every open defect has an owner and a next action |
| Release evidence | The ship call rests on a pass rate or a feeling | A readiness note exists; deferrals have no owner | Note organised by changed journeys; each deferral named, owned, dated |
| Environment stability | Nobody can say how test differs from production | Differences known, not written down; outages cost days | Differences listed in the readiness note; data refreshed routinely |
| Stakeholder alignment | Business owners meet the release in production | UAT happens late, as a demo | Business owners accept named journeys against agreed criteria |
Two rules keep the cells honest. Write behaviour somebody could check, never an adjective: “good”, “mature” and “consistent” tell nobody anything. And describe the way of working, never a person or a team’s effort.
Most of the right-hand cells are practices with their own articles here: acceptance criteria Product can sign, a defect backlog people trust, readiness notes people can read and UAT that ends in a decision. The matrix teaches none of them. It shows which one to work on first.
How to build the matrix with Engineering and Product
A matrix written by QA alone is a QA opinion with borders. I co-write it, in this order.
- Draft from artefacts. I read the backlog, the last few readiness packs and my notes from UAT, and place a first marker on each row with the evidence beside it.
- Walk one row at a time. A working session with the Engineering and Product leads. They can move any marker with evidence, and rewrite any cell that does not describe their world.
- Settle splits downwards. If the room is divided between two levels, the marker goes on the lower one and we note what evidence would move it.
- Write the consequence. For each row below green: what users or the business feel, and roughly what moving up a level would cost.
- Choose one or two rows. A matrix that asks for everything gets nothing.
Disagreement in the second step is the point. If everyone nods at the first draft, the matrix is probably lying, or worded so vaguely that nobody can object. The useful arguments are specific: Product believes criteria are agreed up front, and the recent stories in the backlog say otherwise.
How leadership uses the matrix in planning
On a global delivery programme I supported, the matrix stopped being a QA poster and became a quarterly planning input. Teams could point to a cell, say “release evidence is amber”, and ask for a sprint of investment without a blame spiral. That works because the cell describes a way of working everyone helped to word, the consequence is stated in terms of users, and the ask is small enough to grant.
- Requirements clarity: AMBER. 7 of the last 20 stories had criteria added after development started.
- Defect hygiene: GREEN. No open defect without an owner.
- Release evidence: AMBER. The last three readiness notes list 9 deferrals, none with an owner.
- Environment stability: RED. 6 working days of testing lost to staging outages this quarter. Payment provider stubbed, differences not written down.
- Stakeholder alignment: AMBER. UAT ran in release week for both releases.
- Ask for next quarter: one sprint of platform time on staging, and a written list of differences from production. If not funded: payment changes keep shipping on evidence from a stub.
What goes wrong with QA maturity models
Score theatre. The moment the rows are averaged into one number, the matrix is about the number. A single score hides the one fragile row among four comfortable ones, and it invites a target. I never total it.
League tables. Comparing teams by their markers teaches every team to mark itself generously. I compare a team only with itself, over time.
“It is subjective.” It is a structured judgement, and I say so. The evidence beside each marker makes it arguable, and arguable is enough to plan with. Metrics belong underneath, as evidence for a marker.
I refresh the matrix lightly each quarter, because maturity is a snapshot and a green row can slide. Used this way it changes how a quality problem reaches leadership. It stops arriving as a complaint from QA after an incident and becomes a row that Engineering and Product worded themselves, with a cost beside it, at the moment next quarter’s capacity is being decided.
Questions I get asked
What is a quality maturity matrix?
A quality maturity matrix is a small grid that describes how a team produces quality. Rows are dimensions such as requirements clarity, defect hygiene and release evidence. Columns are levels written as observable behaviour. Each row is marked with where the team is today and the evidence for that marker. Its purpose is to help Engineering and Product decide where to invest next, so it works as a planning input and should never be read as a score.
How many levels should a QA maturity model have?
Three is enough for planning. With five or ten levels people cannot reliably tell neighbouring levels apart, so the discussion becomes a negotiation about the number. Three levels force a plain statement about each dimension: it is hurting users, it works with effort, or it is dependable. If a row needs more nuance, put it in the evidence line beside the marker.
Is a quality maturity matrix the same as a formal test maturity assessment?
No. A formal assessment measures an organisation against a published reference model, often for benchmarking or certification. The matrix I describe is local: the team chooses the dimensions, words the levels with Engineering and Product, and uses the result to pick one or two things to improve next quarter. It makes no claim to compare one company with another.
Filed under Test practice and leadership. Terms: Quality Maturity Matrix.
