Why five participants is enough for a usability test

Five participants is enough for a qualitative usability test, and the number comes from a measured curve rather than a habit. Jakob Nielsen and Thomas Landauer modeled problem detection as a Poisson process in 1993. Their model puts the share of problems that a single participant reveals at about 31%. Five participants cover roughly 85% of them. The same research also says where that limit sits, and most teams skip that part.

Where the number five comes from

Jakob Nielsen and Thomas Landauer published “A mathematical model of the finding of usability problems” at the ACM INTERCHI’93 conference in Amsterdam. They treated each participant as an independent chance to run into a problem. Across 11 studies, the detection of problems fitted a Poisson process well.

The model has two terms. N is the total number of problems in the design. L is the share of those problems that one participant encounters. Their average estimate of L was 31%. With that value, one participant finds about a third of the problems, three find about two thirds, and five find about 85% (Nielsen and Landauer).

The cost side is part of the argument as well. The 2000 article puts the best benefit-to-cost ratio at three or five users, depending on the style of testing. The number five is a rounding of a curve, not a constant of nature.

Nielsen restated the rule for the web in March 2000, in “Why You Only Need to Test with 5 Users” on the Nielsen Norman Group site. Two details there matter more than the headline. The curve shows that covering the full set of problems needs around 15 participants. The recommendation is to spend that budget on three studies of five instead of one study of fifteen, because the goal is a better design and not a longer report.

The 2000 article also states the case that is easy to forget: zero users give zero insights. The first participant is worth more than the fifth, and the fifth is worth more than the twentieth.

Why the return falls fast after the first participants

Each new participant repeats part of what the earlier ones did and adds less that is new. The curve of new findings levels off. The 1993 model and the 2000 article describe the same effect. The second participant teaches less than the first, and the fifth teaches little that the first three didn’t show.

Nielsen’s 2012 follow-up, “How Many Test Users in a Usability Study?”, reports a survey he ran at his UX Conference with 217 respondents. The average practice at their companies was 11 participants per round, more than double the recommendation. The extra sessions cost money and returned few new findings.

The alternative to a bigger sample is a second round. After a redesign, a new group of five tests the fix and also exposes problems the first round missed. The 2000 article places the first round near 85% of the problems and says the second round finds most of the remaining 15%. A third round mops up the rest.

Fixed cost is part of the argument. Recruiting, scheduling and setting up a session cost roughly the same for one participant and for five. The 2012 article notes that a smaller overhead per study is what makes many small studies affordable. Spreading that fixed cost over five people lowers the price of each finding.

What five participants doesn’t solve

Four situations where five is the wrong number.

Measuring is a different story. Five participants support discovery. They don’t support a number. Nielsen’s 2012 article sets the floor for quantitative studies at 20 participants and says that tight confidence intervals need more. Card sorting needs at least 15 per user group. Eyetracking heatmaps need 39. The 2006 article on quantitative studies works the arithmetic behind that floor.

There isn’t always a single audience. The 1993 model holds for comparable users who use the product in similar ways. Nielsen recommends 3 to 4 participants from each category when testing two groups, and 3 from each category for three or more groups. The DHS Usability Testing Kit comes from the United States Department of Homeland Security. It puts the point plainly. “If you think there might be different behaviors for different types of users, include four of each type.”

Rare problems stay invisible. Jeff Sauro, of MeasuringU, worked the binomial math in 2010. Five users give an 85% chance of seeing a problem that affects 31% of users. A problem that affects 10% of users needs 18 users for the same chance. A problem that affects 1% needs 189. Sauro adds that in released software a given problem probably affects fewer than 10% of users. That is why the same five participants find fewer problems in a mature product than in a prototype.

Five is a distribution, not a guarantee. Laura Faulkner tested 60 users in 2003 and drew random sets from the whole group. Her paper, in Behavior Research Methods, Instruments and Computers volume 35, reports that some sets of five found 99% of the problems while other sets found only 55%. With 10 users, the weakest set found 80%. With 20 users, 95%.

That last finding is a fair critique, but it doesn’t kill the rule. It shows that results vary. The average still favors five over fifteen, and the worst case is an argument for a second round and not for a monster session. A different risk sits upstream of the session itself, and invalid research data covers what spoils a study before the first participant arrives.

How to build the task script

A task is a goal plus a starting point. A route through the interface isn’t a task, it is the answer key.

Write the task in the words of the person doing it, never in the words of the interface. “Find the invoice you sent last March” works. “Open the Billing tab and use the date filter” hands over the path and measures nothing.
Test the riskiest flow first. If the session is cut short, the valuable finding is already recorded.
Aim for five to seven tasks in a session. The DHS kit sets the limit at “not more than 15-20 minutes per participant” of activity, and the digital.gov script, from the United States General Services Administration, suggests a session length such as 30 minutes.
Make the ending observable. You need to know whether the task finished, and a task with no finish line can’t be scored.
Cover the state the user arrives in. A first-time visitor and a returning customer are two different tasks, even on the same screen.
Read the task out loud once and then hand it over. If you have to explain it, the task is broken and not the participant.
Run one practice session with a colleague. Recruit the real participants for the real thing.

How to conduct without leading the answer

The moderator’s job is to watch and to listen, not to teach. The DHS kit states the principle in one line: you watch and listen without teaching or training. Every hint you give spends the finding you paid for.

Say the frame first. The digital.gov script opens with the line that there are no wrong answers, and asks the participant to think aloud as much as possible.
Stay silent for ten seconds after the person stops talking. The second sentence is usually the useful one, and it only arrives if you wait for it.
Answer a question with a question. “What were you expecting to happen?” keeps the participant talking. “It is in the top right” ends the session early.
Never confirm. Words like “right”, “exactly” and “good” teach the participant what you want to hear for the rest of the session.
Don’t name the feature. The moment you say the button’s name, you have tested your vocabulary and not the interface.
Put the blame on the design. “This screen is confusing, and that is our fault” frees the person to admit a failure.
Ask for the last real occurrence. “When did you last do this for real, and how did it go?” produces memory instead of opinion.
Skip the opinion questions at the end, or save them for last. What people say they would do and what they do on screen are different data sets.

What to write down during the session

Write what happened, in the order it happened, with the words the participant used.

The action. What the person clicked, typed or scrolled, and where on the screen it was.
The outcome of the task. Completed, completed with difficulty, or failed. This single field carries most of the analysis later.
The quote. One short sentence in the participant’s own words, next to the action it explains.
The expectation. What the person said they expected to see before they acted. The gap between expectation and screen is the finding.
The severity. Does it block the task, does it cost time, or is it cosmetic. Sort on this field, not on how loudly it was said.
The counterexample. Note the participants who sailed through, or you will redesign for the loudest person in the room.
The timestamp. If the session is recorded, the time code turns a vague memory into a clip you can replay for the team.

One moderator and one note taker is the practical split. The DHS kit puts one moderator in the room and has that person take notes. A second observer catches the moments the moderator misses while talking.

How to turn a finding into a decision

Group by problem, not by participant. Three people hitting the same wall is one problem with a count of three, and that count is your frequency estimate.
Rank by severity times frequency. A blocker seen once outranks a cosmetic issue seen five times.
Decide each item in one of three ways: fix it, defer it with a written reason, or accept it as a trade-off. The written reason is what stops the same debate next quarter.
Fix the blockers in the redesign and keep the rest of the list visible. A finding that leaves the room and enters no backlog is a finding you paid for and threw away.
Test the fix with a new group of five. The 2000 article is explicit that a redesign is a claim and not a proof, and that a second round verifies the claim.
Keep the counterexamples in the write-up. They mark the parts of the product that already work and shouldn’t be touched.

The rule survives because the alternative is worse. Five participants, a real task and a moderator who keeps quiet produce more usable decisions per hour than fifteen participants answering a leading script. The number isn’t the method. The task and the script are the method.

A related argument about choosing the layer that already carries the behavior sits in the semantic HTML post. A note on testing a decision before building it sits in the checkout post.

Where Nielsen Norman Group disagrees

NN/g itself doesn’t settle on one number. The 2000 article states the flat rule, and the 2012 follow-up softens it. For low-overhead projects the 2012 article puts the optimum at two users per study, and it reports that NN/g itself often ran around eight users in consulting work. Discount Usability: 20 Years adds that the 1989 recommendation was three users, and that Nielsen moved to five as a compromise.

The quantitative figures move as well. The 2006 article treats 20 users as enough for a margin of error near 19%, while the 2021 article puts narrow confidence intervals at 40 participants or more. This post takes the qualitative rule as settled and reads both quantitative numbers as a range.