Expert review without users: heuristics and walkthroughs

You can evaluate an interface without users when the question is whether it breaks a known guideline, or whether a first-time user can learn a flow. To find out what people actually understand, only a test with users will answer. The three most common inspection methods answer different questions, and picking the wrong one produces a long list of findings nobody knows how to rank.

Inspection isn’t testing

In an inspection, the person looking at the screen is someone on the team, holding a yardstick. In a test, the person looking is someone who will use the product, holding nothing. That is what decides what each method can find.

The NN/g guide to running a heuristic evaluation is blunt about the limit: the method stretches a small research budget and doesn’t replace research with users. The same guide suggests a use that often gets overlooked. Running an inspection before a usability test shows where the session tasks should focus.

In other words, expert review works as triage. It clears the obvious problems, which would waste a session with a participant, and leaves the test free for what only a real person reveals.

Heuristic evaluation: the interface against a yardstick

The most common yardstick is Jakob Nielsen’s ten usability heuristics, but the method works with any set of guidelines. A niche product can add guidelines from its own domain to the ten.

Four decisions in how you run it change the result:

Number of people: a single evaluator misses problems, however experienced. The recommendation is three to five evaluators on the same interface.
Independence: each person evaluates alone and sees the others’ notes only after finishing. If the spreadsheet is shared, it stays closed until then.
Scope: one task, one section, one user group or one device type at a time. The narrower it is, the more detailed the notes.
Two passes: the first is for learning the system, with no judging. The second looks for violations, within a limit of one to two hours.

Then the group consolidates. That is where the team discusses where it agrees, where it disagrees, and which problem needs data before it becomes a fix.

Breaking a heuristic doesn’t confirm a defect

This is where heuristic evaluation misleads beginners most. A violation is a candidate problem, not a confirmed one. The NN/g guide shows a menu hidden behind an icon that breaks the recognition rather than recall heuristic and can still be the right tradeoff on a small screen.

The working rule comes from a Nielsen line quoted in the same guide: “you should not bet that your design is one of the few exceptions”. If you break a guideline on purpose, you need user research to back the exception, not an argument in a meeting.

Cognitive walkthrough: one task through a newcomer’s eyes

The cognitive walkthrough has a different target. It doesn’t judge the whole interface against guidelines. It asks whether a new user can learn to complete one specific task. The NN/g article on cognitive walkthroughs traces it to 1990, when Clayton Lewis and colleagues built it to evaluate ATMs and kiosks, where nobody gets training before use.

The session is a workshop with set roles. A facilitator performs the task and stops at each screen. A recorder writes down the answers. Everyone evaluates, and the group can include product, engineering and domain experts. At each step, the group answers four questions:

Will the user try to reach the right result at this step?
Will they notice the correct action is available?
Will they connect that action with the result they want?
After acting, will they see progress toward the goal?

A single no fails the whole step. At the end, the group gathers the failure points and decides what to redesign.

The same article says where the method is overkill. A flow that follows a familiar pattern, such as a standard store checkout, doesn’t need this level of scrutiny, because nearly everyone arrives with the mental model in place. The walkthrough pays off in flows that are new, complex, or unlike anything the user has seen.

Which one to use

The choice follows from the question the team needs answered:

	Heuristic evaluation	Cognitive walkthrough
Point of view	the evaluator	the new user
What it judges	general usability	learning the task
Reach	whole product or a section	tasks chosen in advance
Best use	catching guideline violations early	flows with no familiar pattern

Both can run in the same cycle, and NN/g admits some findings will overlap. When there is time for only one, the rule is simple. A product built on market patterns calls for heuristics, and an internal tool with a novel flow calls for a walkthrough.

Severity sets the queue, and one rater isn’t enough

A list of fifty unranked findings helps nobody. In 1994 Jakob Nielsen proposed a severity scale from 0 to 4, running from “not a usability problem at all” to a catastrophe that blocks release. The rating combines how often the problem occurs, its impact and its persistence, plus the effect it may have on the market.

How you apply the scale matters more than the scale itself. During the session, evaluators are busy hunting for problems and rate poorly. So severity is rated afterward, on the consolidated list, with enough description for people who didn’t spot that finding. Each evaluator rates without seeing the others’ ratings.

Nielsen writes that ratings from a single evaluator are too unreliable to trust. The mean of three is good enough for most practical decisions.

A rubric replaces opinion with criteria

When a review has to persuade skeptics, a score per item works better than a list of impressions. NN/g built a scoring rubric for error messages with twelve guidelines across three dimensions: visibility, communication and efficiency. Each guideline gets a score from 1 to 4, the average becomes a grade from A to D, and a chart per dimension shows where the message loses points.

The rubric is worth it because it leads to better decisions. Weighing many factors at once leads to worse decisions, and a rubric forces you to judge one at a time, against a written standard. The instructions repeat the logic of heuristic evaluation: the team reviews the criteria together, each person scores in private, and the scores are combined at the end.

The article also says who the tool is for. In organizations with low UX maturity, whoever points out a problem often hears that it is just personal opinion. A score tied to published criteria moves the conversation off that ground. The content of the error guidelines themselves is covered in field errors in forms, and the rubric technique works for any set of guidelines a team wants to measure.

A heuristic isn’t a principle or a pattern

Teams mix these terms and then argue without noticing they mean different things. The NN/g piece on types of design guidance separates four layers. A principle is a value statement about the product and varies by company. A heuristic is a general usability rule, applies to any interface, and is used to evaluate. A pattern is a reusable solution to a recurring problem and lives in the design system. A team charter describes how the team works and decides.

In a review, this has a practical effect. Checking a screen against brand principles is a different activity, useful, but no substitute for a usability inspection. And flagging a component that departs from the design system is a consistency finding, which only becomes a usability problem if it breaks a heuristic.

What only a user test can answer

An inspection measures what the team can predict. It doesn’t show the words people use to look for a feature, the order in which they actually read the screen, or the shortcut nobody on the team imagined. Context of use is also left out, and the heuristic evaluation guide reminds readers that user experience depends heavily on context.

That is why the sequence that makes sense is inspection first and testing after. The inspection clears the obvious, and the test with a few participants keeps the questions no yardstick can answer.

Where the sources disagree

The articles agree that inspection finds real problems without recruiting anyone. They disagree on what to do with the evaluator’s judgment. The heuristic evaluation guide bets on it, saying that with practice evaluators rely less and less on the list of heuristics. The error rubric goes the other way and swaps instinct for written criteria, precisely because free judgment is what the rest of the company challenges.

They also differ on who takes part. Heuristic evaluation calls for evaluators who know usability. The cognitive walkthrough brings product, engineering and domain experts to the same table. Choosing between them also decides who on the team leaves the session understanding the problem.