AI in research synthesis: what it does and what it costs

AI doesn’t replace reading the interview. It replaces the mechanical part of grouping what you already read. The mistake is asking for the synthesis before the data is clean. With a raw transcript, the model groups noise with the same confidence it groups a finding, and the result reads better than the material justifies.

The five stages and what is mechanical in each

Qualitative work moves through five stages. Sort them before you open a tool.

Collection is the interview, the session, the notes you took while someone talked. Almost nothing here is mechanical. The follow-up question depends on what the person just said and on the pause before it.

Cleaning is transcription, name correction, filler removal, speaker marking. This stage is mechanical, and a model handles it well.

Coding is attaching a label to a passage. The mechanical half applies a codebook you already wrote. The other half notices a passage that fits no code, and that half stays with you.

Grouping is placing coded passages side by side and looking for a pattern. The mechanical half is the sorting. The judgment half is deciding which pattern matters for the decision ahead.

Decision is choosing what the finding means for the product. This stage always stays with a person, because a decision needs someone who answers for it, and a model can’t fill that seat.

Where AI accelerates for real

Five steps get faster, and each one has a mechanism behind it.

Transcript cleanup. Punctuation, speaker labels and timestamps are rules, and rules are cheap to automate.
A first pass over a codebook. Give the model thirty excerpts and ask which labels repeat. You get a draft list to correct, not a codebook.
Retrieval inside the material. Ask where a topic appears. Finding a passage is a search, and a search is mechanical.
Format conversion. A session summary for another team, a table of themes, a translated draft of a quote you already selected.
Rigid format. A report with fixed sections and standardized quotes. Nielsen Norman Group says it uses AI in its own editorial process for this kind of finishing, and people still judge the text.

All five follow one pattern. AI shortens the distance between material you already read and an artifact that carries it. The post on AI in the design workflow applies the same split to design work as a whole.

Where it contaminates the result

The bigger risk here isn’t error. It is text that sounds too smooth.

Four failures are worth knowing, because each one looks like success.

Flattened disagreement. A model summarizes what you paste, and summary is compression toward the average. The participant who disagreed with the other seven becomes one sentence inside a paragraph about the theme. Their reason disappears first.

Invented pattern. Ask for themes and you will get themes. The request implies the pattern exists, so the model produces one. A theme list isn’t evidence that the theme is in the data.

The lost exception. The case that contradicts the average is often the finding. A system tuned to be helpful smooths the exception, because the exception looks like noise next to the majority.

Lost context. The pause, the subject the participant avoided, the sentence that came after a long silence. Those signals live in the recording, not in the summary, and they rarely survive a compression step.

Also distrust any number in the summary that has no transcript line behind it. For the same reason, this text describes mechanisms and quotes no rates.

One output is an example, not an evaluation

A synthesis run once is a sample. Nielsen Norman Group points out that generation is nondeterministic, so the same request yields better and worse versions. If your synthesis came from a single attempt, it shows the attempt, not the method. The post on evaluating AI output explains the method in more detail.

Personal data: what must not leave your hands

Anonymize before you paste, not after.

Names, employer, city and team. A role identifies a person in a team of four.
Direct quotes are the most identifying part of a transcript. A quote about a manager is a signature when the team has one manager.
Health, money, legal position, credentials, minors and anything covered by a consent form that didn’t name a third-party processor.
Recordings and raw notes. A cleaned excerpt carries less risk than the full file.

Read the consent form against the pipeline. The form says what happens with the recording, and a model service is a new recipient that the form probably doesn’t name. Check retention, training on your input, and the region where processing happens. Anthropic’s Usage Policy separates universal standards from consumer use cases with an elevated risk of harm, and a research pipeline has to be read against that split. Google’s responsible AI practices place privacy and safety in the same frame. Nielsen Norman Group documents a research team that deleted its participant spreadsheet once recruitment ended, for privacy reasons. Treat the data as having a lifetime.

How to review the output before it becomes a finding

Return to the source for every claim. If the summary says participants found the flow confusing, open the transcripts and count.
Read what the summary dropped as well as what it kept. Start with the participant who disagreed.
Run the same prompt three times and compare. The differences show where the system is unsure.
Ask for the counter-example. Ask which excerpt contradicts the theme. A model can answer that question when you ask it, and it rarely volunteers the answer.
Keep the raw material next to the output. A finding you can’t trace to a line is a guess with better grammar.
Treat the theme list as a hypothesis, never as a result.
Record who approved the final version. For Nielsen Norman Group, AI helps with the writing, but responsibility for the text belongs to whoever signs it.

What still requires the researcher

The question. A model answers what you ask, and choosing the question is the first act of interpretation.
The criterion for what counts as a finding.
The exception. Someone has to defend the case that contradicts the average in a room where the average is popular.
The reading of a person’s experience. A model has no access to the context of a life.
The signature. Accountability can’t be delegated to a service that has no name on the report.

Where the NN/g articles disagree among themselves

Two of the four articles pull in opposite directions on who owns the method. Stop Making Your Team Figure Out AI on Their Own argues that leaving each practitioner alone produces chaos and risk. Ops teams should pilot tools and back adoption with structured guidance. AI Can Help Write an Article, but It Can’t Stand Behind It says the opposite about method inside the same house: NN/g doesn’t prescribe how authors write or even think.

They also differ on what one output is worth. Supercharge UX Research by Automating Workflows and Repetitive Tasks presents automation as a way to code and analyze responses as they arrive. One AI Output Is an Example, Not an Evaluation treats a single output as a sample that can’t carry a decision, and asks for repeated runs before it counts.

The ground they share is small, but useful. Automation belongs in the repetitive part, and the judgment stays with a person.