How do you write a story prompt that invites experience rather than opinion?

By ·

Short answer: Ask about one specific moment that already happened, not a general view. People recall specific episodes differently from how they summarise beliefs [S3], and question wording shapes what they report [S2]. Anchor the prompt in an event, take away any "right answer", ask what happened next, and test the wording with a few real participants before launch [BOOK].

1. Opinion questions and experience questions draw on different memories

"How satisfied are you with the service?" and "Tell us about a time the service helped or got in your way" look similar. They ask people to do different mental work.

Michael Robinson and Gerald Clore reviewed the evidence on how people report feelings [S3]. When people report on a specific, recent episode, they can draw on memory of that episode. When the question is general or about a long period, they rely more on their beliefs about themselves and the situation. A general rating tells you what someone thinks overall. A specific story shows what happened and how it felt at the time.

Both are useful. When the goal is to understand why a pattern exists, you need the episode.

2. The wording of a question shapes the answer

Norbert Schwarz showed that self-reports depend on the question itself: its wording, the response options and the context it sits in [S2]. Respondents use those features to work out what the researcher wants to know, and answer that.

For prompt writers, this has two consequences:

  • Every word is a cue. "Tell us about a problem you had" tells people you want problems. "Tell us about a moment that stood out" leaves the valence to them.
  • Institutional language signals the expected answer. If the prompt uses the organisation's vocabulary, people tend to answer in the organisation's terms.

3. A story about one moment is an established research method

Asking for specific incidents isn't new. John Flanagan set out the critical incident technique in 1954: collect people's accounts of specific, observed incidents rather than general impressions, and analyse those accounts systematically [S1]. The technique is still in use across health, education and service research.

Designing Sensemaking Studies takes the same position for story prompts: "Design prompts to be retrospective: ask about a specific moment that already happened." [BOOK, p.110] And more directly: "Anchor in an event: "Tell us about a time…" beats "How satisfied are you…" when you need meaning, not ratings." [BOOK, p.112]

4. A prompt with a "good answer" gets that answer

Roger Tourangeau and Ting Yan reviewed research on sensitive questions and found that people misreport when an answer seems socially undesirable, and that the setting of the question affects how much [S4]. A prompt doesn't have to be about a sensitive topic to carry this pressure. Prompts that hint at loyalty, gratitude or competence ("Tell us how our team supported you") invite people to sound grateful.

The book's checklist covers this directly: a prompt shouldn't leave participants feeling pressure "to sound competent, loyal, or grateful"; if it does, rewrite it [BOOK, p.113].

5. Test the wording with three to five people before launch

You can't tell from your desk whether a prompt works. The book suggests a short paper test with three to five people like your participants [BOOK, p.111]:

  1. Ask them to answer the prompt out loud: which moment did they choose?
  2. Ask what each interpretation word means to them.
  3. Ask what they didn't say because of the wording.

The third question is the one that catches exclusions. If several people chose not to mention something because of how the prompt was framed, the framing is filtering your data.

6. A five-point check before you ship

Adapted from the book's quick quality check [BOOK, p.113]:

  1. Scene, not stance. Does the prompt take people into one specific moment they can replay?
  2. No "good answer" smell. Could anyone feel they need to sound competent, loyal or grateful?
  3. Consequence included. Does it ask what happened next, what changed or what was at stake?
  4. Interpretation questions are lenses, not tests. When participants interpret their own story, are both ends of each scale reasonable in context, not "good" versus "bad"?
  5. Participant language wins. Would your participants recognise the words as theirs, not policy or consultant vocabulary?

If an item fails, the book's advice is not to add more questions but to fix the invitation and the language first [BOOK, p.113].

Before and after (illustrative):

Opinion prompt Experience prompt
How satisfied were you with the appeal process? Think of a moment during your appeal that stood out to you. What happened, and what happened next?
How well did staff support you? Tell us about a time you needed help during the process. What did you do, and how did it turn out?

Sources