When Not to Use Surveys in Complex Human Situations

Surveys are useful when you already know the main questions, the response categories are stable, and the work calls for comparison, trend tracking, or simple reporting.

They are a poor fit when the real problem is still forming, people are using the same words to mean different things, or the experience only makes sense in context.

Active Sensemaking is not anti-survey. It is the better first move when you need to learn from lived experience before you freeze the problem into a rating scale.

Use surveys when

  • you already know the core dimensions you need to measure,
  • the response options are clear and shared,
  • you mainly need trend, score, or benchmark outputs,
  • the work can tolerate abstracted responses instead of story context,
  • or the decision path is already stable enough that nuance is secondary.

Do not lead with surveys when

  • you are still discovering what matters,
  • stakeholder perspectives conflict and you do not want to erase the contradiction,
  • the same event may be experienced very differently by different groups,
  • trust, legitimacy, care, or dignity are part of the issue,
  • you need to hear what people would never think to rate on a scale,
  • or you need stories plus interpretation, not stories or scores alone.

What goes wrong when you use surveys too early

1. You only hear back on the questions you already thought to ask

That sounds efficient, but it means the study cannot reveal what the team failed to imagine in advance.

2. People give opinions without enough context

A score can tell you that frustration exists. It usually cannot tell you what kind of frustration it was, what it was attached to, or why two people who both chose 3/10 meant completely different things.

3. Contradiction gets flattened too soon

Complex human situations often contain real disagreement. Surveys tend to collapse that disagreement into an average before anyone has worked out what the disagreement is about.

What to do instead

Start with a bounded story invitation and self-signification.

That means:

  • invite a short lived moment, not a rating,
  • let the storyteller interpret the story through signifiers,
  • review patterns as hypotheses rather than verdicts,
  • and return to the stories whenever a pattern starts to matter.

This is one reason Active Sensemaking can hold more nuance at scale without pretending every story means the same thing.

A real practice clue

In one public-service setting, mailed surveys were returning almost nothing. A story-first invitation surfaced not only the expected frustrations, but also issues the team did not know to ask about in advance. That changed both the service understanding and the improvement work that followed.

The key point is not that stories are magically better. The key point is that they reveal categories before you lock the categories too early.

A practical rule

If your first question is still What is really happening here?, do not start with a survey.

If your first question is How much of this known thing is happening, and is it changing?, a survey may be the right tool.

Better next steps