How small can a group be before you stop reporting it separately?

By ·

Short answer: There is no single safe number. Statistical agencies use fixed thresholds: CMS, for example, does not release counts of 1 to 10 [S1][S2]. But the real test is whether someone could work out who is behind a result. Set a minimum count, check what share of the eligible group a slice represents, and watch for rare details at any size [S3][BOOK].

1. Fixed thresholds exist, and they vary

Agencies that publish tables have used minimum cell sizes for decades. The US Centers for Medicare & Medicaid Services (CMS) policy is one of the clearest: no cell containing a value from 1 to 10 may be shown, and no other figure may be shown if it lets someone calculate such a cell [S1][S2].

Other agencies use different numbers. The Federal Committee on Statistical Methodology's working paper on disclosure limitation describes threshold rules across US federal statistical agencies, and the thresholds aren't all the same [S3]. A number is a policy choice that fits a context. It isn't a law of nature.

So the first question for any study isn't "what is the number?" but "what is our number, and why?"

2. A count can be safe on its own and unsafe in combination

Thresholds are usually applied cell by cell. Re-identification usually happens across cells.

The CMS policy gives the classic example: a percentage reported alongside a total lets a reader calculate a small count, even if the count itself was never printed [S1][S2]. The fix in that policy is to collapse categories, coarsen the data, or suppress enough other cells that the small count can't be worked out [S1].

Latanya Sweeney's work on k-anonymity formalised the same idea for individual records: a released record is protected only if it can't be told apart from at least k−1 others on the combination of attributes an outsider might know [S4]. Role, site, and time period together can single someone out even when each is broad on its own.

3. The book's test: could a reasonable person guess who is behind a dot?

Designing Sensemaking Studies frames the threshold as a question rather than a number: "Minimum-N is a proxy for one question: in this context, could a reasonable person guess who is behind a dot?" [BOOK, p.184]

It then proposes two gates for any segmented view [BOOK, p.184]:

  • A count gate. Don't show a slice with fewer than k stories. The book suggests k = 10 as a default, and 15 or more when roles or sites are small, power imbalances are sharp, or participants are vulnerable.
  • A proportion gate. Even if a slice passes the count, don't release it if it covers a large share of the people who could have contributed. If the slice points to a group of fewer than about 30 people, or k would be about 10% or more of that pool, aggregate up: widen the time window, combine categories or show patterns only.

The book also notes that time is a filter: a narrower time window shrinks the pool of people who could have contributed, and raises the risk [BOOK, p.184].

4. Rare details can identify people at any size

Counts don't protect against a story that only one person could have told. As the book puts it: "Minimum-N is necessary but not sufficient: rare events and recognizable details can re-identify people even at higher counts." [BOOK, p.185]

This matches the health-data literature. Khaled El Emam and Fida Dankar showed that the risk of re-identification depends on what an outsider is likely to know and how the data are released, not only on the threshold chosen [S5]. The US Department of Health and Human Services' de-identification guidance, written for health data, likewise allows either removing a fixed list of identifiers or having an expert judge the risk in context [S6]. It doesn't treat any single number as sufficient.

For story data, that means screening open text for rare events, unusual roles, places and dates, as well as applying a count.

5. Decide the rules before you see the data

Setting thresholds after looking at results invites pressure to show the interesting small slice. Decide in advance and write it down:

  1. The eligible pool for each view: how many people could plausibly have contributed to this slice in this period?
  2. The count gate (k) and the conditions that raise it.
  3. The proportion gate: the pool size below which you aggregate up.
  4. Combinations: which attribute combinations (role × site × period) you won't show together.
  5. Derived figures: no percentages, averages or charts that let a reader calculate a suppressed count.
  6. Open text: who screens stories for rare details, and the rule for paraphrasing or withholding them.
  7. Who can see what: a view that is safe for a small internal team may not be safe for a public report.

Reviewers usually ask for exactly this list. Having it in the protocol tends to shorten the review.

Sources