Evidence theatre
Evidence theatre is fact-finding work that goes through the motions without producing the proof it appears to. The team runs the split test or the user interviews, gets a result or a summary, and treats that output as having settled the question, when the work as done never actually settled it: the test caught a bump once but nobody checked whether it held, or the research talked to ten people and called it signal.
Definition
Evidence theatre is evidence-gathering work that performs the form of rigor without the substance: output is produced and treated as evidence, even though the work behind it couldn't support that treatment.
Performing the form of rigor
A team running evidence theatre deploys the tools — it runs split tests, conducts user research, builds dashboards — and produces the expected output: test results, interview summaries, charts. That output then gets treated as evidence for a decision. The defect sits upstream of the output. The work wasn't done in a way that produces the evidence the output is taken to provide, so what reaches the decision is the form of evidence-gathering with the substance hollowed out.
The distinction is between having output and having evidence. A chart exists; whether it warrants the conclusion drawn from it depends on rigor the chart can't display. Evidence theatre is the gap between the two — output present, the rigor that would turn it into evidence absent.
Qualitative and quantitative alike
The pattern spans the full range of evidence-gathering a team performs, not split testing in particular. Common forms:
- Ten-user research — the bar for "we did user research." It clears the request and produces interview summaries, but doesn't generate reliable signal at any rigorous standard.
- Anecdotal compliments as success signal — "a few users said they liked the redesign, so it's working," treated as a result.
- Decorative dashboards — built and maintained, but never consulted when a strategic decision is actually made.
- Confirmatory data reviews — reviews that locate support for a decision already made rather than testing it.
What unites them is form-of-rigor without substance-of-rigor, across both the qualitative and quantitative work the team nominally does.
What produces the pattern
Three structural conditions tend to generate evidence theatre:
- Rigor capacity sits unevenly in the org — often concentrated in a thin analyst function near the executive, away from the people shaping the offering.
- Offering-shapers operate under a low rigor floor that sets what counts as "data-driven."
- Nothing structurally checks whether that floor distinguishes signal from noise.
Under these conditions, output gets treated as evidence because it carries the shape of evidence — including selectively gathered or selectively weighted qualitative signal — even when the rigor underneath couldn't support that treatment.
Sibling to growth theatre
Growth theatre names the same defect at the system level: visible motion in growth-oriented shape without traction against actual opportunities. Evidence theatre names what happens one layer down, where rigor-work performs the form of evidence-gathering without producing the substance. Growth theatre is the organization shipping and experimenting with no connection to commercial outcomes; evidence theatre is the rigor inside that activity coming up empty while looking complete.
Example
A team runs an onboarding A/B test to lift signup-to-first-paid conversion. The variant shows a conversion lift, the team declares the value milestone validated, ships the change, and moves on. No one checks whether conversion among the lifted cohort holds within the metric's normal variation over the following weeks.
The test ran and returned a green number, so the work looks done. But scale verification treats accumulated lift and hold across diverse interventions as what counts; a single lift event with no hold check isn't enough to show the behavior drives conversion rather than correlating with it. Shipping isn't done because a test had a green number once — the activity ran, and the thing the activity was for didn't happen.