Scale verification
Scale verification is the process of finding out whether a behavior that correlates with conversion actually causes it, by guiding more users toward the behavior and watching what happens. Each intervention asks two questions: did the intervention actually get more users to perform the behavior (lift), and did those users still convert at the elevated rate the original correlation suggested (hold). Both passing repeatedly across different interventions is what turns a promising correlation into something the team can trust as a lever.
Definition
The operational discipline by which a candidate behavior accumulates value milestone status — through repeated lift-and-hold testing across interventions designed to amplify the behavior.
The two checks
When an intervention designed to amplify a candidate behavior runs, the team re-runs multiplier analysis on the post-intervention signup cohort and reads two things:
- Lift: the share of the cohort performing the candidate behavior increased — observable as a pre/post change in percent-of-signups.
- Hold: the conversion rate among performers stayed within the metric's natural variation band — observable as the multiplier's stability between pre- and post-intervention cohorts.
Both passing reinforces the candidate behavior's status. The hold band is itself a practitioner-judgment call calibrated against the conversion-rate metric's long-term volatility: jumpier historical trends warrant wider bands, stable trends tighter ones.
What hold tests
Lift alone can come from interventions that move the rate without moving outcomes — friction reduction or gating relaxation can produce more performers without producing more converters. Hold tests whether the original correlation survives the population change the intervention created. When the multiplier collapses under amplification, the original correlation was diagnostic of who the user already was; the behavior signaled intent that existed independently, and amplifying it in users without that underlying intent produces performers without producing converters.
Diagnosis accumulates across interventions
The unit of diagnosis is accumulated evidence across diverse interventions, not the single test. On any individual intervention:
- Both checks pass: one tally toward milestone qualification. Not milestone status yet.
- Lift fails or hold fails: single-test failure. The charitable read is intervention-quality variance — the design may have been incomplete, miscalibrated, or applied to the wrong population. The candidate behavior's status is unchanged; what gets reconsidered is the intervention design.
Patterns across multiple interventions produce the diagnostic conclusions:
- Accumulated lift failures across diverse interventions: intervention-strategy issue. The proxy is unindicted; what's failing is the team's ability to design interventions that serve the underlying intent.
- Accumulated hold failures with lift passing: proxy failure. The candidate behavior was signaling organic intent without functioning as a lever — amplifying it produces performers without producing the user state that drove the original correlation.
Example
A team identifies "completed the project setup wizard within 24 hours of signup" as a candidate behavior with a 3.2x multiplier on first paid conversion. They run an intervention surfacing the wizard more prominently in the post-signup empty state. Post-intervention multiplier analysis shows wizard completion rising from 22% to 38% of the signup cohort (lift), with the conversion rate among completers holding at ~3x baseline (hold). One pass.
Over the next six months, the team runs three more interventions — a contextual prompt mid-session, a guided-tour variant, an activation email nudge — each producing lift, each holding within band. The candidate behavior accumulates milestone status across the four passes.