Proxy failure
Proxy failure is the conclusion that a behavior the team has been treating as a driver of conversion doesn't work as expected. The correlation may have come from users with the right intent tending to do the behavior on their own before converting; pushing more users into the behavior, attempt after attempt, doesn't move conversion in turn. Reaching that conclusion implicates more than the metric: the team's theory of why those users were converting, and the investments built on that theory, all come into question.
Definition
The diagnostic conclusion that a candidate behavior is a signal rather than a lever, established when interventions across diverse designs amplify the behavior but the conversion rate among performers fails to hold.
Signal versus lever
A behavior can correlate with conversion for one of two reasons. It can be a signal — users who organically performed the behavior had the intent that drove their conversion, and the behavior is a downstream marker of that intent. Or it can be a lever — the behavior itself causally contributes to the conversion, such that producing more of the behavior produces more conversion.
Observational data alone can't distinguish them. Both signals and levers show up as correlation in retrospect. The distinction surfaces under intervention: a lever holds up its conversion rate when amplified; a signal doesn't, because amplifying it brings in users who lack the underlying intent the original correlation was tracking. Proxy failure is the empirical conclusion that what was treated as a lever was a signal.
Diagnosed across interventions
The unit of diagnosis is accumulated evidence across diverse interventions, not the single test. A single intervention where lift passed but hold didn't is a single-test failure — the intervention may have been miscalibrated, applied to the wrong population, or otherwise incomplete. The charitable read on a single failure is intervention-quality variance.
What indicts the proxy is the same outcome reached across diverse interventions: lift passes consistently — the behavior amplifies under multiple intervention designs — but the conversion rate among performers consistently fails to hold. The diversity rules out intervention-strategy issues localized to one design. Once that pattern accumulates, the candidate behavior's milestone-candidate status is indicted.
Where the correlation came from
The mechanism behind proxy failure is the gap between observational and prescriptive use of correlation. Observational correlation is evidence about what already-successful users did under their own intent. Prescriptive use assumes the behavior itself produces the success rather than being a downstream signal of the intent that already produced it.
Conflating the two — treating the observational correlation as license for prescriptive engineering — is the canonical route to proxy failure for feature-usage event. The original correlation came from intent-loaded users selecting themselves into the behavior. Amplifying the behavior in users without that intent adds performers who don't convert at the original rate; the conversion rate among performers drops, and hold fails.
Distinct from correlation drift
Correlation drift and proxy failure both surface as a proxy that's no longer doing its job, but the underlying conditions are different. With correlation drift, the proxy was a usable lever under original conditions; conditions changed — intent mix shifted, or the offering surface evolved — and the correlation dissolved. With proxy failure, the proxy was never a usable lever; the original correlation was observational, and amplifying the behavior was never going to produce the conversion the correlation tracked.
The distinction has practical consequences. Drift calls for re-anchoring the proxy to the new conditions or re-running multiplier analysis under the current intent mix. Proxy failure calls for replacing the candidate behavior — the proxy itself is the problem, not the conditions it sits in.
The diagnostic blind spot
The pattern is structurally hard to self-diagnose. Each intervention failure has a local explanation available: measurement issues, attribution, seasonality, the proxy needing to be paired with another driver. The underlying theory of the proxy never gets registered as wrong because each failure absorbs into a local category. Teams operating at the metric level — without an explanatory framework for why the proxy correlates — typically lack the apparatus to read the pattern across failures. The diagnostic surfaces clearly at the model level, where the team has an explicit theory of why the correlation exists and can check whether amplifying the behavior would be expected to amplify the conversion.
Example
Twitter's "follow 30" is the canonical illustration. The folklore version: users who organically followed 30 accounts that mattered to them had assembled a feed worth returning to, which is what the correlation with retention reflected. Onboarding later preloaded 30 celebrity accounts with a select-all checkbox, apparently aimed at manufacturing the count — but not necessarily producing the feed-assembly that organic following would have signaled.
Without knowing what other interventions Twitter tried, the case can't be classified as confirmed proxy failure versus single intervention failure that further attempts might have addressed. The general risk it illustrates: optimizing a proxy without understanding what drives the correlation hits the metric while missing the underlying value.