Hold failure
Hold failure is the conclusion a team reaches after pushing a behavior that correlates with conversion across several different attempts, succeeding at getting more users to do it each time, but finding that conversion doesn't follow. The diagnosis is that the behavior never drove conversion in the first place. It only ever marked the users who were already going to convert.
Definition
A pattern across diverse interventions where the candidate behavior amplifies (lift passes) but the conversion rate among performers doesn't hold within its natural variation band.
What the original correlation reflected
Hold failure indicates the candidate behavior is observational rather than causal. The original correlation between the behavior and the commercial outcome reflected which users did the behavior organically — those users were going to convert anyway, and the behavior was tracking that. When the team manufactures more performers through intervention, the new performers don't carry the same conversion likelihood, so the multiplier drags back toward baseline. The behavior tracks pre-existing intent; amplifying the behavior doesn't manufacture the intent.
Accumulation across interventions
A single test where lift passes but hold doesn't is single-test failure — intervention-quality variance, not a verdict on the candidate behavior. The intervention may have been miscalibrated, applied to the wrong population, or incomplete. Hold failure surfaces only after accumulated hold failures across diverse interventions. The unit of diagnosis is the pattern across tests, not any single test.
The hold band itself is a practitioner-judgment call against the conversion-rate metric's long-term volatility — jumpier historical trends warrant wider bands, stable trends tighter. A hold "failure" inside the noise band isn't a failure of the test.
Triggering proxy failure
Hold failure as operational observation triggers proxy failure as strategic conclusion. The behavior won't accumulate value milestone status — it's removed from contention. Proxy failure reaches further than dropping the metric: the underlying intent theory, route-efficacy assumptions, and path investments tied to the proxy all warrant reexamination. The team built downstream apparatus around a behavior they treated as causal; if the behavior turns out to be observational only, that downstream apparatus is built on the wrong foundation.
Example
The Twitter "follow 30" pattern is the standing illustration. Users who organically followed 30 accounts converted to retained users at higher rates — a clean observational correlation. Twitter's onboarding later preloaded 30 celebrity accounts with a "select all" checkbox, an intervention designed to lift the share of new users hitting the threshold. Lift presumably passed; the count went up. Whether hold did is the question. The folklore reads as if it didn't — the manufactured 30-follows didn't produce the feed-assembly that organic 30-follows had signaled, so the new performers converted at lower rates than the original ones. Without the underlying test data, the case can't be classified as confirmed hold failure versus single intervention failure that further attempts might have addressed, but the structural risk is what the case illustrates: optimizing a proxy without understanding what drives the correlation hits the metric while missing what the metric was tracking.