What is Business Metrics Correlation in ML Systems?

Definition
Business metrics correlation is the practice of linking ML model performance metrics to business outcomes, enabling you to understand how model improvements translate to revenue, engagement, or efficiency gains.
THE DISCONNECT PROBLEM
ML teams optimize for model metrics: accuracy, AUC, NDCG. Business teams care about revenue, conversion rate, customer retention. Without correlation, a model improvement (AUC +2%) might have no business impact—or even hurt business metrics due to unintended side effects.
This disconnect creates two problems. First, you cannot prioritize model work effectively. Should you improve click prediction accuracy or reduce latency? Without knowing which impacts revenue more, you are guessing. Second, you cannot justify ML investment. If you cannot show that a 5% AUC improvement generated $500K in additional revenue, ML becomes a cost center rather than a profit driver.
THE METRIC CHAIN
Model metrics → Proxy metrics → Business metrics form a causal chain. A recommendation model improves its NDCG score. This leads to better click-through rate (proxy metric). Higher CTR leads to more purchases (business metric). Each link in this chain has a correlation coefficient that determines how much improvement propagates.
The challenge: correlations are not constant. A 5% NDCG improvement might yield 3% CTR improvement in one context and 0.5% in another. Context includes user segment, product category, time of year, and competitive environment.
WHAT MAKES THIS HARD
Confounders: Multiple factors affect business metrics simultaneously. Did revenue increase because of the model or because of a marketing campaign? Isolating model impact requires careful experimental design.
Lag: Business metrics may lag model changes by days or weeks. A better recommendation model affects today clicks but next month purchases.
💡 Key Insight: Correlation is not causation. Strong correlation between model metrics and business metrics does not mean improving the model will improve business—you need controlled experiments to establish causation.

💡 Key Takeaways

✓ML teams optimize model metrics (AUC, NDCG); business cares about revenue, conversion, retention—correlation bridges this gap

✓Metric chain: model metrics → proxy metrics → business metrics, each link has context-dependent correlation

✓Challenges: confounders obscure true impact, business metrics may lag model changes by days or weeks

📌 Interview Tips

1Interview Tip: Explain the metric chain concept—how NDCG improvement flows through CTR to revenue.

2Interview Tip: Discuss why correlation is insufficient—you need controlled experiments to establish causation.

← Back to Business Metrics Correlation Overview