SystemOverflow
Learn→ML-Powered Search & Ranking→Evaluation (NDCG, MRR, CTR, Dwell Time)→4 of 6
ML-Powered Search & Ranking • Evaluation (NDCG, MRR, CTR, Dwell Time)Medium⏱️ ~3 min

CTR and Dwell Time: What User Behavior Reveals About Ranking

Core Concept
CTR and Dwell Time measure real user behavior, revealing what users actually do rather than what labels say they should do.

CTR: Clicks Divided by Impressions

CTR = clicks / impressions. If 1000 users see a result and 50 click, CTR = 5%. A 5% to 5.25% lift (5% relative) is often significant. To detect this reliably, you need roughly 100,000 impressions per experiment arm with 80% statistical power.

CTR has a flaw: it measures attraction, not satisfaction. Clickbait gets high CTR but angry users. A sensationalized title might double CTR while halving retention. CTR alone cannot distinguish "users found what they wanted" from "users were tricked."

Dwell Time: Engagement After the Click

Dwell time measures how long users spend on the destination after clicking. Long dwell (2+ minutes) suggests content matched expectations. Short dwell (under 10 seconds) suggests disappointment. Combining CTR with dwell separates genuine relevance from clickbait.

Measurement is tricky: you need to track when users navigate away, requiring JavaScript or detecting when they return to search. For external links, dwell measurement is approximate.

Using CTR and Dwell Together

High CTR, high dwell: Attracted and satisfied. The goal. High CTR, low dwell: Clickbait. Low CTR, high dwell: Hidden gem. Low CTR, low dwell: Irrelevant.

💡 Key Insight: Optimizing CTR alone leads to clickbait. Optimizing dwell alone ignores discoverability. The combination reveals true quality.
💡 Key Takeaways
✓CTR = clicks / impressions. A 5% relative lift is often significant. Need ~100K impressions per arm.
✓CTR measures attraction, not satisfaction. Clickbait gets high CTR but disappoints.
✓Dwell time: long (2+ min) = satisfied, short (<10 sec) = disappointed.
✓Combine CTR and dwell to separate genuine relevance from clickbait.
✓High CTR + high dwell = goal. High CTR + low dwell = clickbait.
📌 Interview Tips
1Explain the CTR trap: optimizing alone leads to clickbait. Pair with dwell.
2Use 2x2: high/low CTR × high/low dwell = goal/clickbait/hidden gem/irrelevant.
3Sample size: ~100K impressions per arm to detect 5% relative lift.
← Back to Evaluation (NDCG, MRR, CTR, Dwell Time) Overview