Learn→ML-Powered Search & Ranking→Relevance Feedback (Click Models, Position Bias)→2 of 6

ML-Powered Search & Ranking • Relevance Feedback (Click Models, Position Bias)Medium⏱️ ~2 min

How Do Click Models Separate Examination from Attractiveness?

Core Concept
Click models decompose user clicks into two probabilities: examination (did the user look at this position?) and attractiveness (given they looked, did they click?).
The Fundamental Decomposition
A click happens only when two conditions are both true: the user examined that position AND found the item attractive. Mathematically: P(click) = P(examine) × P(attract). Examination depends on position: users examine position 1 with probability 0.95, position 5 with 0.40, and position 10 with 0.10. Attractiveness depends on the item itself. By separating these, we estimate true relevance independent of display position.
How Click Models Learn These Probabilities
The Position Based Model assumes examination depends only on position and attractiveness only on the item. You observe clicks where the same item appears at different positions. If item A at position 1 gets 30% clicks and at position 5 gets 6% clicks, you work backwards. Position 1 has examination probability 0.9 and position 5 has 0.2. Dividing: 0.30 ÷ 0.9 = 0.33, and 0.06 ÷ 0.2 = 0.30. The attractiveness is roughly constant, confirming the model works.
Why This Separation Matters For Training
Once you estimate attractiveness separately, use it as a debiased training signal instead of raw clicks. An item with few clicks at position 8 might have high attractiveness because examination probability is low. Dividing observed clicks by examination probability recovers the true relevance signal. This is the foundation of inverse propensity scoring.
💡 Key Insight: The same item at position 1 versus position 10 has dramatically different click rates but identical attractiveness. Click models recover this by dividing out the position effect.

💡 Key Takeaways

✓Clicks decompose into P(click) = P(examine) × P(attract), where examination depends on position and attractiveness on the item

✓Examination probability drops sharply: 95% at position 1, 40% at position 5, 10% at position 10

✓By observing the same item at different positions, you estimate position independent attractiveness

✓Attractiveness estimates become debiased training signals reflecting true relevance

📌 Interview Tips

1Walk through P(click) = P(examine) × P(attract) with numbers at two positions to show how attractiveness stays constant.

2Explain that examination probabilities come from aggregate data where items appear at different positions across many queries.

3Emphasize the goal is recovering attractiveness, which represents true item quality independent of display position.

← Back to Relevance Feedback (Click Models, Position Bias) Overview