ML-Powered Search & Ranking • Relevance Feedback (Click Models, Position Bias)Medium⏱️ ~2 min
How Do Click Models Separate Examination from Attractiveness?
Click models decompose the observed click probability into two factors: the probability a user examines an item at a given position, and the probability they click given they examined it. This factorization lets you learn true relevance without confounding it with position effects.
The Position Based Model (PBM) is the foundation. It models p(click) as p(seen at position and context) multiplied by p(click given seen and item features). The first term captures examination bias that depends on position, device, and layout. The second term captures attractiveness based on the item itself. At inference time, you rank using only the attractiveness component, ignoring position.
More sophisticated models add sequential browsing. The Cascade model assumes users scan top to bottom and stop when satisfied. The Dynamic Bayesian Network (DBN) model adds richer state transitions like satisfaction and abandonment. These models better capture real user behavior but require more data and computation to train.
A practical implementation uses additive decomposition. Train a model that outputs score equals f(features) plus g(position and context). Optimize cross entropy on observed clicks. At serving time, drop the g component and rank using only f(features). ManoMano deployed this approach in production with gradient boosted trees and reported improved ranking quality over naive position as a feature. The key insight is that you train on biased data but serve with an unbiased scoring function.
💡 Key Takeaways
•Click models factorize p(click) into p(seen at position) times p(click given seen), separating position bias from true attractiveness
•Position Based Model is simplest, Cascade models add sequential browsing, Dynamic Bayesian Networks add satisfaction state
•Practical implementation trains score equals f(features) plus g(position), then uses only f at inference time for unbiased ranking
•ManoMano deployed additive decomposition in gradient boosted trees in production and achieved better ranking quality than using position as a raw feature
•Assumes examination depends only on position and context, not item specifics, which can be violated by visually salient thumbnails or titles
📌 Examples
Train a neural network that outputs relevance_score equals item_embedding_layer plus position_bias_layer. Optimize binary cross entropy on observed clicks. At serving, compute rankings using only item_embedding_layer and ignore position_bias_layer.
Google Search uses cascade click models to understand that users scan sequentially. If position 1 is clicked, the user likely did not examine position 2 or 3. This informs how much weight to give negative signals (no click) at lower positions.