Recommendation Systems • Position Bias & Feedback LoopsHard⏱️ ~3 min
Production Implementation: Eventing, Calibration, and Whole Page Optimization
Production debiasing starts with correct eventing. Viewport visible impressions are critical: counting server side insertions as impressions overstates exposure because many items in infinite scroll or below the fold are never actually seen. Industry standard visibility definitions (Interactive Advertising Bureau or IAB guidelines) require 50% of pixels visible for at least 1 second. Without viewport tracking, negative examples in training data are polluted with items users never saw, distorting your model's understanding of relevance.
Position aware counters maintain CTR at position and context curves continuously. If position 1 on mobile averages 8% CTR and overall average is 2%, you can construct position weights: impressions at position 1 get weight equals 0.08 / 0.02 equals 4; position 5 at 1% CTR gets weight 0.01 / 0.02 equals 0.5. Inverse weighting corrects clicks: position 1 clicks get weight 0.02 / 0.08 equals 0.25 (downweight inflated top positions); position 5 clicks get weight 0.02 / 0.01 equals 2 (upweight underexposed positions). These curves must be recomputed per device, layout, and surface because position effects differ dramatically: mobile above fold is tighter, tablets show more items, desktop grids change visual hierarchy.
Whole page optimization recomputes candidate scores per position rather than reusing a single score. If you're ranking 50 candidates for a 10 item page and your model includes position or context features, you need to evaluate each candidate at each target position, scaling inference cost to 50 × 10 equals 500 scoring operations. This matters when page layout strongly modulates visibility (carousels, heterogeneous modules, above fold cutoffs). Netflix uses per row scoring because row 1 on the homepage has 10x the visibility of row 5. The CPU budget tradeoff is steep: inference latency must stay under 100 milliseconds end to end for real time serving, forcing aggressive model simplification, caching, or hybrid approaches where only top K candidates get full per position rescoring.
Monitoring and recalibration prevent drift. Track position lift diagnostics: compare metrics between randomized buckets and production ranking. If randomized traffic CTR is 3.5% but production is 4.2%, and your debiasing predicts only 3.8%, you're undercorrecting by 0.4 percentage points, indicating your position curves or factorization assumptions need updating. Trigger automatic recalibration after UI changes, device mix shifts, or quarterly reviews. Use shrinkage and regularization across sparse positions to avoid overfitting: if position 47 has only 100 impressions, borrow strength from nearby positions rather than learning a noisy curve from limited data.
💡 Key Takeaways
•Viewport visible impressions (IAB standard: 50% of pixels visible for 1+ second) prevent polluting negative examples with never seen items in infinite scroll and below fold placements.
•Position weights for training: impressions at position 1 (8% CTR) get weight 0.08 / 0.02 equals 4 times average; clicks at position 1 get inverse weight 0.02 / 0.08 equals 0.25 to downweight inflation.
•Whole page optimization rescores candidates per position: 50 candidates × 10 positions equals 500 inference calls per request. Netflix uses this for homepage rows because row 1 has 10x visibility of row 5.
•Inference latency budget forces tradeoffs: full per position rescoring may push latency from 20 milliseconds to 200 milliseconds, requiring model simplification, caching, or hybrid approaches with partial rescoring.
•Position curves must be recomputed per device, layout, and surface. Mobile above fold is tighter than desktop; tablet grids change visual hierarchy. Recalibrate after every UI change to prevent drift.
•Monitor position lift diagnostics continuously: if randomized CTR is 3.5%, production is 4.2%, and debiased model predicts 3.8%, you're undercorrecting by 0.4 percentage points and need curve updates.
📌 Examples
Meta News Feed: Viewport tracking logs impressions only when post is 50% visible for 0.5 seconds. Removes 40% of server logged impressions, improving negative sampling quality and model calibration for long feeds.
Google Ads per slot calibration: Desktop position 1 has g(pos=1, desktop) equals +0.52 log odds; mobile position 1 is +0.71 because screen is smaller. Separate curves by device prevent mobile ads from being systematically overcharged.
Netflix homepage: Rescores top 30 titles per row (6 rows × 30 candidates equals 180 scoring calls). Row 1 weights 10x row 5. Inference budget is 80 milliseconds; uses distilled 20 layer model instead of full 50 layer to stay under latency target while enabling per row scoring.