← AI HoodmanNo. 001 · Ads

Position Bias

Are your top ads really the best?

01of 06
Use the arrows, swipe or the arrow keys to turn the slides. Click a slide to read it page by page.
  1. Slide 1 of 6: Your ad ranking model keeps putting the same ads at the top. They get the most clicks. Weekly ranking of the top 5 ads Ad A #1 #1 every week #2 Ad C #3 Ad D #4 Ad E #5 Ad B Week 1 Week 2 Week 3 Week 4 But are they the best ads?
  2. Slide 2 of 6: Not necessarily. They might just be in the best spot. Feed slot How many people see it #1 #2 #3 #4 #5 Ads at the top get more clicks simply because more people see them.
  3. Slide 3 of 6: Your click data mixes two signals. One click. Two possible reasons. Reason 1 Reason 2 "This ad is good" "This ad was shown at the top" user_27410 slot #1 ad_A clicked = 1 Both produce the same click, so the model can't tell them apart.
  4. Slide 4 of 6: The loop feeds itself. 01 02 Model ranks the ad at the top More people see it repeat 04 Model thinks it's a great ad 03 It gets more clicks This is called position bias.
  5. Slide 5 of 6: The standard fix: Let the model see position, then neutralize it. 01 During training ad features + position = 3 model The model learns to credit part of each click to the slot, not the ad. 02 At serving time ad A pos 1 ad B pos 1 ad C pos 1 Score every candidate as if it were in slot 1. Same boost for everyone, so position can't change the order.
  6. Slide 6 of 6: Takeaway: Train with position. Fix it when serving.

Keep reading

Position Bias · 1 / 6
  1. Slide 1 of 6: Your ad ranking model keeps putting the same ads at the top. They get the most clicks. Weekly ranking of the top 5 ads Ad A #1 #1 every week #2 Ad C #3 Ad D #4 Ad E #5 Ad B Week 1 Week 2 Week 3 Week 4 But are they the best ads?
  2. Slide 2 of 6: Not necessarily. They might just be in the best spot. Feed slot How many people see it #1 #2 #3 #4 #5 Ads at the top get more clicks simply because more people see them.
  3. Slide 3 of 6: Your click data mixes two signals. One click. Two possible reasons. Reason 1 Reason 2 "This ad is good" "This ad was shown at the top" user_27410 slot #1 ad_A clicked = 1 Both produce the same click, so the model can't tell them apart.
  4. Slide 4 of 6: The loop feeds itself. 01 02 Model ranks the ad at the top More people see it repeat 04 Model thinks it's a great ad 03 It gets more clicks This is called position bias.
  5. Slide 5 of 6: The standard fix: Let the model see position, then neutralize it. 01 During training ad features + position = 3 model The model learns to credit part of each click to the slot, not the ad. 02 At serving time ad A pos 1 ad B pos 1 ad C pos 1 Score every candidate as if it were in slot 1. Same boost for everyone, so position can't change the order.
  6. Slide 6 of 6: Takeaway: Train with position. Fix it when serving.