← AI HoodmanNo. 008 · Embeddings

User ID Embeddings

How does a model tell User 1001 from User 1002?

01of 07
Use the arrows, swipe or the arrow keys to turn the slides. Click a slide to read it page by page.
  1. Slide 1 of 7: User 1001 and User 1002 signed up one second apart. User 1001 User 1002 10-minute pasta Sourdough basics Match highlights What the model sees Esports finals recap 1001 ≈ 1002 One number apart How does it tell them apart?
  2. Slide 2 of 7: The key idea: IDs are names, not numbers. User ID Embedding 1000 [0.12, 0.55, −0.20, …] 1001 [−0.08, 0.31, 0.14, …] 1002 [0.22, −0.17, 0.05, …] 1003 [0.40, 0.18, −0.66, …] Before training: random Cooking fans 1002 Gamers 1001 Each ID gets its own row. It starts out random.
  3. Slide 3 of 7: Training: Every watch nudges the row. 01 02 03 1001 watched a pasta video Model guessed Row 1001 moves toward cooking label = 1 0.30 too low After training Cooking fans Gamers far apart 1001 1002 Users who watch the same things end up close.
  4. Slide 4 of 7: The catch: One row per ID doesn't scale. 2B users × 64 numbers × 4 bytes 512 GB Just for users. Videos, creators and every other ID need tables too. Sharding Hashing Most common How How Split the table across machines row = hash(ID) mod 10M Memory Memory 512 GB 2.56 GB New IDs New IDs Table keeps growing Work instantly Downside Downside More hardware 2B ≈ 200 users per row 10M
  5. Slide 5 of 7: Hashing's catch: Users end up sharing rows. 1001 and 1002 both hash to row 4,812,003 Cooking fans Gamers 1001 pulls 1002 pulls Shared row stuck in between Both users train the same vector. It ends up fitting neither.
  6. Slide 6 of 7: The fix: Give active users their own rows. Shared rows Own rows Promote everyone else +195 +198 enough recent activity Evict +194 30 days inactive many users per row active users 1001 1002 1003 one user per row A promoted user starts from their shared row.
  7. Slide 7 of 7: Takeaway: Don't read the ID. Learn the person behind it.

Keep reading

User ID Embeddings · 1 / 7
  1. Slide 1 of 7: User 1001 and User 1002 signed up one second apart. User 1001 User 1002 10-minute pasta Sourdough basics Match highlights What the model sees Esports finals recap 1001 ≈ 1002 One number apart How does it tell them apart?
  2. Slide 2 of 7: The key idea: IDs are names, not numbers. User ID Embedding 1000 [0.12, 0.55, −0.20, …] 1001 [−0.08, 0.31, 0.14, …] 1002 [0.22, −0.17, 0.05, …] 1003 [0.40, 0.18, −0.66, …] Before training: random Cooking fans 1002 Gamers 1001 Each ID gets its own row. It starts out random.
  3. Slide 3 of 7: Training: Every watch nudges the row. 01 02 03 1001 watched a pasta video Model guessed Row 1001 moves toward cooking label = 1 0.30 too low After training Cooking fans Gamers far apart 1001 1002 Users who watch the same things end up close.
  4. Slide 4 of 7: The catch: One row per ID doesn't scale. 2B users × 64 numbers × 4 bytes 512 GB Just for users. Videos, creators and every other ID need tables too. Sharding Hashing Most common How How Split the table across machines row = hash(ID) mod 10M Memory Memory 512 GB 2.56 GB New IDs New IDs Table keeps growing Work instantly Downside Downside More hardware 2B ≈ 200 users per row 10M
  5. Slide 5 of 7: Hashing's catch: Users end up sharing rows. 1001 and 1002 both hash to row 4,812,003 Cooking fans Gamers 1001 pulls 1002 pulls Shared row stuck in between Both users train the same vector. It ends up fitting neither.
  6. Slide 6 of 7: The fix: Give active users their own rows. Shared rows Own rows Promote everyone else +195 +198 enough recent activity Evict +194 30 days inactive many users per row active users 1001 1002 1003 one user per row A promoted user starts from their shared row.
  7. Slide 7 of 7: Takeaway: Don't read the ID. Learn the person behind it.