from a toolkit of recommendation algorithms into a real business: the gravity r&d experience...
DESCRIPTION
Talk given at Recsys Challenge Workshop in Dublin (@ ACM Recsys 2012), on 13.09.2012.TRANSCRIPT
![Page 1: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/1.jpg)
From a toolkit of recommendation algorithms into a real business:
13.09.2012.
the Gravity R&D experience
![Page 2: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/2.jpg)
The kick-start
13.09.2012.2 From a toolkit of recommendation algorithms into a real business
![Page 3: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/3.jpg)
Facing with real needs
What we had• rating prediction algorithms
• coded in various languages
• blending mechanism
• accuracy oriented
What clients wanted• recommendations that
bring revenue
• robustness
• low response time
• easy integration
• reporting
13.09.2012.3 From a toolkit of recommendation algorithms into a real business
![Page 4: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/4.jpg)
What we do?
13.09.2012.4 From a toolkit of recommendation algorithms into a real business
users
content of service provider
recommender
![Page 5: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/5.jpg)
Explicit vs implicit feedback
No ratings but interactions
sparse vs. dense matrix
requires different learning
13.09.2012.5 From a toolkit of recommendation algorithms into a real business
![Page 6: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/6.jpg)
Increase revenue: A/B tests
against the original solution
internally
13.09.2012.6 From a toolkit of recommendation algorithms into a real business
![Page 7: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/7.jpg)
Robustness
13.09.2012.7 From a toolkit of recommendation algorithms into a real business
IMPRESSApplication Server #1
Reco LAN
IMPRESSApplication Server #2
Reporting Subsystem
Platform OSS/BSS
Database #1 Database #2
Backend LAN
SOAP
CSV over FTP
Firewall
IMPRESS Frontendweb server #1
TV Service LAN
End users
Management LAN
Nagios MonitoringAggregator
IMPRESS Frontendweb server #2
Load Balancer
HP OpenView
SQL SQL
HTTP/ SQL
HTTP/ SQL
HTTP HTTP(S)
SNMP
![Page 8: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/8.jpg)
Time requirements
• Response time: few ms (max 200)
• Training time: maximum few hours
• regular retraining
• incremental training
• Newsletters:
• nightly batch run
13.09.2012.8 From a toolkit of recommendation algorithms into a real business
![Page 9: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/9.jpg)
Productization
13.09.2012.9 From a toolkit of recommendation algorithms into a real business
IMPRESSfor
IPTV, CATV and satellite
Recommends
Personally Relevant
Linear TV, VOD, catch-up TV and more
Gravity personalization platform
AD•APTfor
ad networks and ad server providers
Recommends Personally Relevant
ads
RECOfor
e-commerce
Recommends Personally Relevant
products & services
![Page 10: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/10.jpg)
The 5% question – Importance of UI
Francisco Martin (Strands): „the algorithm is only 5% in the success of the recommender system”
• placement below or above the fold scrolling easy to recognize floating in
• title not misleading explanation like
• widget carrousel static
13.09.2012.10 From a toolkit of recommendation algorithms into a real business
![Page 11: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/11.jpg)
Recommendation scenario
13.09.2012.11 From a toolkit of recommendation algorithms into a real business
Item2Item recommendation
logic: the ad’s profile will be
matched to the profile model of
available ads
![Page 12: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/12.jpg)
Marketing channels
13.09.2012.12 From a toolkit of recommendation algorithms into a real business
Changing the order of two boxes: 25% CTR increase
![Page 13: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/13.jpg)
Cannibalization
• Goal: increase user engagement
• Measurements
• average visit length
• average page views
• Effect of accurate recommendations:
• use of listing page ↓
• use of item page ↑
• Overall page view: remains the same
• Secondary measurements
• Contacting
• CTR increase
13.09.2012.13 From a toolkit of recommendation algorithms into a real business
?
![Page 14: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/14.jpg)
Evolution: increased user engagement
• not a cold start problem
• parameter optimization and user engagement
13.09.2012.14 From a toolkit of recommendation algorithms into a real business
![Page 15: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/15.jpg)
KPIs – may change during testing
13.09.2012.15 From a toolkit of recommendation algorithms into a real business
![Page 16: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/16.jpg)
Complete personalization: coupon-world
• Newsletter (daily + occassionally)
• Ranking all offers on the website
• top1 item
• category preferences
• user metadata (gender, age, …)
• user category preferences (seldom given)
• item metadata
• context
• customer vs. vendor
13.09.2012.16 From a toolkit of recommendation algorithms into a real business
![Page 17: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/17.jpg)
Business rules – driving/overriding ranking
13.09.2012.17 From a toolkit of recommendation algorithms into a real business
![Page 18: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/18.jpg)
Contexts
13.09.2012.18 From a toolkit of recommendation algorithms into a real business
![Page 19: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/19.jpg)
Context at TV program recommendation
• TV (EPG program & video-on-demand) explicit and implicit identification of the user in the household time-dependent recommendation
13.09.2012.19 From a toolkit of recommendation algorithms into a real business
![Page 20: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/20.jpg)
Some results (offline)
13.09.2012.20 From a toolkit of recommendation algorithms into a real business
Recall@20 MAP@20 Recall@20 MAP@20Grocery 64,31% 137,96% 89,99% 199,82%TV1 14,77% 43,80% 28,66% 85,33%TV2 -7,94% 10,69% 7,77% 14,15%LastFM 96,10% 116,54% 40,98% 254,62%
Dataset
Improvement using seasoniTALS iTALSx
Recall@20 MAP@20 Recall@20 MAP@20Grocery 84,48% 104,13% 108,83% 122,24%TV1 36,15% 55,07% 26,14% 29,93%
Dataset
Improvement using SeqiTALS iTALSx
Some results (online)
![Page 21: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/21.jpg)
Anecdotes
• Item2item recommendations – bookstore
• Placebo effect
• buyer vs. seller
13.09.2012.21 From a toolkit of recommendation algorithms into a real business
![Page 22: From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)](https://reader035.vdocuments.us/reader035/viewer/2022062614/5476f68db4af9fa4388b489f/html5/thumbnails/22.jpg)
Conclusion
• Offline and online testing
• From simple to sophisticated
• Many more potential fields of application
13.09.2012.22 From a toolkit of recommendation algorithms into a real business