reco4 @ paris meetup (may 20th)
DESCRIPTION
This presentation shows reco4j features and vision. In particular we add the new concept of context aware recommendation and how we integrate it into reco4j. See the project site for more details here: http://www.reco4j.orgTRANSCRIPT
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013
Reco4J Project Intelligent RecommendaDons for
Your Business
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 1
Recommender Systems • A system that can recommend or present items to the user based on the user’s interests and interacDons
• One of the best ways to provide a personalized customer experience
• Built by exploiDng collecDve intelligence to perform predicDons
• Examples: Amazon, YouTube, NeUlix, Yahoo, Tripadvisor, Last.fm, IMDb
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 2
The Example: NeUlix • The world largest online movie rental services, 33 million members in 40 countries
• 60% of members selecDng movies based on recommendaDons (September 2008)
• NeUlix Prize: US$ 1,000,000 was given to the BellKor's PragmaDc Chaos team which bested NeUlix's own algorithm for predicDng raDngs by 10.06% (September 2009)
• 75% of the content watched on the service comes from its recommendaDon engine (April 2012)
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 3
Why Recommender Systems • Standard uses:
– Increase the number of items sold – Sell more diverse items – Increase the user saDsfacDon – Increase user fidelity – Beger understand what the user wants
• Advanced uses: – Create ad hoc campaigns (per geographic area, per type of users) – OpDmize products distribuDon over a wide area for large retail chains
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 4
Problem • There are no available sohware products for state-‐of-‐the-‐art recommender systems
• A high-‐end recommender engine can be built only through expensive custom projects
• Large scale user/item datasets require a big data approach
• There is no "best soluDon" • There is no "one soluDon fits all” • The NeUlix winner composed 104 different algorithms
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 5
SoluDon: Reco4J
A graph-‐based recommender engine
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 6
Reco4J Main Goals • Implement the state-‐of-‐the-‐art in the recommendaDon on top of a graph model
• Provide sohware / cloud services / consultancy
• Contribute to the RecSys research field
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 7
Reco4J Features • Composable models/algorithms • Persistent models (mulD model supported) • Updatable models • Commercial and research-‐oriented algorithms • Independent from source knowledge datasets • Cluster and cloud-‐ready • Context-‐aware recommendaDons • MulDtenant • Social recommendaDons
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 8
Reco4J Under the Hood • J is for Java • CollaboraDve filtering algorithms – Neighborhood-‐based methods – Latent factor models
• Neo4J graph database: – Data source repository – Persistent model repository
• Hadoop cluster/MapReduce • Apache Mahout
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 9
Algorithms Roadmap • CollaboraDve filtering
– Memory based (Neighborhood) • User/Item based
– Several distance algorithms (Cosine, Euclidean, Tanimoto, etc.) • Graph based
– Path Based Similarity (Shortest Path, Number of Paths) – Random Walk Similarity (Item Rank, Average first-‐passage/commute Dme)
– Model based (Latent factor) • StochasDc gradient descendant • AlternaDng least square
• Social recommendaDon – Trust based approach – ProbabilisDc approach
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 10
Algorithms Roadmap (2) • Crosswise Features – Context awareness – Composability – Real-‐Dme – ParallelizaDon
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 11
Context-‐Aware RecommendaDon “The ability to reach out and touch customers anywhere means that companies must deliver not just compe;;ve products but also unique, real-‐;me customer experiences shaped by customer context”
C. K. Prahalad
• Incorporate contextual informa5on in the recommendaDon process • Modeling contextual InformaDon
– From: User x Item -‐> RaDng – To: User x Item x Context -‐> RaDng
• Hierarchical structure • Three approaches
– Contextual pre-‐filtering – Contextual post-‐filtering – Contextual modeling
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 12
Reco4J Context Model
users items rate(c1, …, cn)
………….
contextualized ra;ngs
Context Tuple DAG
usera-‐mp1 itemc-‐split1
…….. usera-‐mp2
itemc-‐split2
itemd-‐splitk
userb-‐mpk
…….. ……..
user micro profiles item spliEng
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 13
Advantage of graph database • NoSQL database to handle BigData issue • Extensibility • No aggregate-‐oriented database • Minimal informaDon needed • Natural way for represenDng connecDons:
– User -‐ to -‐ item – Item -‐ to -‐ item – User -‐ to -‐ User
• Graph Based/Social Algorithms • Graph ParDDoning (sharding) • Performance
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 14
Example: Find Neighbors
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 15
Why Neo4J? • Java based • Embeddable/Extensible • NaDve graph storage with naDve graph processing engine
• Open Source, with commercial version • Property Graph • ACID support • Scalability/HA • Comprehensive query/traversal opDons
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 16
RecommendaDon Model
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 17
Persistence Model
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 18
Persistence Model
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 19
Persistence Model
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 20
Reco4J + Hadoop • Queue Based Process • Operates both on cluster and cloud • Each process downloads data from
Neo4J/Reco4J before or during computaDon
• Stores data into Reco4J Model
• Scaling augmenDng the number of: • Neo4J Nodes (only one master) • Hadoop Nodes
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 21
Reco4J in the Cloud • Recommenda)on as a service (RaaS) • Reco4J cloud infrastructure offers: – Pay as you need – Pay as you grow – Support for burst – Periodical analysis at lower costs – Test/evaluate several algorithms on a reduced dataset – Compose algorithms dynamically
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 22
Consultancy Goals
Analysis
Data Source
ExploraDon
Process DefiniDon
Import Data
Test/EvaluaDon
Deploy
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 23
Thank you
Alessandro Negro Linkedin: hgp://it.linkedin.com/in/alessandronegro/ Email: [email protected] Luigi Giuri Linkedin: it.linkedin.com/in/luigigiuri Email: [email protected] Reco4J Site: hgp://www.reco4j.org Twiger: @reco4j GitHub: hgps://github.com/reco4j