ii-sdv 2014 recommender systems for analysis applications (roger bradford - agilex technologies,...

Recommender Systems for Analysis Applications

Roger BradfordAgilex Technologies

14 April 2014

International Information Conference on Search, Data Mining and Visualisation

2

• Customers who Shopped for ' A Tale of two Cities' also Shopped for ….

• Customers Who Bought Items in Your Recent History Also Bought ….

• Users who Enjoyed Titanic also Enjoyed ….

Recommender Systems in Internet Commerce

3

Provider Items RecommendedAmazon Items to BuyFastWeb ScholarshipsLeShop Groceries to BuyNetflix Movies to RentPandora Music to Listen toTripadvisor Places to VisitTwitter People to FollowYaHoo Movies to Watch

Popular Commercial Recommender Applications

4

• Business Strategy Development• Investment Analysis• Risk Analysis• IP Analysis• Fraud Detection• Event Monitoring• Technology Monitoring

Example Analysis Applications

In Analytic Applications, Recommender Systems Primarily Function as

Knowledge Discovery Tools

5

Value of Recommender Systems for Analysis

� Automatically Identify Important Information in Large Quantities of Incoming Data

� Reduce the Cognitive Load on Analysts� Aid in Discovery of New Relevant Information

- that the User didn’t Know to Search for� Produce Alerts about Entities of Importance –

not just more Documents to Read

6

Typical Commercial Applications

Typical Analytic Applications

# of Users >> # of Items # of Items >> # of Users

User Interests are Fairly Stable

User Interests are Dynamic

Unambiguous Indicators are Available

Indicators are Mostly Subtle

Missing a Recommendation Typically has Small Impact

Missing a Recommendation may have a Large Impact

Recommender Application Differences

7

Approach Recommendations Based on

• Collaborative Filtering Actions of Other People

• Content-based Characteristics of Items

• Demographic User Characteristics

• Knowledge-based Example Cases or Constraints

• Community-based Social Networks

• Hybrid Combinations of the Above

Implementation Approaches

8

Incoming Reporting Stream

RecommenderEngine

User-provided Exemplars

XxxxxxxxxXxxxxxxxx

.criminalXxxxxxxxx

...crime..

RecommendedDocuments Recommended

Entities

User Action

Artifacts

Jason Brown

Robert Fisher

Walter Williams

Analytic Recommendation Process

9

Example User Interface

Example Documents

used to Define

Interests

Recommended Items in Relevance Order

Confidence Indictors

A 2011 report issued by the US Geological Survey and US Department of the Interior, "China's Rare-Earth Industry," outlines industry trends within China and examines national policies that may guide the future of the country's production. The report notes that China's lead in the production of rare-earth minerals has accelerated over the past two decades. In 1990, China accounted for only 27% of such minerals. In 2009, world production was 132,000 metric tons; China produced 129,000 of those tons. According to the report, recent patterns suggest that China will slow the export of such materials to the world: "Owing to the increase in domestic demand, the Government has gradually reduced the export quota during the past several years." I

User Feedback Mechanism

Exemplar Management

Console

10

Key Requirements forAnalytic Recommenders

� Quickly Identify and Present Desirable Information to the User without Overwhelming the User with Irrelevant Information.

� Be Flexible Enough to Deal with Variability in Individuals and Activities

� Evaluate Complex Associations Based on Multiple Attributes (Including Metadata)

� Incorporate Data from Multiple Sources.� Begin Making Recommendations Based on

Small Amounts of Data

11

� Accommodate Data Volumes that can be Expected to be Very Large

� Deal with Data that is Sparse, Incomplete, and Noisy.

� Make Explanations of the Reasoning Used in Reaching the Recommendations Available to the User.

� Work with Data from Existing Corporate or Government Data Repositories.

Key Requirements forAnalytic Recommenders (Cont’d)

12

• # of Items >> # of Users • Dynamic Items & User

Interests• High Accuracy & Low Miss

Rate Requirements

Requirements Drive Implementation Approach

Primary Recommendation

Technique must be Content-based

Matrix Factorization is the best Available Content-based Approach

13

� 100 Million Ratings of 17,770 Movies by > 480,000 Users

� $1Million (US) Prize for 10% Improvement� 44,000 Entries, From Over 41,000 Teams� Won by Koren and Bell using a Combination

of Techniques, Featuring Matrix Factorization

The Netflix Challenge

14

Matrix Factorization Advantages*

� Prediction Accuracy Superior to Other Techniques.� Use of a Memory-efficient, Compact Model.� Simple Training.� Natural Ability to Integrate Multiple Forms of User Feedback.� Ability to Incorporate Temporal Dynamics of User Interests

and Item Attributes.� No Reliance on Arbitrary or Heuristic Similarities.� Inherent Protection against Overfitting.� Ability to Capture the Totality of Weak Signals in the Data. � Ability to Incorporate Confidence Levels.� High Scalability.

*Koren & Bell, Recommender Systems Handbook, Springer, 2011

15

Rec

omm

enda

tion

Acc

urac

y C

ompa

red

to B

asel

ine

Degree of Text Corruption

Noise Resilience

16

Search Terms

Viewing an Item

Time Spent Viewing an Item

Saving an Item

Printing an Item

Refining User Interests

Explicit Input Implicit Indicators

Exploit both Positive and Negative Indicators

17

• Accuracy• Confidence Indicators for Recommendations• User Control• Explanation

Contributors to User Confidence

18

Explainability - Documents

19

Explainability - Entities

20

Lists

Tables

Text

Analyst’s Notes:

Identified Relevant

Documents

DocumentsIn

NoveltyOrder

Previously Seen Information

PublishedReports

PreviouslyReviewedDocuments

Novelty in Recommendations

21

Crosslingual Recommendations

Documents in Multiple Languages

FarsiArabic

English

Recommendations in Relevance Order

Recommended Items

22

Acc

urac

y +

Com

plet

enes

sof

Cat

egor

izat

ion

Number of Simultaneous Languages

English Documents &English Examples

Documents in Latin Languages & English Examples

Range of Human

Performance

High-Accuracy Multilingual Recommendations

23

Multimedia Recommendations

Integrated Semantic Analysis

Structured Data

Images

Text Audio

8/18/02500 lbPicric Acid

Saif al Adel

ZaidKhayr

DateAmountMaterialSellerBuyer

Sensor Data

Video

Geospatial DataBiometrics

24

High Performance with Modest HardwareTi

me

in H

ours

Number of Documents

K K KK K

Minimum Latency –Single Processor

Maximum Throughput –16-node Hadoop Cluster

25

� Algorithm Scalability� Conversational Recommender Systems� Context-aware Recommenders� Explanations and Evidence� Preference Elicitation� Privacy and Security� Semantic Web Technologies for

Recommendation� Trust and Reputation

Recommender Topics of Current High Interest

26

� The ACM Recommender System Conference (RecSys 2014), Foster City, California, USA, 6-10 October 2014

� Recommender Systems Handbook , F. Ricci, L. Rokach, B. Shapira, and P. Kantor, Springer Publishing, 2011 118€

� Recommender Systems , P. Melville and V. Sindhwani, In Encyclopedia of Machine Learning, Springer, 2010. Available at: http://www.prem-melville.com/publications/recommender-systems-eml2010.pdf

� Matrix Factorization Techniques for Recommender Systems , Y. Koren, Y., R. Bell, and C. Volinsky, IEEE Computer, August 2009, pp. 42-49. Available at: http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf

Resources

27

Questions or Comments

Roger BradfordAgilex Technologies Incr.bradford@agilex.com