summit eu machine learning
DESCRIPTION
The greater promise of Big Data lies not in doing old things in slightly new ways. Instead, it lies in doing new things that were previously not possible. One major class of new things is adding intelligence to large-scale systems. In this session I will present a survey of how machine learning can be applied to real-life situations without having to get a PhD in advanced mathematics. These systems can be built today from open source components to increase business revenues by understanding what customers need and want. I will provide real world examples of best practices and pitfalls in machine learning including practical ways to build maintainable, high performance systems.TRANSCRIPT
![Page 1: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/1.jpg)
Revenue GrowthThrough Machine Learning
Ted Dunning – March 21, 2013
![Page 2: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/2.jpg)
Agenda
• Intelligence – Artificial or Reflected• Quick survey of machine learning– without a PhD– not all of it
• Available components• What do customers really want
![Page 3: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/3.jpg)
Artificial Intelligence?
![Page 4: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/4.jpg)
Artificial Intelligence?
• Turing and the intelligent machine
• Rules?
• Neural networks?
• Logic?
![Page 5: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/5.jpg)
Reflected Intelligence!
• Society is not just a million individuals
• A web service with a million users is not the same as a million users each with a computer
• Social computing emerges
![Page 6: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/6.jpg)
What is Machine Learning?
• Statistics, but …• New focus on prediction rather than
hypothesis testing• Prediction means held-out data, not just the
future (now-casting)
![Page 7: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/7.jpg)
The Classics
• Unsupervised– AKA clustering (but not what you think that is)– Mixture models, Markov models and more– Learn from unlabeled data, describe it predictively
• Supervised– AKA classification– Learn from labeled data, guess labels for new data
• Also semi-supervised and hundreds of variants
![Page 8: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/8.jpg)
Recent Insurgents
• Collaborative learning– models that learn about you based on others
• Meta-modeling– models that learn to reason about what other
models say
• Interactive systems– systems that pick what to learn from
![Page 9: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/9.jpg)
Techniques
• Surprise and coincidence• Anomalous indicators• Non-textual search using textual tools• Dithering• Meta-learning
![Page 10: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/10.jpg)
Surprise and coincidence
• What is accidental or uninteresting?
• What is surprising and informative?
![Page 11: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/11.jpg)
A vice president of South Carolina Bank and Trust in Bamberg, Maxwell has served as a tireless champion for economic development in Bamberg County since 1999, welcoming industrial prospects to the county and working with existing industries in their expansion efforts. Maxwell served for many years as the president of the Bamberg County Chamber of Commerce and remains an active member today.
![Page 12: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/12.jpg)
The goal of learning is prediction. Learning falls into many categories, including supervised learning, unsupervised learning, online learning, and reinforcement learning. From the perspective of statistical learning theory, supervised learning is best understood.
![Page 13: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/13.jpg)
Surprise and Coincidence
• Which words stand out in these examples?
• Which are just there because these are in English?
• The words “the” and “Bamberg” both occur 3 times in the second article– which is the more interesting statistic? Why?
![Page 14: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/14.jpg)
More Surprise
• Anomalous indicators– Events that occur before other events– But occur anomalously often
• Indicators are not causes
• Nor certain
![Page 15: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/15.jpg)
Example #1- Auto Insurance
• Predict probability of attrition and loss for auto insurance customers
• Transactional variables include– Claim history– Traffic violation history– Geographical code of residence(s)– Vehicles owned
• Observed attrition and loss define past behavior
![Page 16: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/16.jpg)
Derived Variables
• Split training data according to observable classes
• Define LLR variables for each class/variable combination
• These 2 m v derived variables can be used for clustering (spectral, k-means, neural gas ...)
• Proximity in LLR space to clusters are the new modeling variables
![Page 17: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/17.jpg)
Example #2 – Fraud Detection
• Predict probability that an account is likely to result in charge-off due to fraud
• Transactional variables include– Zip code– Recent payments and charges– Recent non-monetary transactions
• Bad payments, charge-off, delinquency are observable behavioral outcomes
![Page 18: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/18.jpg)
Derived Variables
• Split training data according to observable classes
• Define LLR variables for each class/variable combination
• These 2 m v derived variables can be used directly as model variables
![Page 19: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/19.jpg)
Search Abuse
• Non-textual search using textual tools– A document can contain non-word tokens– These might be anomalous indicators of an event
• SolR and similar engines can search for indicators– If we have a history of recent indicators, search
finds possible follow-on events
![Page 20: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/20.jpg)
Introducing Noise
• Dithering– add noise– less for high ranks, more for low ranks
• Softens page boundary effects • Introduces more exploration
![Page 21: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/21.jpg)
Meta-learning
• Which settings work best?• Which indicators?
• A/B testing for the back-end
![Page 22: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/22.jpg)
Available components
• Mahout– LLR test for anomaly– Coocurrence computations– Baseline components of Bayesian Bandits
• SolR– Ready to roll for search
![Page 23: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/23.jpg)
History matrix
One row per user
One column per thing
![Page 24: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/24.jpg)
Recommendation based on cooccurrence
Cooccurrence gives item-item mapping
One row and column per thing
![Page 25: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/25.jpg)
Cooccurrence matrix can also be implemented as a search index
![Page 26: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/26.jpg)
Input Data• User transactions– user id, merchant id– SIC code, amount
• Offer transactions– user id, offer id– vendor id, merchant id’s, – offers, views, accepts
![Page 27: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/27.jpg)
Input Data• User transactions
– user id, merchant id– SIC code, amount
• Offer transactions– user id, offer id– vendor id, merchant id’s, – offers, views, accepts
• Derived user data– merchant id’s– SIC codes– offer & vendor id’s
• Derived merchant data– local top40– SIC code– vendor code– amount distribution
![Page 28: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/28.jpg)
Cross-recommendation
• Per merchant indicators– merchant id’s– chain id’s– SIC codes– offer vendor id’s
• Computed by finding anomalous (indicator => merchant) rates
![Page 29: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/29.jpg)
Search-based Recommendations
• Sample document– Merchant Id– Field for text description– Phone– Address– Location
![Page 30: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/30.jpg)
Search-based Recommendations
• Sample document– Merchant Id– Field for text description– Phone– Address– Location
– Indicator merchant id’s– Indicator industry (SIC) id’s– Indicator offers– Indicator text– Local top40
![Page 31: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/31.jpg)
Search-based Recommendations
• Sample document– Merchant Id– Field for text description– Phone– Address– Location
– Indicator merchant id’s– Indicator industry (SIC) id’s– Indicator offers– Indicator text– Local top40
• Sample query– Current location– Recent merchant
descriptions– Recent merchant id’s– Recent SIC codes– Recent accepted offers– Local top40
![Page 32: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/32.jpg)
SolRIndexerSolR
IndexerSolrindexing
Cooccurrence(Mahout)
Item meta-data
Indexshards
Complete history
![Page 33: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/33.jpg)
SolRIndexerSolR
IndexerSolrsearchWeb tier
Item meta-data
Indexshards
User history
![Page 34: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/34.jpg)
Objective Results
• At a very large credit card company
• History is all transactions, all web interaction
• Processing time cut from 20 hours per day to 3
• Recommendation engine load time decreased from 8 hours to 3 minutes
![Page 35: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/35.jpg)
Platform Needs
• Need to root web services and search system on the cluster– Copying negates unification
• Legacy indexers are extremely fast … but they assume conventional file access
• High performance search engines need high performance file I/O
• Need coordinated process management
![Page 36: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/36.jpg)
Additional Opportunities
• Cross recommend from search queries to documents
• Result is semantic search engine
• Uses reflected intelligence instead of artificial intelligence
![Page 37: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/37.jpg)
• What do customers really want?
![Page 38: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/38.jpg)
Another Example
• Users enter queries (A)– (actor = user, item=query)
• Users view videos (B)– (actor = user, item=video)
• A’A gives query recommendation– “did you mean to ask for”
• B’B gives video recommendation– “you might like these videos”
![Page 39: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/39.jpg)
The punch-line
• B’A recommends videos in response to a query– (isn’t that a search engine?)– (not quite, it doesn’t look at content or meta-data)
![Page 40: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/40.jpg)
Real-life example
• Query: “Paco de Lucia”• Conventional meta-data search results:– “hombres del paco” times 400– not much else
• Recommendation based search:– Flamenco guitar and dancers– Spanish and classical guitar– Van Halen doing a classical/flamenco riff
![Page 41: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/41.jpg)
Real-life example
![Page 42: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/42.jpg)
Hypothetical Example
• Want a navigational ontology?• Just put labels on a web page with traffic– This gives A = users x label clicks
• Remember viewing history– This gives B = users x items
• Cross recommend– B’A = label to item mapping
• After several users click, results are whatever users think they should be
![Page 43: Summit EU Machine Learning](https://reader035.vdocuments.us/reader035/viewer/2022062703/554f5b09b4c905524c8b54cc/html5/thumbnails/43.jpg)
Next Steps
• That is up to you• But I can help– platforms (Solr, MapR)– techniques (Mahout, math)
[email protected]@ted_dunning@ApacheMahout