semantic search in the cloud
DESCRIPTION
Presented by Roberto Masiero, Vice President ADP Innovation Lab, ADP In this presentation we will cover ADP's Semantic Search strategy and implementation. From the use cases to the design to support semantic searches on a vast set of data, to crawling data from hundreds of data sources. We will also cover our architecture to scale the search service on a multi-tenant SaaS environment.TRANSCRIPT
2011 – ADP #87
2011 - Rank #7
2010
2010 & 2011
PINNACLE AWARDS 2011 Global Outsourcing Services
Provider of the Year
ADP #6
Financial Data Services Rank #2
>300,000 SaaS clients
>18 Million users
81 countries & expanding
Widest breadth of HCM solutions
BPO/Service offerings extend reach beyond traditional SaaS
ADP is the Largest Global SaaS Provider of Cloud-based Enterprise Software
2
Research and Incubation…
9
Current Major Research Areas
Personal Cloud
Big Data / Analytics & Search
Global SOR
Mobility Social Platform
© Copyright 2012 ADP, Inc. Proprietary Information.
National Employment Analytics
11
Insights to US economic activity
500,000+ companies
Several dimensions
‘Large’ Data + R
© Copyright 2013 ADP, Inc. Proprietary Information.
Innovation: ADP Intelligence & Analytics
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information. 12
User Friendly Analytics
Embedded and Contextualized Content
Benchmarks by Industry/Location
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information. 13
Behavioral Intelligence
• Using BigData to create an unparalleled user experience
• Driving performance using in-memory data
• 3Vs (Volume, Velocity and Variety)
U i Bi D t t t
Payroll – Heat map time slice
BigData - allow us to learn from data
impressions
(map)
al Int
(reduce)
ggMary will probably check her Payroll and 401k
today at 9:10AM
Pre-Cache (in-memory) payStatements and retirementPlan objects
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.
ADP Semantic Search: The Requirements
Enterprise Search Goals Search as a service across multiple ADP products and clients Search as the primary navigation tool Semantic Search: verbs and nouns
Key Search Architecture Elements Multi-Tenancy Multi-Datacenter Large # of sources Search High-Availability Crawling and Indexing HA Disaster Recovery (Active-Active)
Key Functional Design Elements Instant Search across all object types Partial text and phrase match Highlighting Faceting by document types Query suggestions User’s click history Access control
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.
ADP Semantic Search
HIRE = task/verb
JOHN Doe= object/noun
applies to CANDIDATE object
is there a candidate named John Doe?
invoke action ONBOARD JOHN DOE
Just ask the app
A new navigation paradigm
Foundation for voice command
Simpler/Faster algorithm than NLP
Search objects (nouns) and tasks (verbs) to create (actions)
User behavior feeds taxonomy/relevance on both positive and negative searches
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.
ADP Semantic Search: The Stack
Data
Index
Query
User
MetaData
Using Lucene/Solr as search index
for both data and meta data
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.
ADP Semantic Search: Simple APIs
GET /v1_0/search/warmupHeader: Realm, Org OID, Associate OID, Locale Params: NoneResult: Language-specific labels JSON
GET /v1_0/searchHeader: Realm, Org OID, Associate OID, LocaleParams: q=<query text>, groupValue=<Document type to filter on> Result: Search Results if q is non-empty. Click history if q is empty.
POST /v1_0/search/clicks/{canonicalURI}Header: Realm, Org OID, Associate OID, Locale Result: Status OK (200)
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.
ADP Semantic Search: SOLR Out-of-the-box vs Custom Features
Out-of-the-Box • Text search using inverted
index • REST APIs w/ JSON
Output • Track query execution time • Fuzzy search using Ngrams
and Phonetics • Result Grouping and
Faceting • Result highlighting • Indexed vs. Stored fields • Multi-lingual Search (UTF-8)
Custom Features • Multi-tenancy (enforced via predicates) • Access Control (enforced via
predicates) • Instant Search (regular search from Solr
viewpoint) • Dynamic relevance (via custom
predicates built in Proxy) • Synonym enrichment (via meta index) • Smart actions via Task-Object mapping
(via meta • Index) • Semantic Search (via custom
predicates built in Proxy) • Click-history based personalized result
boosting (via custom predicates based on captured clicks)
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.
User Experience: Changing our Mental Model
Start Here
© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.
User Experience: The Design Conversation has two aspects
Intuitive Complex
Inse
curit
y C
onfid
ence
Overly Simplistic
Specialists Required
Not User-friendly
Engaging