amazon cloudsearch meetup august 15, 2012

© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Amazon CloudSearchMeetup

August 15, 2012


Welcome

Housekeeping

Slides will be posted

Drawing


AgendaIntroduction to CloudSearch• Jon Handler, CloudSearch Solutions Architect

Relevance and Ranking• Jack Conradson, Software Engineer

Case Study: Reddit• Keith Mitchell, Programmer

Q&A


Introduction to CloudSearch


Introduction to Search


Inverted Index

US President


Search On The WebRelevance/RankingFacetingRange SearchingFielded SearchingBoolean QueriesComplex Relevance


Search On The WebRelevance/RankingFacetingRange SearchingFielded SearchingBoolean QueriesComplex relevance


Search On The WebRelevance/RankingFacetingRange-SearchingFielded SearchingBoolean QueriesComplex Relevance


Amazon CloudSearch


Amazon CloudSearch

Fully-managed, full-featured search service Automatically scales for data & trafficHandles both structured and unstructured dataNear real-time indexingUp and running in less than 1 hour


SEARCH CLIENTwww.example.com

SEARCH DEVELOPER

Search API Console

SEARCH ENDPOINT DOCUMENT SERVICE ENDPOINT CONFIGURATION SERVICE ENDPOINT

ConfigurationAPI

CommandLine Tools

ConsoleDocumentService API

CommandLine Tools

Console

SEARCH SERVICESearch Documents

DOCUMENT SERVICEAdd Documents

Update Documents

Delete Documents

Create Domains

Configure Domains

Delete Domains

CONFIGURATION SERVICE

SendSearchRequests

SearchResults

Use the Search Tester

SendDocuments

Create andManage Domains

ACCESS CONTROL ACCESS CONTROL ACCESS CONTROL

Amazon CloudSearch Architecture


Automatic Scaling: Data & Traffic

SEARCH INSTANCEIndex Partition n

Copy 1

SEARCH INSTANCEIndex Partition 2

Copy 2

SEARCH INSTANCEIndex Partition n

Copy 2


Copy n

SEARCH INSTANCE

DATADocument Quantity and Size

TRAFFICSearch Request Volume and Complexity

Index Partition nCopy n

SEARCH INSTANCE


Copy 1


Copy 1


Copy 2


Copy n

Index Partition 1Copy 1


Example: Build Your Playlist


Use Case

Million song dataset http://labrosa.ee.columbia.edu/millionsong/

Search documents are songs• Attributes: title, artist names, years, genre, artist familiarity

We’ll use this to create a “Build Your Playlist” web application.

http://labrosa.ee.columbia.edu/millionsong/




Demo


SDF Documents[

{"type":"add", "id": "sombzze12a8c134960",

"version":5, "lang":"en", "fields":

{"title":"Cajun Twisters", "artist_name":"Adam Ant", "year":"1993", "song_id":"sombzze12a8c134960", "artist_familiarity":449425, "genre":["alternative", "electronic", "instrumental", "rock"] }

}, … ]


Configuration

cs-configure-from-sdf• Analyzes source files for fields and types. Heuristic

Individually


Upload Documents


PHP Integration$results =

file_get_contents(http://search-mn-songs-5bbplyghbb5tk257rsb7iamlsy." ."us-east-1.cloudsearch.amazonaws.com" . "/2011-02-01/search?q=" . $keyword . $bqParam . "&return-fields=title,artist_name,year,genre_result,artist_familiarity&"."facet=year_facet,genre&" . "facet-year_facet-sort=alpha&" ."facet-genre-sort=alpha&" ."facet-genre-top-n=100000&" . "facet-year_facet-top-n=100000&" ."t-year=1985..&" ."t-title=a..&" ."rank=-" . $rank);$resultsObj = json_decode($results);

http://search-mn-songs-5bbplyghbb5tk257rsb7iamlsy./




Common Feature Requests

Field Weighted RelevanceAdditional Regions and LanguagesHigh AvailabilityTighter integration with other AWS services (Dynamo/S3)Support For Very Large Use CasesGeo Sorting


Field-Weighted Values


Field Weights Use Case

Music Search• Dataset composed of the following fields:

• Title• Album• Artist• Lyrics• Popularity

Results without field weights• May end up with results based heavily on lyrics when searching for an artist’s

name (Guns & Roses vs. roses, guns)Results with field weights• Possibly apply a greater weight to artist than lyrics


FWV in Rank Expressions

Rank expressions can be used within CloudSearch to customize relevance computations for better returned search results.• song_relevance = text_relevance + popularity

Natural to extend rank expressions to allow field-weighted values using JSON objects.• song_relevance = cs.text_relevance({weights: {artist=3.0,

song=4.0}, default_weight=0.5} + 0.5*popularity


Query-Time Rank Expressions

Each set of defined rank expressions may take a while to be deployed to your search domain.Query-time rank expressions would allow rank expressions to be defined during a query without having to wait• q=‘guns roses’&rank-qtre=cs.text_relevance({weights:

{artist=3.0, song=4.0}, default_weight=0.5}&return-fields=qtre&rank=-qtre


ResourcesAmazon CloudSearch Overview Pagehttp://aws.amazon.com/cloudsearch/• FAQs• Community Forum• Documentation & Getting Started Tutorial (IMDb)

Demos and Tutorials• What Is Amazon CloudSearch • Introducing Amazon CloudSearch (Features)• Building a Search Application Using Amazon CloudSearch• Getting Started Tutorial

http://aws.amazon.com/cloudsearch/

http://aws.amazon.com/cloudsearch/


Upcoming Events

Enterprise Search Summit/KMworld, DC, Oct. 17-19Bay Area Amazon CloudSearch Group: Oct. 24

Las Vegas, November 27-29


Q&A


Thank You