beyond search queries
TRANSCRIPT
![Page 1: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/1.jpg)
Beyond search queriesJano Suchalsearchd.co
![Page 2: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/2.jpg)
Searchas seen by developers
{ "query": { "query_string": { "query": "elasticsearch book" } }}
return response.hits.hits
![Page 3: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/3.jpg)
Searchas experienced by users
query: elasticsarchTypo in query. No results.
query: elasticsearch Too many hits. Not relevant.
query: elasticsearch bookClick!Success! Or?
![Page 4: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/4.jpg)
Measuring search quality
![Page 5: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/5.jpg)
Cpt. Obvious: “Hits, clicks and order
do matter.”
![Page 6: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/6.jpg)
Accurately interpreting clickthrough data as implicit feedback
Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in Information retrieval, SIGIR ’05, pages 154–161, New York, NY, USA, 2005. ACM.
![Page 7: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/7.jpg)
Accurately interpreting clickthrough data as implicit feedback
![Page 8: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/8.jpg)
Search quality metrics
● Mean Average Precision @ N○ probability of target result being in top N items
● Mean Reciprocal Rank○ 1 / rank of target result
● Normalized Discounted Cumulative Gain● Expected Reciprocal Rank
![Page 9: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/9.jpg)
Search KPIs
● CTR trend
● # of queries w/o results or clicks
● # of searches per session
● Search engine latency
![Page 10: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/10.jpg)
Search quality optimization
![Page 11: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/11.jpg)
Optimizing search engines using clickthrough data
Thorsten Joachims. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’02, pages 133–142, New York, NY, USA, 2002. ACM.
![Page 12: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/12.jpg)
Optimizing search engines using clickthrough data
![Page 13: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/13.jpg)
Query chains: learning to rank from implicit feedback
Filip Radlinski and Thorsten Joachims. Query chains: learning to rank from implicit feedback. In KDD ’05: Proceeding of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pages 239–248, New York, NY, USA, 2005. ACM.
![Page 14: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/14.jpg)
Fighting Search Engine Amnesia: Reranking Repeated Results
Milad Shokouhi, Ryen W. White, Paul Bennett, and Filip Radlinski. Fighting search engine amnesia: reranking repeated results. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’13, pages 273–282, New York, NY, USA, 2013. ACM.
In this paper, we observed that the same results are often shown to users multiple times during search sessions. We showed that there are a number of effects at play, which can be leveraged to improve information retrieval performance. In particular, previously skipped results are much less likely to be clicked, and previously clicked results may or may not be re-clicked depending on other factors of the session.
![Page 15: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/15.jpg)
searchd.coSearch Analytics
![Page 16: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/16.jpg)
searchd.co dashboard
![Page 17: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/17.jpg)
![Page 18: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/18.jpg)
![Page 19: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/19.jpg)
![Page 20: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/20.jpg)
![Page 21: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/21.jpg)
A/B testing
![Page 22: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/22.jpg)
A/B testing lists
![Page 23: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/23.jpg)
A/B testing
A B
![Page 24: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/24.jpg)
A/B testing with interleaving
A B
![Page 25: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/25.jpg)
Interleaving & scoring
● Balanced● Team Draft● Probabilistic
● Binary preference● Linear rank difference● Inverse rank difference
![Page 26: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/26.jpg)
A/B testing with interleaving
![Page 27: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/27.jpg)
A/B testing with interleaving
![Page 28: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/28.jpg)
A/B testing with interleaving
● Pros○ Lower risk of loosing conversions
● Cons○ Harder to interpret○ Harder to implement
![Page 29: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/29.jpg)
searchd.coSearch Analytics
● Identify and fix key search problems ● KPIs for site search● Actionable tips for search tuning● Safe A/B testing
● Easy setup
● In Beta, sending out invites
![Page 30: Beyond search queries](https://reader034.vdocuments.us/reader034/viewer/2022052400/55a20a231a28aba0368b4632/html5/thumbnails/30.jpg)
Bad search experience is a lost opportunity. Let's fix it.
searchd.coSearch Analytics