1. Finding anything: Real-time search with IndexTank

• Tim Spence April 19, 2011

2. About the Presenter Tim Spence

Senior Infrastructure Engineer at MedHelp (http://www.medhelp.org/)

3. Former .NET developer 4. Recently converted to Ruby 5. In love with Open Source Software 6. More athttp://whyhello.im/tim 7. Agenda

State of search today

8. Quick survey: how much time/effort did YOU spend implementing search on your webapp? 9. Examples of services that need improved search 10. IndexTank to the rescue 11. Case study:reddit.com 12. Agenda, continued

How I found out about IndexTank

13. Two apps I built with IndexTank 14. Live Demo 15. 16. The State of Search Today

Not well implemented at all

Search works, but...

17. Barely How many pages of results do you typically browse through before finding what you were looking for? 18. Or do you give up and head for google site search instead? 19. Survey Time!

How much time/effort did YOU spend implementing search on your webapp?

20. How many times have you iterated on your search feature? 21. When was the last time someone thanked you for building a powerful, reliable search feature for your webapp? 22. My Opinion

Search as an in-app feature is an afterthought

23. Minimal implementation is the norm 24. If it wasn't for MySQL/MS-SQL full text indexing, most apps probably wouldn't even have a search feature 25. Most good web apps don't make it easy for users to find specific content outside of predetermined navigation 26. Let's pick on some apps!

These are companies with great products, but their search comes up short

27. Don't worrythey can take it! 28. App #1: Github 29. App #1: Github 30. App #1: Github

Interface is decent

Search repos, code, users, or everything

31. Search by language However...

Can't do much with results but browse

32. Check out this example 33. App #1: Github 34. App #1: Github

Why these results aren't so hot

Can't search by most recently maintained

35. Can't search by most popular (most watched) 36. Are you ready to browse 1,297 results? Advanced search capabilities exist, but not the best interface

recency/popularity implemented, but require specific arguments

37. App #2: Amazon Web Services

Hey, I bet I can find an AMI from the community for the exact EC2 setup I need

38. Fact: probably not 39. App #2: Amazon Web Services 40. App #2: Amazon Web Services

Notice something missing?

No search

41. Only sort by date, title Ready to browse 934 results?

I'd rather build my own AMI

Incredible missed opportunity

o/s search

42. Stack search 43. etc... 44. Fact: Github & Amazon aren't the only ones

Lots of good web services

45. Massive quantities of quality content 46. Unfortunately not discoverable in meaningful ways 47. Interlude: Sites with great search

Foodspotting

Proximity

48. Recency 49. Rating Medhelp

Content category

50. Promoted content Other sites I overlooked?Whose search do you like? 51. What was the point of that last slide?

Search can be useful if it is valued as a feature

52. Any company willing to invest in the resources can build and host a high quality search engine 53. However, must you roll your own? 54. Enter Search as a Service

No need for you to invest in additional infrastructure

55. No need to reinvent the wheel

Search is a solved problem

56. Let the experts refine it 57. IndexTank to the rescue!

Hostedno load on your infrastructure

58. Powerful

We'll get into the details next

Always Improving

Search IS their product

Freemium 59. Easy to implement 60. Let's talk features

Real-time search

Real-time indexingresults immediately available

Custom scoring 61. Autocomplete 62. Faceting 63. Geo search 64. Advanced text search 65.

Real-time search

Real-time indexing

results immediately available

Index multiple docs/sec 66. Overwrite existing docs as you wish

Changes also immediately available

67. Custom Scoring

Implementer has full control over how results are returned

68. Choose which fields are searched 69. Use pre-written scoring functions 70. Or write your own 71. Custom Scoring 72. Everyone loves autocomplete

Saves users time

73. Potentially avoids spelling errors

Not for hunters/peckers

Adds a degree of intelligence to the search process 74. Faceting

Does it make sense for you to categorize documents in your index?

In all cases, YES

Consider your advanced users and the narrow results they seek

Don't make anyone sift through irrelevant results

75. Faceting 76. Geo

It's 2011

Location is more relevant than ever before

77. Mobile is skyrocketingevery client has a GPS IndexTank has built-in geo proximity search capability 78. Geo 79. Advanced Text Search (Beta)

Fuzzy search (Did you mean...?)

80. Stemming

Alternate word forms (tense, possession, etc...)

Alternate spellings

Misspellings

81. Other Benefits

Zero maintenance

82. Scalability included for free 83. Easy implementation

Clients available in many languages

84. Excellent documentationLet's check it out Excellent support

Humans or bots?You decide

Dog food: their site search is done well 85. 86. Case Study:reddit.com

High traffic news aggregator (> 1.0E9 pvs/mo) with tons of content

87. Who remembers how bad reddit's search was?

When it even worked

Can't blame them for trying

Many attempts, but none worked

IndexTank excelled in all areas 88. Let's check it out now 89. My experience with IndexTank

Discovered through Heroku/IndexTank contest

90. Built my first irl Rails app in an afternoon/evening w/ fellow hacker Chris Saylor (@cwsaylor) 91. Didn't win the contest but learned how easy it is to quickly create highly targeted search 92. App #1: Toxosis

Searchable database of toxic release data supplied by U.S. E.P.A.

93. Hosted athttp://toxosis.heroku.com/ 94. Search enabled on many fields including city/state/zip, toxin 95. Additional fields can be added to index

When I have time, of course...

96. More personal backstory

Still in the business of reinventing myself as a Rails developer

97. How to get a Rails gig?Developan appmultiple Rails apps and showitthem off 98. Opportunities are everywherecontests, hackathons, and weekend hacks for developer community 99. App #2: SXSWdex

Searchable database of 2011 SXSW attendees

100. Hosted athttp://sxswdex.heroku.com/ 101. Design goal: do a better job than SXSW official site 102. Search within bio, company, location, name 103. Facets: company, city/state 104. The moment we've all been waiting for

Let's build an app!

105. Questions?

Q&A time with an IndexTank engineer