ELASTICSEARCH

The “new school” search engine

About me - @nignatov

• Founder at NodeSpark

• Using PHP since 2006

• Using Drupal since 2009

• Elasticsearch Connector maintainer

• Love to experiment with new technologies

After the session you should know:

• What is Elasticsearch • How Elasticsearch is working in general • What you can do with Drupal and

Elasticsearch. • Roadmap of the project and some

brain food.

The main goal

GET INLOVE! GET INVOLVED!

Shay Banon • http://thedudeabides.com/ • https://github.com/kimchy • https://twitter.com/kimchy/

Who made Elasticsearch?

Why “new school”?

Who else is using it?

Who is using it?

Visit Elastic{ON} to get more

• Distributed out of the box - Incredible easy to make cluster, just start new node!

• High availability

• Real time index/search • REST API with JSON • Build on top of Apache Lucene and Open Source

http://github.com/elasticsearch/elasticsearch • Schema free • Document oriented

What is Elasticsearch?

Vertical and horizontal scaling

S01

S01

S01

S01 S02 S03 S04+

The feature I really love

How to install it?

How to install it?

How to install it?

… or Found.no

https://found.no

Some terms before we start

RELATIONAL DB ELASTICSEARCH

Database Index

Tables Types

Rows Documents

Column Field

Indices API – Create index

index

How distributed works?

Node2

Node 1

PUT /twitter index: number_of_shards:3 number_of_replicas:1

1P 2P

3P

2P 1R

3R

2R

Node3

3P 1R3R

2R3R

Node2

Multiple indices

Node 1

1P 2P2R

Node3

3P 1R3R

PUT /news?pretty index: number_of_shards:3 number_of_replicas:1

1NP 2NR 2NP 3NR 3NP 1NR

Node2

Node 1

1P 2P2R

Node3

3P 1R3R

PUT /news?pretty index: number_of_shards:3 number_of_replicas:1

1NP 2NR 2NP 3NR 3NP 1NR

index type id

Index API - Request

Node2

Node 1

PUT /twitter/tweet/1{….}

1P 2P2R

Node3

3P 1R3R

Distributed indexing

Index API - Response

Get API - Req and Res

Node2

Node 1

GET /twitter/tweet/1

1P 2P2R

Node3

3P 1R3R

Distributed GET

PUT the document again or…

Update document

Delete document

Search API - Request

Node2

Node 1

GET /twitter/_search?q=vienna

1P 2P2R

Node3

3P 1R3R

Distributed Search

Delete index

How full text search works?

Search for: “Drupal Camp Vienna has been great!”

1. Separate the string into words 2. Create sorted unique list 3. Specify which documents contains this terms

TERMS DOC 1 DOC 2 DOC 3

drupal X X

camp X X

vienna X X

Has X X

been X X

great X

Inverted index

Tokenization + Normalizationor tokenizer + token filters (>=0)

Analysis

DSL Query language Aggregation API (Facets on steroids) AliasingPercolatingGEO searching Attachments many more

More features

Elasticsearch and Drupal

Drupal 7 EcoSystem

1. Elasticsearch Connector 2. EC Search API 3. EC Views 4. EC Statistics 5. EC Watchdog 6. EC Sequence 7. EC Devel

Drupal 7 Watchdog

Drupal 7 Search API

Drupal 8… still patchable!

Roadmap

• Drupal Abstraction Layer • Vagrant, Travis-CI, and automated

tests • Search API integration • Stand alone views module • Statistics / Personalization • Automatic the “Easy install” library

packing.

Brainstorm – Content hubSite X

Site Y

Site Z

Content Hub

Feed Site

Brainstorm – Headless

NodeJS

Data Warehouse

Drupal CMS

Frontend

Brainstorm – SLA HubSite 01

Site 02

Site 03

Site 04

Site 05

Support Department Monitoring

Brainstorm - PerformanceWeb 01

Web 02

Web 03

Web 04

Web 05

Special thanks!

Thank you!

Questions ?