how to run solr on docker and why

O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A

How to run Solr on Docker. And why.Rafał Kuć, Radu Gheorghe

Sematext Group, Inc.

3

01Next ~40 minutes

4

02Common issues we all know

overprovisionedserversresources not utilized

!= != !=

differences in environments

$$$money loss

5

02One of the solutions

productionQAtestdevelopment

6

03Bare metal

Hardware

Operating System

Libraries

Application Application

bare metal

7

03Traditional virtual machine

Hardware

Operating System

Libraries

bare metal

Hardware

Host Operating System

Hypervisor

Guest OS Guest OS

Libraries Libraries



traditional VM

8

03Docker container

Hardware

Operating System

Libraries

bare metal

Hardware


traditional VM

Hypervisor

Guest OS Guest OS

Libraries Libraries


Hardware


Docker Engine

Libraries Libraries



container

9

03Unikernel architecture

Hardware

Operating System

Libraries

bare metal

Hardware


traditional VM

Hypervisor

Guest OS Guest OS

Libraries Libraries


Hardware


container

Docker Engine

Libraries Libraries


Hardware

unikernel

Kernel + App + Libs Application Application


Hypervisor

10

01Solr + Docker how to

+

11

01Docker + Solr – how to

docker run solr:latestdocker run -p 8983:8983 solr:latestdocker run --name lr_single_solr -d -p 8983:8983 -t solr:6.2.1docker run --name lr_alpine_solr -d -p 8983:8983 -t solr:6.2-alpinedocker run -it --name lr_solr_mem -p 8983:8983 -d -m 1G solr:6.2.1docker run -p 8983:8983 -d --memory-swappiness=0 solr:6.2.1docker run -it --cpuset-cpus="0,1" -p 8983:8983 -d solr:6.2.1docker run --name solr_one -d -p 8983:8983 –p 9983:9983 -t solr:6.2.1 -cdocker run --name solr_two -d --link solr_one -p 6983:6983 -t solr:6.2.1 -z solr_one:9983

12

01Demo time!

https://github.com/sematext/lucene-revolution-samples/tree/master/2016/solr_docker

DEMOTIME

13

01

14

01Very simple test infrastructure

Ubuntu + Solr 6.2Ubuntu + Docker 1.12.2 (native)

VMs (Ubuntu) + Solr 6.2

15

01Solr configuration – collection

16

01Solr configuration

10 stored fields Hard commit – 15 sec10 indexed fields Soft commit - 1 sec7 doc values

17

01Other parts

Completely out of the box experience!

18

01Ubuntu + Solr, 1 Solr instance

Test 1Indexing throughput - 3348 docs/sec


Querying throughput - 209 q/secSprzedaż

CPU Mem I/O

19

01Ubuntu + Solr, 4 Solr instances

Test 1Indexing throughput - 3618docs/sec


Querying throughput - 267,7 q/secSprzedaż

CPU Mem I/O

20

01Ubuntu + 1 VM, 1 Solr instance




CPU Mem I/O

21

01Ubuntu + Docker, 4 Solr instances




CPU Mem I/O

22

01Performance summary

Test results on OSX != Test results on Linux

Bare metal is the fastest from those tested

Docker will be faster than traditional VM

Bare metal > Docker > VMs

23

01

24

01Bare metal based architecture

One per machine vs Multiple per machineMultiple per machine may be a better choiceHarder to scale -> machine provisioning neededFull control over resourcesNoisy neighbour problems can appearBest or almost best performance

25

01Virtual machine based architecture

Multiple VMs per machine allowedEasy to scale, but requires preparationClear boundaries between VMGuest operating system overheadWorst performance compared to the rest

26

01Docker based architecture

Multiple per host easily possibleEasy to scale until host resources allowEasily add constraints per containerLow overhead compared to traditional VMNative on Linux == best possible performance

27

01

http://cliparts.co/clipart/3733708

28

01Should I care about Solr?

Tuning is still needed, no matter what ENV you use

Keep in mind ENV constraintsWatch out for:

- merges – adjust or split data- memory issues- I/O subsystem- CPU utilization- garbage collection- proper Solr configuration

29

01Should I care about Docker?Docker reports eventsWe can listen and react to eventsExample of events to react to

- kill- start- stop- destroy- die- restart - pause- unpause- oom

30

01Should I care about Docker volumes?

Docker volumes also report eventsWe can listen and react to those as wellWe can automatically react to:

- create- mount- unmount- destroy

31

01Solr + Docker, should I?

Yes, when in need of dynamic scalingYes, when in need of resources utilizationLots of nodes

-> look at swarm, mesos and kubernetes

32

01Thank you! If you want to get in touch

Rafał[email protected]@ kucrafal

[email protected]@ radu0gheorghe

http://sematext.com@ sematext

Join US!

http://sematext.com/jobs/

how to run solr on docker and why

Technology