indexing and searching logs with elasticsearch/solr by radu gheorghe from sematext
DESCRIPTION
This presentation shows how Logstash, Apache Flume or rsyslog can help you parse, buffer and ship your logs to Elasticsearch or Solr. We’ll show the pros and cons of each tool, so you can decide what’s best for your use case. The presentation also puts Elasticsearch and Solr side by side ands shows how to make them handle write-heavy applications like those presented in the first part. It goes over how to tweak various buffers, merge policies, how to structure indices/collections, and more. And when one machine isn’t enough, we’ll look at how Elasticsearch and Solr (through SolrCloud) can be scaled horizontally to handle more and more data.TRANSCRIPT
![Page 1: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/1.jpg)
![Page 3: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/3.jpg)
http://eddysuaib.com/wp-content/uploads/2012/12/Keyword-icon.pnghttp://www.clker.com/cliparts/a/f/7/9/12279738201383389510Farmeral_Wood_Icon.svg.hi.pnghttp://www.honduply.com/images/image/plywood.gifhttp://designingagainstthegrain.com/wp-content/uploads/2012/10/Boards-Ready-to-Dry.jpg
![Page 5: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/5.jpg)
![Page 6: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/6.jpg)
![Page 7: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/7.jpg)
![Page 8: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/8.jpg)
![Page 9: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/9.jpg)
http://www.gollihurmusic.com/images/gspeakon1.gifhttp://www.tfw2005.com/transformers-news/attach/7/3/8/6/3/Transformers-3-Dark-of-the-Moon-China-2_1304957777.gi
![Page 10: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/10.jpg)
https://flume.apache.org/_images/DevGuide_image00.png
![Page 11: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/11.jpg)
![Page 12: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/12.jpg)
Source
Channel
HDFS
Solr
Channel
![Page 13: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/13.jpg)
http://4vector.com/i/free-vector-james-bond-secret-agent-007-black-white-silo_131156_jamesBond.png
![Page 14: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/14.jpg)
![Page 15: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/15.jpg)
![Page 16: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/16.jpg)
http://www.chowdownchamps.com/wp-content/uploads/2013/06/Bud-light.pnghttp://librocket.com/chrome/site/images/rocket-home.gif
![Page 17: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/17.jpg)
http://4vector.com/i/free-vector-thread-rolls-clip-art_108874_Thread_Rolls_clip_art_hight.pnghttp://www.ecofirewood.co.uk/imagesProducts/large/%7B06C86DD3-E258-4CB7-9DD9-D61041557208%7D.jpg
![Page 18: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/18.jpg)
http://blog.trifork.com/wp-content/uploads/2013/04/elasticsearch-logo.pnghttps://lucene.apache.org/images/solr.png
![Page 19: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/19.jpg)
![Page 20: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/20.jpg)
![Page 21: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/21.jpg)
![Page 22: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/22.jpg)
![Page 23: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/23.jpg)
![Page 24: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/24.jpg)
![Page 25: Indexing and Searching Logs with Elasticsearch/Solr by Radu Gheorghe from Sematext](https://reader034.vdocuments.us/reader034/viewer/2022052522/554f45a7b4c905524c8b45b0/html5/thumbnails/25.jpg)
http://www.staff.amu.edu.pl/~zbzw/glob/glob.gif