sarine's big data journey by rostislav aaronov
TRANSCRIPT
![Page 1: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/1.jpg)
Shaping diamonds with ElasticRostislav Aronov, System Architect, Sarine
![Page 2: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/2.jpg)
• Rostislav Aronov• System Architect @ Sarine• More then 25 years in SW industry• From coder via DBA, IT, DevOps to architect• Currently leading key projects at Sarine -
• Cloud connected planning application• Sarine Profile • Others
About Me
On Time, On quality, On budget!
![Page 3: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/3.jpg)
• Public company – Singapore exchange• Market share at manufacturing – more than 80%• Millions of the world’s diamonds are touched by Sarine
technology• CUT grading – ALL major labs – NGTC, GIA, AGS, HRD,
IGI…• Number of employees – 600 World Wide
• Video - https://youtu.be/0PL9BtO-xl4
About Sarine Technologies
![Page 4: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/4.jpg)
About Sarine Technologies
![Page 5: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/5.jpg)
About Sarine Technologies
Mining & Rough Trade Manufacturing Gemological
Reporting Polish Trade Retail
![Page 6: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/6.jpg)
![Page 7: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/7.jpg)
![Page 8: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/8.jpg)
About Sarine Technologies
Perfect StoneHigh Clarity, Colour and Cut
grades Low Clarity Low Colour Low Cut
Sarine ProfileTM
![Page 9: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/9.jpg)
About Sarine Technologies
Sarine Planning application
![Page 10: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/10.jpg)
• Stand-alone (disconnected) desktop application• No usage statistics• No customer feedback• Support done by screen sharing• Application/Client Security only• Applanders - Escaping Appland!
About Sarine Technologies
![Page 11: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/11.jpg)
Sarine – Cloud connected application
Work station (client)
Sarine Application
Back End Service for Data processing
Data StoreSQL-RDS/
DynamoDB/S3
Elasticsearch
LogstashKibana
Lambda Statistic collector
![Page 12: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/12.jpg)
• Application centric logs – requests and results statistic data• .Net Services activity logs – log4net info and error level logs• Client sense logs – requests round trip statistics• Lambda activity logs – info and errors
Sarine – Cloud connected application
![Page 13: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/13.jpg)
• CEO – Now this is 21st century!• PM – We have valuable feedback• CC – It is easy to help our customers• DevOps – We know how to monitor and maintain the production
system• Dev – We have cool technology• And more…
Elastic and Sarine - Business Value
![Page 14: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/14.jpg)
• Elasticsearch cluster with 3 nodes• 125 indices (daily, weekly, monthly)• 268 shards• 400 millions docs• 1 terabyte data• And counting…
Elastic – How we use it?
![Page 15: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/15.jpg)
• Shield• Marvel• Watcher• Head• Kopf• CB – XDCR• Cloud – AWS for S3 backups• Curator
Elastic – How we use it?
![Page 16: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/16.jpg)
• DataZone• POC• Architecture and Sizing• First implementation• Team education
• Elastic subscription• Fine tuning and best practices recommendation• Production monitoring, 24/7 support• Team education
Partners for our Big Data Journey
![Page 17: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/17.jpg)
• More Sarine projects on elastic• Sarine Profile
• Main data store – Couchbase• Main log/history system• Main search system• Main Data visualization system
• Automatic tests• Results collection• Data visualization, reports
• Main monitoring tool• Alerts based on watchers• Main dashboard
Elastic – Is a Strategic Partner
![Page 18: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/18.jpg)
• Use elastic for real-time data collection, analysis and visualization• Use elastic for super fast search as a part of web and desktop
application• Plan index structure (shards, periodic, size)• Plan the cluster size, use at least 3 nodes to avoid split brain• Design index mapping carefully. Use templates, analyzers
according to your needs. Default setting are good enough for general case only!• Educate you team, it is new and not intuitive for most people
technology• Use partners and consulting at least at the begging• Re-indexing is expensive. Avoid it.
Elastic – Recommendations
![Page 19: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/19.jpg)
Q & A
Sarine - Shaping diamonds with Elastic
![Page 20: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/20.jpg)
• Careers at Sarine• Client Ninja / Developer for Presentation Team• Full stack Developers for Cloud Services & Presentation
Teams• Test Engineer to lead QA in Cloud Services team
Sarine - Shaping diamonds with Elastic
![Page 21: Sarine's Big Data Journey by Rostislav Aaronov](https://reader035.vdocuments.us/reader035/viewer/2022081604/586fd91a1a28ab18428b5837/html5/thumbnails/21.jpg)
Thank You!