monitora le performance della tua applicazione python ... · monitora le performance della tua...
TRANSCRIPT
![Page 1: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/1.jpg)
Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana
PyCon Nove Florence20/04/2018
![Page 2: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/2.jpg)
About me
MATTEO ZUCCON
@matteo_zuccon
https://whiletrue.run
Solution Developer |
#PyconNove #PyconIT @pyconit
![Page 3: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/3.jpg)
Agenda
• What is performance? Definition and meaning
• APM - Application Performance Management
• Open Source APM with and
• Monitor Flask Application
![Page 4: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/4.jpg)
What is performance?
“total effectiveness of an application or computer system, including throughput ,
individual response time, and availability”
Something is successful in producing a desired result; success
![Page 5: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/5.jpg)
What is performance? Monitoring
• Application Avg Response Time
• Application Load
• Monitoring CPU of servers
• Tracking and log errors
• Everything useful for you – depending from your use case and application
![Page 6: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/6.jpg)
What is performance? Management and Monitoring
Performance matters!
NOT ONLY
monitoring
ALSO
management
![Page 7: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/7.jpg)
APM - Application Performance Management
“In the fields of information technology and systems management, application performance management (APM) is
the monitoring and management of performance and availability of software applications.”
“the translation of IT metrics into business meaning”
![Page 8: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/8.jpg)
Application performance managementor
application performance monitoring.
APM - Management and Monitoring
![Page 9: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/9.jpg)
APM - Management and Monitoring
being more proactive
only being reactive when it comes to the performance
Application performance managementor
application performance monitoring.
![Page 10: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/10.jpg)
APM - It seems easy
…monitoring and management of performance
![Page 11: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/11.jpg)
APM – Components – 1 of 2
• Performance of individual web requests or transactions
• Usage and performance of all application dependencies like databases, web services, caching, etc
• Detailed transaction traces down to specific lines of code
• Code level performance profiling
• Basic server metrics like CPU, memory, etc
![Page 12: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/12.jpg)
APM – Components – 2 of 2
• Evaluate performance management criteria
• Application aware and focused – understand application context
• Custom applications metrics created by the dev team or business
• Application log data and errors
• Real time monitoring
![Page 13: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/13.jpg)
APM – Tools
• New Relic
• Glimpse
• Miniprofiler
• XRebel
• Stackify Prefix
• App Dynamics
• Scout Devtrace
• Rack trace
• Zend Z-Ray
![Page 14: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/14.jpg)
APM – Tools – Cloud Providers
• X-Ray e Pinpoint
• Application Insights
• APM
![Page 15: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/15.jpg)
APM – Literature
![Page 16: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/16.jpg)
APM – Practices
![Page 17: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/17.jpg)
APM – Not that easy- Where do I start?
A lot of components, tools, books, practices….
![Page 18: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/18.jpg)
Elasticsearch and Kibana
Elasticsearch is a distributed, RESTful search and analytics engine, heart of the Elastic Stack, it centrally stores your data.
Kibana lets you visualize your Elasticsearch data and navigate the Elastic Stack
![Page 19: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/19.jpg)
Open Source APM - Elastic
Already housing logs and system metrics in Elasticsearch?
Expand to application metrics with Elastic APM.
Four lines of code lets you see a bigger picture to quickly fix issues and feel good about the code you push.
(basic subscription – Free license)
Elasticsearch and doubling down on “open”: https://changelog.com/podcast/292
![Page 20: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/20.jpg)
Elastic APM - Components
• Elasticsearch
• APM Server
• APM Agents
• Kibana APM UI
![Page 21: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/21.jpg)
APM – Components - Elasticsearch
•Elasticsearch
• APM Server
• APM Agents
• Kibana APM UI
Centrally stores the application metrics
![Page 22: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/22.jpg)
APM – Components – APM Server
• Elasticsearch
•APM Server
• APM Agents
• Kibana APM UI
Open source application written in Go which runs on your servers. It receives data from agents periodically.
The API is a simple JSON based HTTP API.
APM Server builds Elasticsearch documents from the data received from agents.
![Page 23: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/23.jpg)
APM – Components – APM Agents
• Elasticsearch
• APM Server
•APM Agents
• Kibana APM UI
Hook into your application and start collecting performance metrics and errors automatically.
Collected data is buffered for a short period and sent on to APM Server.
![Page 24: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/24.jpg)
APM – Components - Kibana
• Elasticsearch
• APM Server
• APM Agents
•Kibana APM UI dashboard to visualizethe APM metrics
![Page 25: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/25.jpg)
APM – Components Schema
![Page 26: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/26.jpg)
APM - Agents
• Python*
• NodeJS
• Ruby
• Java
• Go
• JavaScript
Version 1.8 – 2.0
Version 0.10+
* no support async frameworks like Twisted or Tornado
![Page 27: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/27.jpg)
Elastic APM - Pros
• Open Source, free and easy to use
• APM Agents
• Fully integrated with Elasticsearch
• Built in Kibana Dashboard
![Page 28: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/28.jpg)
APM – Python Flask - Configuration
pip install elastic-apm[flask]
For apm-server 6.2+, make sure you use version 2.0 or higher of elastic-apm
![Page 29: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/29.jpg)
APM – Python Flask - Routes
Automatically track transactionsand capture uncaught exceptions within Flask.
![Page 30: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/30.jpg)
APM – Python Flask - Dashboard
![Page 31: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/31.jpg)
APM – Python Flask - Dashboard
List of requests with details
![Page 32: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/32.jpg)
APM – Python Flask - Dashboard
Requests impact
![Page 33: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/33.jpg)
APM – Python Flask - Dashboard
Response Times
![Page 34: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/34.jpg)
APM – Python Flask - Dashboard
Request per minute by HTTP Code
![Page 35: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/35.jpg)
APM – Elasticsearch Document
![Page 36: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/36.jpg)
APM – Python Flask – Arbitrary Exception
Send additional events by raising an exception or logging a generic message
![Page 37: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/37.jpg)
APM –Extra Data and ignore specific routes
Send additional data Ignore specific route
![Page 38: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/38.jpg)
APM – Instrumenting custom code
Decorator or context manager Transaction sample
![Page 39: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/39.jpg)
APM – Instrumenting custom code
Decorator or context manager Transaction sample
Transaction Sample Timeline
![Page 40: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/40.jpg)
APM – Instrumenting custom code
Response Time Distribution
![Page 41: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/41.jpg)
APM – Sanitizing data
Add processors
Custom Processor
![Page 42: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/42.jpg)
APM –Tuning and Overhead considerations
• CPU time
• Memory
• Bandwidth use
• Elasticsearch storage
• Transaction Sample Rate:randomly sample only a subset of transactions
• Transaction Queue:reduce queue size and flush time
• Spans per transaction:transactions should have well below 100 spans
![Page 43: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/43.jpg)
APM – Elastic - Python
• Agents ready to go (Flask and Django)
• Few lines of code
• Custom exceptions and messages
• Transactions and requests
• Exclude route and sanitize data
• APM Server public API
![Page 44: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/44.jpg)
APM – Elastic - Python
Thank youQuestions?
@matteo_zuccon
https://whiletrue.run
#PyconNove #PyconIT @pyconit
https://bit.ly/2JYGZtZ
![Page 45: Monitora le performance della tua applicazione Python ... · Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana 20/04/2018 PyCon Nove Florence](https://reader036.vdocuments.us/reader036/viewer/2022062601/5d67246f88c99340518b483c/html5/thumbnails/45.jpg)