doxlon november 2016 - elk stack and beats
TRANSCRIPT
ELK STACK WITH BEATS
November, 2016Jon Hammant – Head of DevOps & Cloud UK/EU EPAM Systems
ABOUT EPAM
Q12016Revenue
$264.5M
CONSTANT GROWTH 4
Continents
25Countries
REVENUE BY GEOGRAPHY
NorthAmerica
Europe
APAC
CIS
58%
36%
2%
4%
20,000+Engineers,designersand
consultants
FOUNDEDIN
1993USHEADQUARTEREDPUBLICCOMPANY
(NYSE:EPAM)
SERVICE MIX
SoftwareEngineering&Product/PlatformDevelopment
QAandTestAutomation
ManagedServices
Infrastructure&Licensing
20+% YOYorganicgrowth
21 ReportedConsecutive
Quarters2016RevenueGuidance
$1.15B
Financial Services Travel&Consumer Software& Hi-tech
Media&Entertainment
Lifesciences&Healthcare
INDUSTRY FOCUS
27% 24%
14%
21%
8%
Emerging
6%
PROBLEM
Too many s y s t ems a n d n o t e n ou gh v i s i b i l i t y
Ma s s i v e l y d i s t r i b u t e d
I n c re a s i n g n umbe r o f
m i c ro s e r v i c e s
F u l l d e - c e n t ra l i z a t i o n
Pa i n f u l p ro c e s s
We need l o g g i n g & me t r i c s
WHY DO WE NEED METRICS?
B l ood l e t t i n g
S t a r t e d a ro und 1 0 0BCE
Con t i n u ed u n t i l 1 9 th C e n t u r y
Hund re d s o f T h ou s and s h ave d i e d
I t wa s d on e b e c a u s e p e op l e c a r e d
Th e y j u s t d i d n ’ t h ave r i g h t t h e i n fo rma t i o n
WHY ELK?
Ea sy t o s e t u p
Ma s s i v e l y Powe r f u l
S c a l e s v e r y we l l
Op en sou r c e
Ava i l a b l e a s a s e r v i c e
1 0 m i n u te s e t u p
• WHOWESERVESO THAT’S IT?
• WHOWESERVEWE NEED A WAY OF GETTING LOGS IN
Wedon’twanttorunSyslogeverywhere
Increasinglytheapplicationsarerunningoncloudnativesystems
Foralightweightprocesswecan’tadd
heavyweightlogging
Nopointwritingloadsofloggingcode
• WHOWESERVEWHAT ARE BEATS?
BeatsaretheElasticsearchplatformforsinglepurpose,lightweightdatashippers.
Designedtobesmall& portable
Logstashisstillimportantfordataenrichment,reformatting
ReplacesLogstashForwarder&more
• WHOWESERVECORE BEATS
F i l e b e a t P a c ke t b e a t
Me t r i c b e a t W i n l o g b e a t
• WHOWESERVEFILEBEAT
SimplestoftheBeatplugins
Thinkofitascatonsteroids
Cansendatextfiletocentralhost
ReplacesLogstashForwarder
Hasconceptofbackpressuretostopremotehostbeingoverloaded
• WHOWESERVEMETRICBEAT
Systemlevelmonitoring– CPU,Memory,filesystem,IOstatistics
Includesmodulesforcommonservices–Apache,Nginx,MongoDB,MySQL,Postgres&more
Containerready– deployonecopytomonitorallotherDockercontainers
• WHOWESERVEPACKETBEAT
NetworkPacketCapture
Understandsapplicationlayerprotocols–HTTP,DNS,ICMP,AMQP
Greatforsecurityandlatencyanalysis
Canoffer”whatwentwrong”packetflowanalysis
• WHOWESERVEWINLOGBEAT
MonitoringofWindowsLogchannels
PullWindowslogsalongwithLinuxLogs
WHEN LOGGING & METRICS WORK
“ Eve r y t h i n g we k n ow i n a v i a t i o n , e v e r y r u l e i n t h e r u l e b oo k , e v e r y p ro c e du re we h ave , we k n ow b e c au s e s omeone s omewhe re d i e d …
We h ave p u r c h a s e d a t g r e a t c o s t , l e s s o n s l i t e ra l l y b ro u gh t w i t h b l o od ”
-
" S u l l y " S u l l e n b e rg e r
COMMUNITY BEATS
EverythingbasedonGo- libbeat
Over34differentcommunitycreatedBeatsnowavailable
https://github.com/elastic/beats/blob/master/libbeat/docs/communitybeats.asciidoc
• WHOWESERVEOPENSOURCE HIGHLIGHTS
h t t p b e a t
Po l l a h t t p e n d p o i n t
my s q l b e a t
R u n a s c h e d u l e d q u e r y o n a my S q ls e r v e r
Many mo r e u s e f u l B e a t s a v a i l a b l e o r w r i t e y o u r own
C l o u d t r a i l b e a t , P i n g b e a t , C o n s u l b e a t e t c . .
e xe c b e a t
Pe r i o d i c a l l y r u n c omman d s a n d s e n d o u t p u t a n d e r r o r
• WHOWESERVEdockbeat
git clone clone https://github.com/Ingensi/dockbeat.git
wget https://github.com/Ingensi/dockbeat/releases/download/v1.0.0/dockbeat-v1.0.0-x86_64
chmod +x dockbeat-v1.0.0-x86_64
vi dockbeat/dockbeat.yml
ReplaceDocker_Socket &ElasticsearchorLogstashhost
./dockbeat-v1.0.0-x86_64 -c dockbeat/dockbeat.yml -v –e
(canalsobestartedinacontainerorswarmandpermissioned)
• WHOWESERVEEXAMPLE DASHBOARD - Metricbeat
• WHOWESERVEUSE!
D i s c o v e r
L i s t h i s t o r i c C PU u s a g eF i n d o u t w h i c h c o n t a i n e r s w e r e r u n
A n a l y z e f o r i n s e c u r e c o n t a i n e r s
Me t r i c s
S h ow r e a l t i m e m e t r i c s o f s y s t em u s eD i s p l a y b u s i n e s s v a l u e
V i ew t h e w h o l e s y s t em a t o n e
V i s u a l i z e
L o o k b a c k a t p e r f o rm a n c e s t a t sC o r r e l a t e c o s t / p e r f o rm a n c e a n d r e v e n u e
S h ow l o n g t e rm t r e n d s
A l e r t
U s e E l a s t A l e r t o n c o n t a i n e r sB e i n f o rm e d w h e n t h i n g s s t o pK n ow w h e n c a p a c i t y i s a n i s s u e
• WHOWESERVESUCCESS!
Storageischeap
Logeverythingandremovelater
Packetbeat isextremelyuseful
go-audit(auditctl)andsyslogarefantastic
Black-boxthinking,learnfrommistakes