JustlogPuGong

Induction trainingofDevTeam

Agenda

• What’sLog

• Whatshouldbelogged

• Howtousethelog• Examplesof Log System

• Q&A

What’sLog

• arecordofajourneymadebyashiporaircraft,detailingallevents,or

thebookinwhichitiskept

• (Computers)Anyofvariouschronologicalrecordsmadeconcerning

theuseofacomputersystem,thechangesmadetodata,etc.

• The logsareoftenmet

• Transaction Log�Binlog

• Operation Log• Application Log

Aretheylog

•&���� ������*�� ,�

• Black-box

Keypointoflog

• Timestamp

• Sequence• Meaningful

• Formatofrecords

• Contents• Immutable

• StructuredvsUnstructured

WhyLogareimportant

• Complianceandregulations:Provideanaudittrailofwho,what,

where,whenandwhy

• Situationalawareness• Incidentreponse• Realtimealerts

Operationlog

• Purpose• Keepthetrackofwhatuserhaddone• ForAUDIT• ForTrackofrecordchange

• Keyelements

• When - Timestamp

• Who- User

• What- whatwasdid

• Where- IP/Host

• Identifier- Table(moudle)Name,record_id

SampleofOperationLog

Applicationlog

• Purpose• Keepnecessaryapplicationrunninginformation

• Foronlineproblemanalysis

• Fordebug• Keyelements

• When- Timestamp

• What

• LogLevel

• (Error)Message

• Stacktrace

• Where– Host/IP

• Secure– removesensitiveinformation

• Centralize

LogLevel

• Debug:Usedonlyfordevelopmentandtesting.Temporaryopenon

productiontofindmoreinformation.(Cautionwiththelogsize)

• Information:Usedtokeeptheinformationthatisusefulforsystem

runningandmanagement. Theentryandexitpointsofkeyfunctions

shouldbekeptinthislevel.

• Warning:Usedtokeepthehandledexceptionsorotherimportantlogevents.

• Error:Usedtokeeptheunhandledexceptions• Fatal:Reservedforspecialexceptions/conditionsthatneedtobetakencareof.

SampleofApplicationLog

Howtolog- Metriclog

• Purpose• KeepApplicationrunningstat,mainlynumbersaboutbusiness

• Monitor

• Alert• Keyelement

• When– Timestamp

• Who– AppIdentifier

• Where– Host/IP/Tags

• What- Metrics

SampleofMetricLog

Howtolog- TraceLog

• Purpose• AnuniqueIdtolinkthelogsindifferentapplication

• Generatedattheverybeginningattherequest

• Saveineverylogsasafieldoratag

• Onlineproblemanalysis

• Userbehaviortracking• KeyElements

• What– uniquetracke Idinotherlog

• Others– almostthesameas

Howtousethelogs

• Metricsformonitorandalert

• Wherealertsrings,gotoapplicationlogfordetailinformation

• Usetracetofindassociationlogsinotherappisnecessary

• Prediction

Howtouselogs

Collect

Alert

Store

Search,

Report,

Analytics

Make

conclusions

Act

Humansneeded!

Files,syslog,etc

SMS,E-mail,etc

Immutablelogs

LogSystem

• ELK– Metrics,applicationlogetc

• Statsd+Grafana /statsd +graphite– Metrics

• Splunk – commercial

• Customized

ELK

Watcher Shield

Splunk

StatSD +Grafana

ACustimized Logsystemarchitecture

Logagent

SumUp

• Carefulchooseloglevel• Centralizethelogs• Securethelogs

• DoLog• DoUsethelog:

• Monitor&Alert

• Analysisthelogs

Reference

• TheLog:Whateverysoftwareengineershouldknowaboutreal-timedata's

unifyingabstraction

• ���!�(��$�+�'#)"������"%���• LogEverythingAllTheTime

• http://play.grafana.org/• ElasticSearch,Logstash&Kibana• Splunk:http://www.splunk.com/

• Zabbix:https://www.zabbix.com/

• Cacti:http://cacti.net/• nagios:https://www.nagios.org/