machine data 101

Copyright©2014SplunkInc.

MachineData101

GaryBurgettSr.SE

11/1/2016

WhatDoesMachineDataLookLike?Sources

OrderProcessing

Twitter

CareIVR

MiddlewareError

2

MachineDataContainsCriticalInsightsCustomerID OrderID

Customer’sTweet

TimeWaitingOnHold

TwitterID

ProductID

Company’sTwitterID

CustomerIDOrderID

CustomerID

Sources

OrderProcessing

Twitter

CareIVR

MiddlewareError

3

MachineDataContainsCriticalInsightsOrderID

Customer’sTweet

TimeWaitingOnHold

ProductID

Company’sTwitterID

OrderID

CustomerID

TwitterID

CustomerID

CustomerID

Sources

OrderProcessing

Twitter

CareIVR

MiddlewareError

4

StructuredRDBMS

SQL Search

SchemaatWrite SchemaatRead

Traditional Splunk

SplunkApproachtoMachineData

Copyright © 2014 Splunk Inc. 5

ETL UniversalIndexing

Volume Velocity Variety

Unstructured

Splunk:ThePlatformforMachineData

6

DeveloperPlatform

Reportand

analyze

Customdashboards

Monitorandalert

Adhocsearch

OnlineServices

WebProxy

DataLossPrevention

Storage Desktops

PackagedApplications

CustomApplications

Databases

CallDetailRecords

SmartphonesandDevices

FirewallAuthentication

Fileservers

Endpoint

ThreatIntelligence

Asset&CMDB

Employee/HRInfo

DataStoresApplications

ExternalLookups

Badgingrecords

Emailservers

VPN

Anyamount,anylocation,anysource

Schema-on-the-fly

Universalindexing

Noback-endRDBMS

Noneedtofilterdata

PlatformforOperationalIntelligence

TheSplunkPortfolio

RichEcosystemofApps&Add-Ons

SplunkPremiumSolutions

MainframeData

RelationalDatabases

MobileForwarders Syslog/TCP IoTDevices

NetworkWireData

Hadoop

Agenda

§ Non-TraditionalDataSources

§ DataEnrichment

§ LevelUponSearchandReportingCommands

§ DataModelsandPivot

§ AdvancedVisualizationsandtheWebFramework

8

WorkshopSetup

Non-TraditionalDataSources

Non-TraditionalDataSources

§ NetworkInputs

§ HTTPEventCollector

§ LogEventAlertAction

§ SplunkAppforStream

§ ScriptedInputs

§ DatabaseInputs

§ SplunkODBCDriver

§ ModularInputs

§ zLinux Forwarder

§ MINT

§ Non-SplunkDatastores

11

TraditionalDataSources§ Captureseventsfromlogfilesinrealtime

§ Runsscriptstogathersystemmetrics,connecttoAPIsanddatabases

§ Listenstosyslog andgathersWindowsevents

§ Universallyindexesanydataformatsoitdoesn’tneedadapters

12

Windows• Registry• Eventlogs• Filesystem• sysinternals

Linux/Unix• Configurations• Syslog• Filesystem• Ps,iostat,top

Virtualization• Hypervisor• GuestOS• GuestApps

Applications• Weblogs• Log4J,JMS,JMX• .NETevents• Codeandscripts

Databases• Configurations• Audit/querylogs• Tables• Schemas

Network• Configurations• syslog• SNMP• netflow

NetworkInputs

§ CollectdataoveranyUDPorTCPport§ Somedevicesonlysenddataoveranetworkport

§ BestPractice:usesyslog-ng orrsyslog§ Offerspersistence§ Categorizesdatabyhost

13

HTTPEventCollector(HEC)

§ CollectdataoverHTTPorHTTPSdirectlytoSplunk§ ApplicationDeveloperfocus– fewlinesofcodeinapp

tosenddata§ HECFeaturesInclude:

§ Token-based,notcredentialbased§ IndexerAcknowledgements– guaranteesdataindexing§ RawandJSONformattedeventpayloads§ SSL,CORS(CrossOrigion access),andNetworkRestrictions

14

LogEventAlertAction

§ UseSplunkalertingtoindexacustomlogevent§ Splunksearchableindexofcustomalertevents

§ ConfigurableFeaturesInclude:§ Host§ Source§ Sourcetype§ Index§ Eventtext– constructtheexactsyntaxofthelogevent,

includinganytext,tokens,orotherinformation

15

TheSplunkAppforStream

WireDataEnhancesthePlatformforOperationalIntelligence

Efficient,Cloud-readyWireDataCollection

SimpleDeploymentSupportsFastTimetoValue

16

Stream=BetterInsightsfor*

SolutionArea ContextualData WireData Enriched View

ApplicationManagement

applicationlogs,monitoringdata,metrics,events

protocolconversationsondatabaseperformance,DNSlookups,clientdata,businesstransactionpaths…

Measureapplicationresponsetimes,deeperinsightsforroot-causediagnostics,tracetxpaths,establishbaselines…

IT Operations applicationlogs,monitoringdata,metrics,events

payloaddataincludingprocesstimes,errors,transactiontraces,ICAlatency,SQLstatements,DNSrecords…

Analyzetrafficvolume,speedandpacketstoidentifyinfrastructureperformanceissues,capacityconstraints,changes;establishbaselines…

17

Stream=BetterInsightsfor*SolutionArea ContextualData WireData Enriched View

Security app+infralogs,monitoringdata,events

protocolidentification,protocolheaders,contentandpayloadinformation,flowrecords

Buildanalyticsandcontextforincidentresponse,threatdetection,monitoringandcompliance

DigitalIntelligence

websiteactivity,clickstreamdata,metrics

browser-levelcustomerinteractions

CustomerExperience – analyzewebsiteandapplicationbottleneckstoimprovecustomerexperienceandonlinerevenues

CustomerSupport(online,callcenter)– fasterrootcauseanalysisandresolutionofcustomerissueswithwebsiteorapps

18

ScriptedInputs

19

§ SenddatatoSplunkviaacustomscript§ Splunkindexesanythingwrittentostdout§ Splunkhandlesscheduling§ Supportsshell,Pythonscripts,WINbatch,PowerShell§ Anyotherutilitythatcanformatandstreamdata

StreamingMode§ Splunkexecutesscriptandindexesstdout

§ Checksforanyrunninginstances

WritetoFileMode§ Splunklaunchesscriptwhichproducesoutputfile,noneedforexternalscheduler

§ Splunkmonitorsoutputfile

UseCasesforScriptedInputs

20

§ Alternativetofile-baseornetwork-basedinputs§ Streamdatafromcommand-linetools,suchasvmstat andiostat§ Pollawebservice,APIordatabaseandprocesstheresults§ Reformatcomplexorbinarydataforeasierparsingintoeventsandfields§ Maintaindatasourceswithsloworresource-intensivestartup

procedures§ Providespecialorcomplexhandlingfortransientorunstableinputs§ Scriptsthatmanagepasswordsandcredentials§ Wrapperscriptsforcommandlineinputsthatcontainspecialcharacters

DatabaseInputs

§ Createvaluewithstructureddata§ Enrichsearchresultswithadditionalbusinesscontext

§ Easilyimportdatafordeeperanalysis§ IntegratemultipleDBsconcurrently§ Simpleset-up,non-invasiveandsecure

DBConnectprovidesreliable,scalable,real-timeintegrationbetweenSplunkandtraditionalrelationaldatabases

21

ConfigureDatabaseInputs

22

§ DBConnectApp§ Real-time,scalableintegrationwithrelationalDBs§ Browseandnavigateschemasandtablesbeforedataimport§ Reliablescheduledimport§ SeamlessinstallationandUIconfiguration§ Supportsconnectionpoolingandcaching

§ “Tail”tablesorimportentiretables§ Detectandimportnew/updatedrowsusingtimestampsoruniqueIDs

§ SupportsmanyRDBMSflavors§ AWSRDSAurora,AWSRedShift,IBMDB2forLinux,Informix,MemSQL,MSSQL,MySQL,

Oracle,PostgreSQL,SAPSQLAnywhere(akaSybaseSA),SybaseASEandIQ,Teradata

SplunkODBCDriver

23

§ Interactwith,manipulateandvisualizemachinedatainSplunkEnterpriseusingbusinesssoftwaretools

§ LeverageanalyticsfromSplunkalongsideMicrosoftExcel,TableauDesktoporMicrostrategy AnalyticsDesktop

§ Industry-standardconnectivitytoSplunkEnterprise§ Empowersbusinessuserswithdirectandsecureaccesstomachinedata

§ Combinemachinedatawithstructureddataforbetteroperationalcontext

ODBC:HowitWorks

24

ModularInputs

25

§ Createyourowncustominputs§ Scriptedinputwithstructureandintelligence§ FirstclasscitizenintheSplunkmanagementinterface§ AppearsunderSettings>DataInputs

§ Benefitsoversimplescriptedinput§ Instancecontrol:launchasingleormultipleinstances§ Inputvalidation§ Supportmultipleplatforms§ StreamdataastextorXML§ SecureaccesstomodinputscriptsviaRESTendpoints

ExampleModularInputs

26

Twitter§ StreamJSONdatafromaTwittersourcetoSplunkusingTweepy

AmazonS3OnlineStorage§ IndexdatafromtheAmazonS3onlinestoragewebservice

JavaMessagingService(JMS)§ PollmessagequeuesandtopicsthroughJMSMessagingAPI§ Talkstomultipleproviders:MQSeries (Websphere MQ),ActiveMQ,TibcoEMS,HornetQ,RabbitMQ,NativeJMS,WebLogic JMS,SonicMQ

SplunkWindowsInputs§ RetrieveWINeventlogs,registrykeys,perfmon counters

MoreModularInputs

27

zLinux Forwarder

28

§ EasilycollectandindexdataonIBMmainframes

§ Collectapplicationandplatformdata

§ DownloadasnewForwarderdistributionfors390xLinux

ExtendOperationalIntelligencetoMobileApps

29

DeliverBetterPerforming,MoreReliableApps

DeliverReal-TimeOmni-Channel

Analytics

End-to-EndPerformanceandCapacityInsights

MonitorAppUsageandPerformance

• Improveuserretentionbyquicklyidentifyingcrashesandperformanceissues

• Establishwhetherissuesarecausedbyanapporthenetwork(s)

• Correlateapp,OSanddevicetypetodiagnosecrashandnetworkperformanceissues

30

IntegratedAnalyticsPlatformforDiverseDataStoresFull-featured,IntegratedProduct

FastInsightsforEveryone

WorkswithWhatYouHaveToday

Explore Visualize Dashboards

ShareAnalyze

HadoopClusters NoSQLandOtherDataStores

Hadoop ClientLibraries StreamingResourceLibraries

Bi-directionalIntegrationwithHadoop

ConnecttoNoSQLandOtherDataStores

• Buildcustomstreamingresourcelibraries

• SearchandanalyzedatafromotherdatastoresinHunk

• InpartnershipwithleadingNoSQLvendors

• UseinconjunctionwithDBConnectforrelationaldatabaselookups

VirtualIndexes

§ EnablesseamlessuseofalmosttheentireSplunkstackondata

§ AutomaticallyhandlesMapReduce

§ Technologyispatentpending

DataEnrichment

Agenda

§ Tags – categorizeandaddmeaningtodata

§ FieldAliases – simplifysearchandcorrelation

§ CalculatedFields – shortcutcomplex/repetitivecomputations

§ EventTypes – groupcommoneventsandshareknowledge

§ Lookups – augmentdatawithadditionalexternalfields

35

§ Addsinlinemeaning/context/specificitytorawdata

§ Usedtonormalizemetadataorrawdata

§ Simplifiescorrelationofmultipledatasources

§ CreatedinSplunk

§ Transferredfromexternalsources

WhatisDataEnrichment?

36

§ Addmeaning/context/specificitytorawdata

§ Labelsdescribingteam,category,platform,geography

§ Appliedtofield-valuecombination

§ Multipletagscanbeappliedforeachfield-value

§ Casesensitive

Tags

37

CreateTags

38

SHOW

§ Searcheventswithtaginanyfield

§ Searcheventswithtaginaspecificfield

§ Searcheventswithtagusingwildcards

FindtheWebServersTagsinAction

39

tag=webserver

tag::host=webserver

tag=web*

§ Tagthehostaswebserver

§ Tagthesourcetypeasweb

1

2

3

4

5

SHOW

BacktoSlides

§ Normalizefieldlabelstosimplifysearchandcorrelation§ Applymultiplealiasestoasinglefield

§ Example:Username|cs_username |Userà user§ Example:c_ip |client|client_ipà clientip

§ Processedafterfieldextractions+beforelookups

§ Canapplytolookups

§ Aliasesappearalongsideoriginalfields

FieldAliases

40

Re-LabelFieldtoIntuitiveNameCreateFieldAlias

41

1

2

3

SHOW

§ Createfieldaliasofclientip=customer

§ Searcheventsinlast15minutes,findcustomerfield

§ Fieldalias(customer)andoriginalfield(clientip)arebothdisplayed

SearchusinganIntuitiveFieldNameFieldAliasinAction

42

1

3

2

sourcetype=access_combined

SHOW

§ Shortcutforperformingrepetitive/long/complextransformationsusingevalcommand

§ Basedonextractedordiscoveredfieldsonly

§ Donotapplytolookuporgeneratedfields

CalculatedFields

43

ComputeKilobytesfromBytesCreateCalculatedField

44

1

21

2

3

SHOW

§ Createkilobytes=bytes/1024

§ Searcheventsinlast15minutesforkilobytesandbytes

SearchUsingKilobytesinsteadofBytesCalculatedFieldsinAction

45

1

2

sourcetype=access_combined

SHOW

BacktoSlides

§ Classifyandgroupcommonevents

§ Captureandshareknowledge

§ Basedonsearch

§ Useincombinationwithfieldsandtagstodefineeventtopography

EventTypes

46

§ BestPractice:Usepunctfield§ Defaultmetadatafielddescribingeventstructure§ Builtoninterestingcharacters:",;-#$%&+./:=?@\\'|*\n\r\"(){}<>[]^! »§ Canusewildcards

CreateEventTypes

47

event punct

####<Jun3,20145:38:22PMMDT><Notice><WebLogicServer><bea03><asiAdminServer><WrapperStartStopAppMain><>WLSKernel<><><BEA-000360><ServerstartedinRUNNINGmode>

####<_,__::__>_<>_<>_<>_<>_<>_

172.26.34.223- - [01/Jul/2005:12:05:27-0700]"GET/trade/app?action=logoutHTTP/1.1"2002953

..._-_-_[:::_-]_\"_?=_/.\"__

§ Showpunctforsourcetype=access_combined

§ Pickapunct,thenwildcarditafterthetimestamp

§ AddNOTstatus=200

§ Saveas“bad”eventtype+Color:red+Priority:1(shiftreloadinbrowsertoshowcoloring)

ClassifyEventsasKnownBadCreateEventType

48

eventtype=bad

sourcetype="access_combined" punct="..._-_-_[//_:::]*" NOT status=200

1

2

3

4

SHOW

BacktoSlides

LookupstoEnrichRawData

LDAPAD

WatchLists

CRM/ERP

CMDB

ExternalDataSources

Insightcomesout

DatagoesinCreateadditionalfieldsfromtherawdatawithalookuptoanexternaldatasource

§ Augmentraweventswithadditionalfields§ Providecontextorsupportingdetails

§ Translatefieldvaluestomoredescriptivedata§ Example:addtextdescriptionsforerrorcodes,IDs§ Example:addcontactdetailstousernamesorIDs§ Example:adddescriptionstoHTTPstatuscodes

§ File-basedorscriptedlookups

Lookups

50

51

1.Upload/createtable

2.Assigntabletolookupobject

3.Maplookuptodataset

Convert a Code into a DescriptionConfigure a Static Lookup

SHOW

§ GetthelookupfromtheSplunkWiki(saveto.csv file)http://wiki.splunk.com/Http_status.csv

§ Lookuptablefiles>Addnew§ Name:http_status.csv (musthave.csv fileextension)§ Upload:<pathto.csv>

§ Verifylookupwascreatedsuccessfully

1.CreateHTTPStatusTable

52

SHOW

| inputlookup http_status.csv

1

2

3

§ Lookupdefinitions>Addnew§ Name:http_status§ Type:File-based§ Lookupfile:http_status.csv

§ Invokethelookupmanually

2.AddLookupDefinition

53

SHOW

1

2

sourcetype=access_combined | lookup http_status status OUTPUT status_description

§ Automaticlookups>Addnew§ Name:http_status (cannothavespaces)§ Lookuptable:http_status§ Applyto:sourcetype=access_combined§ Lookupinputfield:status§ Lookupoutputfield:status_description

§ Verifylookupisinvokedautomatically

3.ConfigureAutomaticLookup

54

SHOW

1

2

sourcetype=access_combinedBacktoSlides

§ Temporallookupsfortime-basedlookups§ Example:IdentifyusersonyournetworkbasedontheirIPaddress

andthetimestampinDHCPlogs

§ Usesearchresultstopopulatealookuptable§ … | outputlookup <tablename|filename>

§ Callanexternalcommandorscript§ Pythonscriptsonly§ Example:DNSlookupforIPßà Host

§ Createalookuptableusingarelationaldatabase§ ReviewmatchesagainstadatabasecolumnorSQLquery

FancyLookups

55

§ CreatingandManagingAlerts(JobInspector)

§ Macros

§ WorkflowActions

MoreDataEnrichment

56

LevelUponSearch&ReportingCommands

Agenda

§ Doingmorewithbasicsearchcommands

§ Advancedsearchcommands

§ Doingmorewithbasicreportingcommands

58

SearchSyntaxComponents

59

AnatomyofaSearch

60

Disk

§ top– limit§ rare– sameoptionsastop§ timechart– parameters§ stats– functions(sum,avg,list,values,sparkline)§ sort– inlineascendingordescending§ addcoltotals§ addtotals

DoingMorewithBasicSearchCommands

61

WorkshopNotesforPresenter

Tip#5:Inthenextsection,aftereachsearch,havetheparticipantssavethesearchasadashboardpanel.Attheend

oftheworkshop,theywillhavealivingdocumentoftheworkshopexercisestoreferencelater.

Acompleteversionofthisdashboardispackagedasanapp.ItisuploadedtotheBoxfolderasaleavebehind.

62

§ Commandshaveparametersorqualifiers

§ topandrarehavesimilarsyntax

§ Eachsearchcommandhasitsownsyntax– showinlinehelp

FindMostandLeastActiveCustomersUsingthetop+rareCommands

... | top limit=20 clientip

... | rare limit=20 clientip

IPswiththemostvisits

IPswiththeleastvisits

SHOW

§ Sortinlinedescendingorascending

64

... | stats count by clientip | sort - count

... | stats count by clientip | sort + count

Numberofrequestsbycustomer- descending

Numberofrequestsbycustomer- ascending

SorttheNumberofCustomerRequestsUsingthesortCommand

SHOW

§ ShowSearchCommandReferenceDocs§ Functionsforeval+where§ Functionsforstats+chartandtimechart

§ Invokeafunction

§ Renameinline

65

... | stats sum(bytes) by clientip | sort - sum(bytes)

... | stats sum(bytes) as totalbytes by clientip | sort - totalbytes

Totalpayloadbycustomer- descending

Totalpayloadbycustomer- ascending

DetermineTotalCustomerPayloadUsingfunctions+renamecommand

SHOW

§ Listallvaluesofafield

§ Listonlydistinctvaluesofafield

66

... | stats values(action) by clientip

... | stats list(action) by clientip

Activitybycustomer

Distinctactionsbycustomer

ObserveCustomerActivityUsingthelist+valuesFunctions

SHOW

§ Showdistinctactionsandcardinalityofeachaction

67

sourcetype=access_combined| stats count(action) as value by clientip, action| eval pair=action + " (" + value + ")"| stats list(pair) as values by clientip

AnalyzeCustomerActivityCombinelist+valuesFunctions

SHOW

§ Addcolumns

§ Sumspecificcolumns

68

... | stats count by clientip, action

2cols:clientip +action

... | stats sum(bytes) as totalbytes, avg(bytes) as avgbytes, count as totalevents by clientip | addcoltotals totalbytes, totalevents

Sumtotalbytesandtotaleventscolums

BuildingaTableofCustomerActivityAddColumnsandSumColumns

SHOW

69

... | stats sum(bytes) as totalbytes, sum(other) as totalother by clientip | addtotals fieldname=totalstuff

Foreachrow,addtotalbytes+totalother

Abetterexample:physicalmemory+virtualmemory=

totalmemory

BuildingaTableofCustomerActivitySumAcrossRows

SHOW

70

... | stats sparkline(count) as trendline by clientip

Incontextoflargereventset

... | stats sparkline(count) as trendline sum(bytes) by clientip

Inlineintables

TrendIndividualCustomerActivitySparklinesinAction

SHOW

BacktoSlides

AdvancedSearchCommandsCommand ShortDescription Hints

transaction Groupeventsbyacommonfieldvalue. Convenient,but resourceintensive.cluster Clustersimilareventstogether. Canbeusedon_raworfield.associate Identifiescorrelationsbetweenfields. Calculatesentropybtn fieldvalues.correlate Calculatesthecorrelationbetween

differentfields.Evaluatesrelationshipof allfieldsinaresultset.

contingency Buildsacontingencytablefortwofields. Computesco-occurrence,or%twofieldsexistinsameevents.

anomalies Computesanunexpectednessscoreforanevent.

Computessimilarityofevent(X)toasetofpreviousevents(P).

anomalousvalue Findsandsummarizesirregular,oruncommon,searchresults.

Considers frequencyofoccurrenceornumberofstdev fromthemean

§ Seweventstogether+createsduration+eventcount

§ Sparklinesinlineintables

72

... | transaction JSESSIONID | table JSESSIONID, action, product_id

GroupbyJSESSIONID

ViewCustomerActivitybySessionUsingthetransactionCommand

SHOW

§ Intelligentgroup(createscluster_countandcluster_label)

§ Sparklinesinlineintables

Cluster

73

SHOW

... | cluster showcount=1 | table _raw, cluster_count, cluster_label

BacktoSlides

§ Predictovertime

§ ChartOverlaywithandwithoutstreamstats

§ Mapswithiplocation+geostats

§ Singlevalue

§ Meteredvisualswithgauge

DoingMorewithBasicReportingCommands

74

§ Predictfuturevaluesusinglower/upperbounds– singleandmultipleseries

75

... | timechart count as traffic | predict traffic

PredictWebsiteTrafficUsingthepredictCommand

SHOW

76

sourcetype=access_combined (action=view OR action=purchase)| timechart span=10m count(eval(action="view")) as Viewed,

count(eval(action="purchase")) as Purchased

CompareBrowsingvs.BuyingActivitySimpleChartOverlay

SHOW

77

... | iplocation clientip | geostats count by clientip

CombineIPlookupwithgeomapping

MapCustomerActivity GeographicallyGeolocation inAction

SHOW

78

... | stats count

DisplayaSimpleCountofEventsSingleValueinAction

SHOW

DisplayCountsUsingGaugesSingleValue,RadialandFillerGaugesinAction

79

... | stats count | gauge count 10000 20000 30000 40000 50000

SHOW

BacktoSlides

DataModelandPivot

Agenda

§ Whatisadatamodel?

§ Buildadatamodel

§ PivotInterface

§ Accelerateadatamodel

81

PowerfulAnalyticsAnyoneCanUse

Enablesnon-technicaluserstobuildcomplexreportswithoutthesearchlanguage

Providesmoremeaningfulrepresentationofunderlyingrawmachinedata

Accelerationtechnologydeliversupto1000xfasteranalyticsoverSplunk5

82

Pivot

DataModel

AnalyticsStore

DefineRelationshipsinMachineDataDataModel• Describeshowunderlyingmachinedataisrepresentedandaccessed

• Definesmeaningfulrelationshipsinthedata

• Enablessingleauthoritativeviewofunderlyingrawdata

Hierarchicalobjectviewofunderlyingdata

Addconstraintstofilteroutevents

TransparentAcceleration

• Automaticallycollected– Handlestimingissues,

backfill…• Automaticallymaintained– Usesaccelerationwindow

• Storedontheindexers– Peertothebuckets

• Faulttolerantcollection

Timewindowofdatathatisaccelerated

Checktoenableaccelerationofdatamodel

HighPerformanceAnalyticsStore

Easy-to-UseAnalytics

• Drag-and-dropinterfaceenablesanyusertoanalyzedata

• Createcomplexqueriesandreportswithoutlearningsearchlanguage

• Clicktovisualizeanycharttype;reportsdynamicallyupdatewhenfieldschange

Selectfieldsfromdatamodel

Timewindow

Allcharttypesavailableinthecharttoolbox

Savereporttoshare

Pivot

§ Definesleastcommondenominatorforadatadomain

§ Standardmethodtoparse,categorize,normalizedata

§ Setoffieldnamesandtagsbydomain§ PackagedasaDataModelsinaSplunkApp

§ Domains:security,web,inventory,JVM,performance,networksessions,andmore

§ MinimalsetuptousePivotinterface

CommonInformationModel(CIM)App

86

§ Apps>FindMoreApps>

§ Search:“CommonInformationModel”

§ Installfree

§ Showfieldsforweb+WebDataModel

DownloadCIMApp

87

SHOW

1

2

3

4

BacktoSlides

DataModel&PivotTutorial

http://docs.splunk.com/Documentation/Splunk/latest/PivotTuto

rial/WelcometothePivotTutorial

88

CustomVisualizationsandtheWebFrameworkToolkit

Agenda

§ DeveloperPlatform

§ WebFrameworkToolkit(WFT)

§ RESTAPIandSDKs

§ GetaFlyingStart

90

OptimizingtheAnalyticsProcess

91

Focusonthedata– intuitivetoolstoenabletheanalyst

Nosinglevisualizationexiststohandlealldatasets.

Neverlosesightoftherawdata

SplunkAnalytics

Explore

Context

Visualize

Algorithms

6.0+6.1:Simple,Interactive,andExtensible

92

VISUALIZATIONEXPLORATION

CUSTOMIZABLEFRAMEWORK

POWERFULANALYTICS

PivotDataModels

InteractiveFormsContextualDrilldown

DashboardEditorWebFramework

TheSplunkEnterprisePlatform

Collection

Indexing

SearchProcessingLanguage

CoreFunctions

Inputs,Apps,OtherContent

SDKContent

CoreEngine

UserandDeveloperInterfaces

WebFramework

RESTAPI

What’sPossiblewiththeSplunkEnterprisePlatform?

PowerMobileApps

LogDirectly

ExtractData

CustomerDashboards

IntegrateBITools

IntegratePlatformServices

Developer Platform

PowerfulPlatformforEnterpriseDevelopersDevelopers Can Customize and Extend

RESTAPI

BuildSplunkApps ExtendandIntegrateSplunk

SimpleXML

JavaScript

HTML5

WebFramework

JavaJavaScriptPython

RubyC#PHP

DataModels

SearchExtensibility

ModularInputs

SDKs

SplunkSoftwareforDevelopers

GainApplicationIntelligence

BuildSplunkApps

IntegrateandExtendSplunk

AWealthofSplunk AppsOver1,100appsavailableontheSplunkappssite

APISDKs UI

Server, Storage, Network

Server Virtualization

Operating Systems

Custom Applications

Business Applications Cloud Services

App Performance MonitoringTicketing/ and Other

WebIntelligence

Mobile Applications

Stream

§ Interactive,cut/pasteexamplesfrompopularsourcerepositories:D3,GitHub,jQuery

§ Splunk6.xDashboardExamplesApphttps://apps.splunk.com/app/1603

§ CustomSimpleXML ExtensionsApphttps://apps.splunk.com/app/1772

§ SplunkWebFrameworkToolkitApphttps://apps.splunk.com/app/1613

ExampleAdvancedVisualizations

98

99

http://www.d3js.org

AddaD3BubbleChart

100

1. GotoFindMoreAppsandInstalltheSplunk6.xDashboardExamplesApp

2. EntertheApp3. GotoExamples>CustomVisualizations>

D3BubbleChart4. Copyautodiscover.js (file)+components/bubblechart (dir)

from:$SH/etc/apps/simple_xml_examples/appserver/staticto:$SH/apps/search/appserver/static

5. CopyandpastesimpleXMLtonewdashboard

SHOW

BacktoSlides

Resources

SplunkDocumentation

102

• http://docs.splunk.com• OfficialProductDocs• Wikiandcommunitytopics• Updateddaily• Canbeprintedto.PDF

SplunkAnswers

103

• http://answers.splunk.com• Communitydriven• Splunksupported• Knowledgeexchange• Q&A

SplunkEducation

104

• RecommendedforUsers– UsingSplunk– Searching&Reporting

• RecommendedforUI/DashboardDevelopers– DevelopingApps

• Instructor-LedCourses– Web– Onsite