technical*deep*dive:*dataintegraon* between*splunk*and ... · whataboutstructured*data?* 5 customer...
TRANSCRIPT
![Page 1: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/1.jpg)
Copyright © 2013 Splunk Inc.
Raanan Dagan Sr. Sales Engineer -‐ Hadoop Domain Export, Splunk #splunkconf
Technical Deep Dive: Data IntegraIon Between Splunk and RelaIonal Databases
![Page 2: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/2.jpg)
Legal NoIces During the course of this presentaIon, we may make forward-‐looking statements regarding future events or the expected performance of the company. We cauIon you that such statements reflect our current expectaIons and esImates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-‐looking statements, please review our filings with the SEC. The forward-‐looking statements made in this presentaIon are being made as of the Ime and date of its live presentaIon. If reviewed aTer its live presentaIon, this presentaIon may not contain current or accurate informaIon. We do not assume any obligaIon to update any forward-‐looking statements we may make. In addiIon, any informaIon about our roadmap outlines our general product direcIon and is subject to change at any Ime without noIce. It is for informaIonal purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligaIon either to develop the features or funcIonality described or to include any such feature or funcIonality in a future release.
Splunk, Splunk>, Splunk Storm, Listen to Your Data, SPL and The Engine for Machine Data are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respecCve
owners.
©2013 Splunk Inc. All rights reserved.
2
![Page 3: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/3.jpg)
Agenda
! Background and Overview ! DB Connect Demo ! Technical Overview ! Customer Examples and Summary
3
![Page 4: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/4.jpg)
Background and Overview
4
![Page 5: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/5.jpg)
What About Structured Data?
5
Customer profile
Product a1ributes
Employee details
Pricing and Rate plans
Asset info
![Page 6: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/6.jpg)
Machine Data – Delivers Real-‐Ime Insights
6
Media server logs
(machine data)
Mar 01 19:18:50:000 aaa2 radiusd[12548]:[ID 959576 local1.info] INFO RADOP(13) acct start for [email protected] 10.164.232.181 from 12.130.60.5 recorded OK.!2013-03-01 19:18:50:150 10.2.1.34 GET /sync/addtolibrary/01011207201000005652000000000053 - 80 - 10.164.232.181 "Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3" 503 0 0 825 1680!Mar 01 19:18:50:163 aaa2 radiusd[12548]:[ID 959576 local1.info] INFO RADOP(13) acct stop for [email protected] 10.164.232.181 from 12.130.60.5 recorded OK.!
Phone Number IP Address Track ID
![Page 7: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/7.jpg)
Structured Data – Contains Business Context
7
Media server logs
(machine data)
Mar 01 19:18:50:000 aaa2 radiusd[12548]:[ID 959576 local1.info] INFO RADOP(13) acct start for [email protected] 10.164.232.181 from 12.130.60.5 recorded OK.!2013-03-01 19:18:50:150 10.2.1.34 GET /sync/addtolibrary/01011207201000005652000000000053 - 80 - 10.164.232.181 "Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3" 503 0 0 825 1680!Mar 01 19:18:50:163 aaa2 radiusd[12548]:[ID 959576 local1.info] INFO RADOP(13) acct stop for [email protected] 10.164.232.181 from 12.130.60.5 recorded OK.!
Track ID ArIst Title Format ID Run Ime
01011207201000005652000000000053 Maroon 5 Moves like Jagger MP3 4:30
Phone # Subscriber ID
2172618992 53546
Subscriber ID
First Name Last Name Age State Customer Score
53546 Jim Morrison 25 CA 93
Customer, product databases
Phone number IP address Track ID
![Page 8: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/8.jpg)
Enrich Machine Data with Structured Data
8
Structured databases
CSV lookup
DB Connect >10,000 downloads
![Page 9: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/9.jpg)
Introducing Splunk DB Connect
! Enrich search results with addiIonal business context
! Easily import data into Splunk for deeper analysis
! Integrate mulIple DBs concurrently ! Simple set-‐up, non-‐evasive and secure
Reliable, scalable, real-‐Jme integraJon between Splunk and tradiJonal relaJonal databases
MicrosoT SQL server
JDBC
Database lookup
Database query
ConnecIon pooling
Other databases
Oracle database
Java Bridge Server
9
![Page 10: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/10.jpg)
Splunk DB Connect Demo
10
![Page 11: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/11.jpg)
Splunk DB Connect Technical Overview
![Page 12: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/12.jpg)
Splunk DB Connect: Main Features
12
! Database connecIon management ! SQL database lookups ! Splunk search language extensions – Database query – Database info – Database output
! SQL database input ! Access control
![Page 13: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/13.jpg)
Installing Splunk DB Connect
13
! Simple app setup, no configuraIon files to touch ! AutomaIcally checks for the required Java version
![Page 14: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/14.jpg)
Database ConnecIon Management Configure new database connecIon sefngs in minutes
from the Splunk user interface
14
![Page 15: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/15.jpg)
MicrosoT SQL server
JDBC
Database lookup
Database query
ConnecIon pooling
Other databases
Oracle database
Java Bridge Server
Works With Many Databases
15
! Supports mainstream databases – Oracle database – MicrosoT SQL server – MySQL – PostgreSQl – Sybase – DB2 – Generic JDBC support
! Database connecIon pooling limits load on database
![Page 16: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/16.jpg)
Database Lookups
16
Enrich machine data by adding structured data from tradiIonal relaIonal databases
![Page 17: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/17.jpg)
Three Steps to Enriching Machine Data
17
1. Connect
2. Configure
3. Enrich
![Page 18: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/18.jpg)
Splunk Search Language Extensions
18
Execute database queries directly from the Splunk user interface with new Dbquery, Dbinfo, and DBoutput Splunk search commands
![Page 19: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/19.jpg)
Explore Database Structure
19
! Wrapping dbinfo and dbquery
Browse and navigate database schemas and tables from the Splunk DB Connect user interface
![Page 20: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/20.jpg)
Import and Index Database Data Combine machine data with structured data from relaIonal databases
20
New dbmon-‐tail and dbmon-‐dump input types can be used to import rows from the database
![Page 21: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/21.jpg)
Access Control Database ConnecIon
21
Access Control 1. Splunk administrator can set
users / roles with permissions for the connecIon (block, read, write)
2. Database connecIon can be set to a ‘Read-‐Only’
3. DBA can set permission on the database side
![Page 22: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/22.jpg)
Technical Summary
• Quick to set-‐up, scales to mulIple concurrent databases • Enrich machine data with database data in three easy steps • Execute SQL queries to visualize database data directly in the
Splunk user interface • Import and index database data for historical analysis and
correlaIon with machine data
22
![Page 23: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/23.jpg)
Success Stories
![Page 24: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/24.jpg)
Enabling ExcepIonal Customer Service
24
Users to customers mapping
SQL SQL
User acJvity
= Customer details, external/internal details
Database
+
Machine Data
= User acIvity data from SaaS applicaIon, websites
SaaS
Real-‐Jme visibility of customer experience
Website
![Page 25: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/25.jpg)
Driving ProacIve Network Management
25
Network Switch Data
Splunk DB Connect imports millions of records per day: ! Visualize graph of outliers ! Detect high uIlizaIon paqerns
Import and index data
• CPU uIlizaIon • Inbound packets • Network staIsIcs • Data for 10,000+
switches
Problem management dashboards
![Page 26: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/26.jpg)
Summary
• Machine data contains a categorical record of acIvity and behavior
• Enrich with structured data to provide business context – for beqer IT, security and business insights
• Splunk DB Connect delivers reliable, scalable, real-‐Ime integraIon between Splunk and tradiIonal relaIonal databases
26
![Page 27: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/27.jpg)
Next Steps
27
1
2
3
Download the .conf2013 Mobile App If not iPhone, iPad or Android, use the Web App
Take the survey & WIN A PASS FOR .CONF2014… Or one of these bags!
Go to the Splunk DB Connect demo staJon on level 3
![Page 28: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/28.jpg)
QuesJons Raanan Dagan [email protected]
![Page 29: Technical*Deep*Dive:*DataIntegraon* Between*Splunk*and ... · WhatAboutStructured*Data?* 5 Customer profile) Product) aributes Employee) details Pricingand)) Rate)plans) Asset) info](https://reader034.vdocuments.us/reader034/viewer/2022042119/5e9873e3d743931ede7ac49e/html5/thumbnails/29.jpg)
THANK YOU