big data for satellite business...
TRANSCRIPT
![Page 1: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/1.jpg)
Big Data for Satellite Business Intelligence
GSAW 2015 Loic COULET, Kratos ISE
© 2015 by Kratos ISE. Published by The Aerospace Corporation with permission.
![Page 2: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/2.jpg)
2 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Who’s talking?
Loic COULET Software Engineer
Kratos ISE
Systems Intregration
Software
Passionate
Now Learning
Presenting today
10 years
M&C
CSM
Satellite C2
Java
Databases
Web
Computer Science
Big Data
Analytics
NoSQL
Big Data
Business Intelligence
Learning
![Page 3: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/3.jpg)
3 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Kratos Integral Systems Europe (KISE)
Subsidiary of Kratos / Kratos ISI, Toulouse, France
Toulouse
![Page 4: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/4.jpg)
4 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
KISE provides Ground stations solutions
![Page 5: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/5.jpg)
5 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Why am I talking today?
SBI Press Release caught attention…
http://ir.kratosdefense.com/releasedetail.cfm?releaseid=870415
![Page 6: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/6.jpg)
6 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Es’hailsat – The Qatar Satellite Company
The Qatar Satellite
Company
![Page 7: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/7.jpg)
7 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Why am I talking today?
Kratos Brings Big Data Analysis to Satellite Operations for Es'hailSat New Satellite Business Intelligence (SBI) Dashboard Correlates Data Across Key Ground Segment Solutions to Enhance Service Level Management
![Page 8: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/8.jpg)
8 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
… Using smart keywords… …For which…
![Page 9: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/9.jpg)
9 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
…I have 20 15 minutes to speak about
How Big? What kind?
What does SBI mean?
How is it presented?
What kind of correlation?
How does it work?
1
2
3
4
5
6
![Page 10: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/10.jpg)
10 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How big is Big?
![Page 11: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/11.jpg)
11 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Big Data = 3V’s
3 V’s
![Page 12: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/12.jpg)
12 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Big Data = 3V’s
NUMBER OF
METRICS
STORAGE REQUIREMENTS
THROUGHPUT
![Page 13: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/13.jpg)
13 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Data Source Systems
CSM Monics Compass Epoch IPS Neuralstar
M&C Satellite C2 Network Mgmt
![Page 14: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/14.jpg)
14 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Number of metrics
CSM Monics Compass Epoch Neuralstar
M&C Satellite C2 Network Mgmt
High High Low High
![Page 15: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/15.jpg)
15 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Data Troughput
CSM Monics Compass Epoch Neuralstar
M&C Satellite C2 Network Mgmt
High High High Low
![Page 16: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/16.jpg)
16 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Storage requirements
CSM Monics Compass Epoch Neuralstar
M&C Satellite C2 Network Mgmt
High High High Medium
![Page 17: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/17.jpg)
17 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Big Data Problem?
• Store all data for any length of time ?
• Correlation between data sources?
• Further analysis to detect unknown information?
• Learning model to anticipate failures?
AND THEN
…
Systems generate…
data amount
Real-Time processing
Legacy Storage
is everything archived?
How efficiently is data stored
and used?
![Page 18: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/18.jpg)
18 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How Big? What kind?
What does SBI mean?
How is it presented?
What kind of correlation?
How does it work?
![Page 19: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/19.jpg)
19 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
What does “Satellite Business Intelligence”
mean?
![Page 20: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/20.jpg)
20 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Satellite Business Intelligence = ?
Business…
![Page 21: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/21.jpg)
21 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Business Evaluation
Analyze
Implement KPI
Evaluate
Satellite Business Intelligence = ?
+ Intelligence…
![Page 22: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/22.jpg)
22 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
KPI?
![Page 23: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/23.jpg)
23 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
…For your satellite services
![Page 24: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/24.jpg)
24 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How Big? What kind?
What does SBI mean?
How is it presented?
What kind of correlation?
How does it work?
![Page 25: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/25.jpg)
25 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
What kind of analysis?
![Page 26: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/26.jpg)
26 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Correlations for services monitoring dashboard
Metrics KPI Limit
checking Rules
SLA Check Rules
Services Report
Data Source System
Correlations
Data Source System
+ Business rules
![Page 27: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/27.jpg)
27 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
1. Generic predictive analysis (in the query engine) is being implemented
2. Several predictors : linear (exponential Smoothing, holt, least squares), or
dymanic with Dynamic Linear Model (DLM)
Time Series Prediction analysis
Actual Data DLM Prediction
![Page 28: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/28.jpg)
28 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How Big? What kind?
What does SBI mean?
How is it presented?
What kind of correlation?
How does it work?
![Page 29: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/29.jpg)
29 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How is it presented?
![Page 30: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/30.jpg)
30 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How is it presented?
As you like !
![Page 31: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/31.jpg)
31 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
A Dashboard
Grafana Time Series Dashboard
![Page 32: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/32.jpg)
32 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
THE Dashboard
Es’hailsat Monitoring Dashboard displays on a Web Browser with real-time information
![Page 33: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/33.jpg)
33 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Configuration using a CSV file
The CMC Monitoring Dashboard configuration file (CSV file) is edited manually by the CMC operator. Configures: • KPI Thresholds • Monitoring Plans • Monitored Services
![Page 34: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/34.jpg)
34 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
CMC Monitoring Dashboard (Views)
The CMC Monitoring Dashboard User can see 2 different tables (one with the KPIs and one with the MARGINs) Click on the “KPI” button to view the latest Keys Performance Indicators
System A System B System C
![Page 35: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/35.jpg)
35 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
CMC Monitoring Dashboard (Margin View)
Click on the “MARGIN” button to view the margins
![Page 36: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/36.jpg)
36 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
CMC Monitoring Dashboard (KPI view – Plots)
Double Click on a KPI to see the real time plot
![Page 37: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/37.jpg)
37 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
CMC Monitoring Dashboard (KPI view – Trend)
Click on the Trend View Button to view the historical values of the KPIs
![Page 38: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/38.jpg)
38 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How Big? What kind?
What does SBI mean?
How is it presented?
What kind of correlation?
How does it work?
![Page 39: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/39.jpg)
39 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
What kind of correlation What kind of Correlations?
![Page 40: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/40.jpg)
40 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Correlations for services monitoring dashboard
Metrics KPI Limit
checking Rules
SLA Check Rules
Services Report
Data Source System
Correlations
Data Source System
+ Business rules
![Page 41: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/41.jpg)
41 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Correlate one reference series to many others
Search correlations
Interactive histogram represents most correlated series
A Single chart is displayed by default presenting the reference data
Green to red colors indicate reliability of the correlation score (based on relative number of samples) The higher the bar, the most correlated the series (first is the reference to itself)
eirp carrier=Carrier_1_Ref
![Page 42: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/42.jpg)
42 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Visualizing Correlation results
Time Series Correlations Analysis - Search
Click on a bar to visualize a correlated series on the bottom of the page.
The first plot always is the reference data
This plot is the second most correlated series
This plot is the fourth most correlated series…
This shows real data samples…
eirp carrier=Carrier_1_Ref
eirp carrier=Carrier_3_tst
eirp carrier=Carrier_2
![Page 43: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/43.jpg)
43 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Same query model than in correlations search
Time Series Correlations Analysis - Discover
Interactive correlation matrix
Do clustering using metric tags
Change reliability threshold
Mouse over a square in the matrix provides information about correlated metrics. Click shows the two plots.
![Page 44: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/44.jpg)
44 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How Big? What kind?
What does SBI mean?
How is it presented?
What kind of correlation?
How does it work?
![Page 45: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/45.jpg)
45 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How does it work?
• Time Series Database (KairosDB) • NoSQL Database as storage backend (Apache
Cassandra) • Domain expertise and deep integration
How does it work?
![Page 46: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/46.jpg)
46 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How does it work?
• Time Series Database (KairosDB) • NoSQL Database as storage backend (Apache
Cassandra) • Domain expertise and integration skills
![Page 47: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/47.jpg)
47 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
KairosDB is modular
core
Real-time KPI &
Dashboard Module
Skyminer Analytics Module
![Page 48: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/48.jpg)
48 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Solution
Carrier Monitoring Monics
M&C Compass
Satellite C2 Epoch IPS
NMS Neuralstar
Data Collector agent
Data Collector agent
Data Collector agent
Data Collector agent
Data Integration Frontend
Reporting & analytics Frontend
Storage
Skyminer Web UI
External Analytics systems
Other Data Sources
![Page 49: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/49.jpg)
49 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Primary Goals
1. Time series monitoring & analytics
2. Fast
3. Flexible & Scalable
4. Fault tolerant
5. Incorporates useful analysis features
6. Open to other systems
7. Cost-controlled
![Page 50: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/50.jpg)
50 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Typical System(s)
Fault management Data replication Low cost Quick start Easy administration Scale to any size Best performances
Fault-Tolerant Small Cluster Using Apache Cassandra DB
![Page 51: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/51.jpg)
51 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
1. All features are provided as web services (HTTP / REST)
2. Open APIs
3. Interoperable data format based on JSON
4. Intuitive Web UI for starting using the system
5. APIs include: • Data acquisition
• Data querying
• Analysis features (prediction, correlations)
Interoperability Features
![Page 52: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/52.jpg)
52 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Query Engine & aggregations
• Ad-hoc queries and statistics calculation
• Business Intelligence features already implemented (aggregate, drill & pivot)
• Data aggregates: Min, Max, Sum, Average, Count, Rate, Std Deviation…etc
• Multi-level Group-by feature using tags, value, or time
• Filter by tags values
![Page 53: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/53.jpg)
53 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Using BIRT reporting tool
Reporting
![Page 54: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/54.jpg)
54 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
How Big? What kind?
What does it mean?
How is it presented?
What kind of correlation?
How does it work?
![Page 55: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/55.jpg)
55 © 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. | www.integ-europe.com | GSAW 2015 |
Conclusion
Usual Real-Time Processing
• Virtually keep all data forever… Scale to any size!
• Correlation between data sources!
• Automated analysis for new information?
• Learning model to anticipate failures?
AND THEN
…
Systems generate…
data amount
Big Data
Storage
![Page 57: Big Data for Satellite Business Intelligencegsaw.org/wp-content/uploads/2015/04/2015s11e_coulet.pdf · Big Data for Satellite Business Intelligence GSAW 2015 ... • Time Series Database](https://reader030.vdocuments.us/reader030/viewer/2022011723/5aa5563c7f8b9a2f048d450b/html5/thumbnails/57.jpg)
Any Questions?