data vault reconnect speed presenting pm part four
DESCRIPTION
Third set of 5x5 Speed Presenting Updates: 1) Selling Data Vault - Elwyn Lloyd Jones 2) DV as Leverage for Data Migration - Antoine Stelma 3) QUIPU in Banking - Juan-José van der Linden 4) Volvo Data Vault Automation - Frederik Naessens 5) Are we ready for a DV Conversion Standard - Frederik Naessens, Stijn Roelens, Kristof VandurenTRANSCRIPT
![Page 1: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/1.jpg)
Selling Data Vault
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Elwyn Lloyd Jones5 June 2014
Sevenoaks Systems [email protected]
![Page 2: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/2.jpg)
Types of Argument
1. ad Verecundiam / ad HominemNSA, US Army, JP Morgan, ABN AMRO, KPN, Bank of England, Volvo, Mercedes, Volkswagen, Bill Inmon
2. ad Populum800 organizations
3. ad BaculumAudit-able system of records
4. Reducto ad Misericordiam Throw away and start from scratch Source integration => Enterprise crash
5. Up and UnderWhile client watches the BI - we’ll watch the DWH
6. Scallability (cost) – busting Moore’s LawNo columnar Netezza, Exadata, etc.
![Page 3: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/3.jpg)
Data Vault as leverage for Data Migration
Exploring the benefits
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Antoine StelmaJune 5 2014Presentation
Centennium [email protected]@antoinestelma
![Page 4: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/4.jpg)
Challenges
Client: • wants new ERP• 10 old source systems• own plant with intensive processes• MD/MT data is spread• unify & simplify architecture• business process changed
![Page 5: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/5.jpg)
Quest
A data warehouse that:• helps speeding up and unifying data migration• contains current data and processes• is a solid base for a new ERP • unifies master & meta data
![Page 6: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/6.jpg)
Two scenario’s
Source Source
Source SourceE D W
Source Source
Source Source
E D W
ERP
Module 1
Module 2 Module 3
Module 4
Module 5
Migrated data ETL
Source/MD/MT
Classic approach
Suggested approach
ERP
Module 1
Module 2 Module 3
Module 4
Module 5
Migrated data
ETL
Organization
![Page 7: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/7.jpg)
How?
• scope of data for migration• type DV • collect master & metadata in H+S• unify & data cleaning • relations/links/etc. • alignment between old source and new ERP
![Page 8: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/8.jpg)
Lessons (to be) Learned
#lesson1: An EDW can help you with data migration by using DV#lesson 2: EDW does not have to wait for full ERP implementation#lesson 3: It’s all about Data! # be creative
![Page 9: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/9.jpg)
in Banking
Enabling market risk data analysis
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Juan-José van der LindenJune 5, 2014
[email protected]@OS_Quipu
![Page 10: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/10.jpg)
Bank Example
- Prudential Regulation Authority (PRA) - Prevent new financial crisis- Regular collection and storage data of firms’
trading book positions- Simulate risk scenarios
![Page 11: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/11.jpg)
High-level requirements
Deliver an effective stress testing capability:
– Report stress results with material accuracy and on a timely basis
Requires an industrial-strength infrastructure for data collection, risk analysis and reporting
Challenges:Need for a flexible data repository:
– Effective data management (model changes)
– Facilitate historical data analysis – Facilitate data provenance
management
![Page 12: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/12.jpg)
Architecture
Metadata
Data
Legend
Oracle data modeler
SQL Server
SSIS/SSAS/SSRS
Quipu
Tableau
Source data files
Matadata flow
Data flow
Based on
Stg DV 3NFETL
Src
SrcBM
xls Stg DV 3NFSSISSSIS
SSAS
SSIS
SSRS
Tableau
![Page 13: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/13.jpg)
Proof of Concept and Project
Three-week proof of concept:- 1 week business data model support- 1 week data warehouse development - 1 week demonstrating changes and developing
Tableau reporting
Project support:- Model validation- Quipu training- Architecture support and validation
Ongoing:- Quipu product support as part of software
license
![Page 14: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/14.jpg)
Takeaways
# 1 – Invest in getting the input 3NF business model.# 2 – Automation requires good data models.# 3 – Model and code generation drastically speeds up projects.# 4 – Generation tooling needs to support model changes. # 5 – Business value is made visible at reporting level.# 6 – DV modeling is an enabler, not the goal.
![Page 15: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/15.jpg)
Volvo DV Automation
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Frederik NaessensJune 5 2014In cooperation with Stijn Roelens & Kristof [email protected]
![Page 16: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/16.jpg)
Volvo: criteria Automation Tool selection
Agile way of working– Handle migration between DEV-QA-PROD– Handle parallel development
Automation– Perform automatic data migration when data
model is modified– Automatically generate ELT code
Respect industry standards– Be netezza compliant– Generate a 100% Data Vault compliant data
model
Fluent interaction between business & it– Provide or interact with a business data model
![Page 17: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/17.jpg)
3D – Configure extended properties
![Page 18: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/18.jpg)
3D – Use predefined but flexible Model Conversion rules
![Page 19: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/19.jpg)
3D – Configure extended properties
![Page 20: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/20.jpg)
Are we ready for a Data Vault Standard?
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Frederik NaessensJune 5 2014In cooperation with Stijn Roelens & Kristof [email protected]
![Page 21: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/21.jpg)
Wherescape 3D
What we do: • Configure Source/Business model using 3NF
modeling conventions• Add additional metadata at Table /
Column / Link / … level• Based on a set of preconfigured rules,
generate:– Data vault model– Staging model– Staging Out views
![Page 22: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/22.jpg)
Marketplace
Isn’t this what different software packages or consultancy firms are currently providing in the marketplace?
See:• Presentation of Lulzim Bilali on DWA
congress• QUIPU• BIReady• Generators built on top of data modeling
tools (Powerdesigner, ERWIN,… )
![Page 23: Data Vault ReConnect Speed Presenting PM Part Four](https://reader035.vdocuments.us/reader035/viewer/2022081414/54b24bf24a7959e4098b456e/html5/thumbnails/23.jpg)
DV Model Conversion standard
Is now not the moment to define a DV model conversion standard?Why?• Different tools can align to a single standard
and create synergies• This can be a catalysator to convince
organisations or people which are new to DV concepts
• Brings the focus to the really important concepts of Data Vault:
• Business keys• UOW• …