light speed with test data management
TRANSCRIPT
![Page 1: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/1.jpg)
Light Speed with Test Data ManagementEfficiency Out of Chaos
Kellyn Pot’Vin-Gorman | Technical Intelligence Manager
![Page 2: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/2.jpg)
2© 2016 Delphix Corporation
Kellyn Pot’Vin-GormanTechnical Intelligence Manager, Delphix• Multi-platform DBA, (Oracle, MSSQL, MySQL,
Sybase, Postgres…..)• Oracle ACE Director, (Alumni)• Oak Table Network• APEX Women in Technology Award, CTA 2014• STEM education with Raspberry Pi and Python• Board of Director, RMOUG• Training Days Conference Director• Author, blogger, (http://dbakevlar.com)
![Page 3: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/3.jpg)
This session is focused on the database and application environment. No actual database or vendor platform special
knowledge is required to gain value from the session, although I can answer questions on most platforms due to my
multi-platform technical background.
![Page 4: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/4.jpg)
© 2016 Delphix Corporation 4
AgendaStory Time1What is Test Data Management2Agile Testing, Speed is Everything3Data Masking4Code Control5
![Page 5: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/5.jpg)
5© 2016 Delphix Corporation
Story Time
• Large company
• 4TB transactional database, (small by today’s standards)
• Financial data, aggregated to other financial systems.
• Agile development released, (most often) directly to production
• Archaic development, (1/3 of size of production) and a rarely used test environment.
![Page 6: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/6.jpg)
6© 2016 Delphix Corporation
After 4-6 Weeks of Research…
• Over 40% data corruption to main transactional system that feeds into datamarts.
• Even higher percentage of corruption impact in marts due to poor agile development practices and percentage of highly volatile source data.
I now have to reveal my findings to senior management as the new Lead DBA…
![Page 7: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/7.jpg)
7© 2016 Delphix Corporation
“Accuracy is over-rated…”
![Page 8: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/8.jpg)
8© 2016 Delphix Corporation
https://www.tutorialspoint.com/software_testing_dictionary/test_data_management.htm
Test Data management is very critical during the test life cycle. The amount of data that is generated is [often] enormous to test the required changes.
Over 80% of organizations stated that RECEIVING or REFRESHING the data to perform tests was the largest consumer of testing time, (over 90%) leaving actual work to consume less than 10% of the overall testing scenario.
Test Data Management
![Page 9: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/9.jpg)
9© 2016 Delphix Corporation
What Is Test Data Management?• Tools that assist in producing data sets for testing.
• Not only produce, but do so in an interval that is able to match the testing cycle.
• An ability to quickly isolate and deliver test FAILURES for development to investigate.
• Ability to identify code changes by versions
• To deliver in a timely manner, many TDM applications create synthetic or subsets of data that aren’t a full copy of the production experience.
• Don’t take copy data management and security into consideration.
![Page 10: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/10.jpg)
10© 2016 Delphix Corporation
As important as development environments are, providing proper
testing that’s able to handle the speed of your companies
deployments are a bigger hurdle.
![Page 11: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/11.jpg)
11© 2016 Delphix Corporation
Cloning- Can’t Be Done Effectively
http://www.informationweek.com/pdf_whitepapers/approved/1345732672_back_to_basics.pdf
![Page 12: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/12.jpg)
12© 2016 Delphix Corporation
What is the “Right Size”?
These are not the same data or the same challenges the
developer and tester will face once it gets to production!
http://www.informationweek.com/pdf_whitepapers/approved/1345732672_back_to_basics.pdf
![Page 13: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/13.jpg)
13© 2016 Delphix Corporation
▶▶▶
Virtualize and Deployed ▶ ▶▶
Virtualization Eliminates 90%+ of Storage Issues
Storage Pool for Delphix
QADEV PATCH TEST
PRODUCTIONDatabase/App Tier
1 TB
1 TB
600GBRead From Production
Each Virtual Database takes up around 5-10Gb upon creation, (dependent upon parameters)
TEST
Read AND Write
![Page 14: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/14.jpg)
14© 2016 Delphix Corporationhttp://www.idtheftcenter.org/images/breach/ITRCBreachReport_2016.pdf
Data Security
Almost 30 Million!
![Page 15: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/15.jpg)
15© 2016 Delphix Corporation
Confidential data
Production
Non-Production
Exposure
![Page 16: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/16.jpg)
16© 2016 Delphix Corporation
Confidential data
Production
Non-Production
Exposure
Encryption
Masking
Solution
![Page 17: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/17.jpg)
17© 2016 Delphix Corporation
As 80% of data in a company are copies, then 80% of data won’t be subject to security like a production environment. Securing this data is not just a priority, but in many cases, subject to legal ramifications, (i.e. PCI/PII)
• Masking Requirements• Masking shouldn’t be reversible• The masked data should be representative of the original data type to ensure
performance is consistent.• Referential Integrity should be maintained as part of the masking process.
Masking should be a simple, repeatable process with a user interface that ensures it is simple.
Masking in the Picture
![Page 18: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/18.jpg)
18© 2016 Delphix Corporation
Do I Have to Mask Data? Nah….
Type of Data Year Passed RulingData Masking in the EU
2014 ARTICLE 29 DATA PROTECTION
HIPAA 1996 Health Insurance Portability and Accountability Act
PCI 2016, (Updated)
Payment Card Industry Standards
PII Personably Identifiable Information
SOX 2002 Sarbanes-Oxley Act
![Page 19: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/19.jpg)
19© 2016 Delphix Corporation
Ability to Deliver and Mask Data for Testing- FAST
DB
APP
DB
APPDB
APP
DB
APP
DB
APP
PRODUCTION DB
Mask PCI/PII and then virtualize
Develop Test Deploy
Refresh and Start Over
![Page 20: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/20.jpg)
20© 2016 Delphix Corporation
Source Control
“A component of software configuration management, version control, also known as revision control or source control, is the management of changes to documents, computer programs, large web sites, and other collections of information.”
![Page 21: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/21.jpg)
21© 2016 Delphix Corporation
Branching and Bookmarking• The ability to mark each iteration of development with a
bookmark
• Simplify to lock and deliver while testing a consistent image via a virtual database, (VDB)
• If a test goes wrong, the ability to “bookmark”, (and subsequent snapshot) to deliver to development to address.
![Page 22: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/22.jpg)
22© 2016 Delphix Corporation
![Page 23: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/23.jpg)
23© 2016 Delphix Corporation
Environments
Snapshot
Previous Snapshots
Virtual to Physical
Timeflow Slider
Actions
![Page 24: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/24.jpg)
24© 2016 Delphix Corporation
How This All Comes Together…
• Virtualization is the key to fast, efficient and FULL copies of production environments for agile and automated testing for agile shops.
• Data masking that can be done once, easily maintained with a repeatable process via a strong discovery and implementation as part of the virtualization process secures the 80% of data that is outside the control of production.
• Virtualized environments that are built with development and testing in an Agile or DevOps environments makes it simple to accomplish what may see impossible and do so at light speed.
![Page 25: Light Speed with Test Data Management](https://reader036.vdocuments.us/reader036/viewer/2022062823/5872e7171a28abfa548b62ff/html5/thumbnails/25.jpg)
Kellyn Pot’Vin-GormanTechnical Intelligence [email protected]://dbakevlar.com