how we became world leaders in agile data warehousing - teradata partners - october 2013
DESCRIPTION
Presentation at the Terdata Data Partners User Group conference in Dallas, TX on 24th October 2013.TRANSCRIPT
How We Became World Leaders in Agile Data Warehousing Em Campbell-Pretty
General Manger. Strategic Delivery (EDW)
Business Intelligence CoE, Telstra
@PrettyAgile
The Enterprise Data Warehouse
80 TB of business data
28 Teradata nodes 3 TB of new data per month
350 SQL users 47 business systems feeding into
the EDW
400 million records loaded daily 13 billion records extracted daily
43 billion records in largest table
4 tables with over 35 billion records
124 different client apps
6.8 million queries daily 16K tables, 28K views
and 2K macros 7K jobs running daily
21K processes
The Team
Average delivery cycle time down from 12 month to 3 months
Frequency of delivery increased from quarterly to fortnightly
Cost to deliver down 50%
95% decrease in product defects
100% projects delivered on time and on budget
Happy project sponsors
Happy teams
The Results
WHERE SHOULD YOU START?
START WHERE YOU ARE
http://www.flickr.com/photos/joelogon/4613630072/
Read, Understand, Adapt & Apply
http://www.flickr.com/photos/duluoz_cats/5300924038/
Wide Pipes & Batch Patterns
Agile Pilot
5 “Wagile” Teams
Image source: http://www.vectorloft.com/img/purple-person.jpg
www.scaledagileframework.com
17PRESENTATION TITLE | PRESENTER NAME | DATE |
Deployment
Services
Development Services
Pipeline
Services
Structuring the Agile Release Train
18PRESENTATION TITLE | PRESENTER NAME | DATE |
Agile at the Program Level
MAKE THE INVISIBLE INVENTORY VISIBLE
Reduce Batch Size by Breaking Down the Work
21PRESENTATION TITLE | PRESENTER NAME | DATE |
Agile Feature Teams
Scrum Provides Cadence & Fast Feedback
Image Siource: http://innolution.com/uploads/misc/Essential_Scrum_Chapter_2.pdf
Smaller, Multi-skilled Teams Counter Balance Variability
Project Manager
Product Owner
Business Analyst
Change Lead Technical
Lead
Logical Data Modeller
Physical Data Modeller
Physical Data Modeller
Vender Project Manger
On Site ETL On Site ETL
On Site ETL Off shore ETL
Off shore ETL Off shore ETL
DBA
Business Intelligence
Test Lead
Test Analyst
Enterprise Architect
Data SME
Scrum Master
Project Manger
Scrum Master
Technical Lead
Test Lead Developer Developer Developer Developer Developer
As is
As was
EPIC OWNER
FEATURE OWNER
PRODUCT OWNER
Scale the Product Owner to Balance Centralised & Decentralised Control
Extreme programming improves quality
Extreme Programming Improves Quality
Limiting Work in Process
(WIP) Improves Flow
27PRESENTATION TITLE | PRESENTER NAME | DATE |
The System Team
Build the Infrastructure to Enable Economically Viable Smaller Batches
29PRESENTATION TITLE | PRESENTER NAME | DATE |
The Agile Release Train
Cocktail Hour Proves a Daily Communication Cadence
9:00am 9:15am 930am
9:45am
Unity Day Leads to One Team Culture
31
A Little Taste of Our Culture
http://www.prettyagile.com/2013/05/the-power-of-haka.html
The Software House of Lean
Respect for
People
Product
Development
Flow
Kaizen
1. Take an economic view
2. Actively manage queues
3. Understand and exploit variability
4. Reduce batch sizes
5. Apply WIP constraints
6. Control flow under uncertainty: cadence and synchronization
7. Get feedback as fast as possible
8. Decentralize control
The Principles of Product Development Flow
Respect for People: The Net Promoter System (NPS)
Source: http://www.netpromotersystem.com/
Net Promoter Score
Kaizen Mindset: Continuous Improvement and Learning Culture
3
Source: https://dl.dropboxusercontent.com/u/1018963/Articles/SpotifyScaling.pdf
The Foundation is Servant Leadership
“This is probably the most well-scaled agile data warehousing group I’ve ever observed, and they are still getting better” Ken Collier
Director of Agile Analytics at Thoughtworks & author of Agile Analytics
Source: http://theagilist.com/2013/09/03/scaling-agile-data-warehousing/
WHERE SHOULD YOU START?
START WHERE YOU ARE
http://www.flickr.com/photos/joelogon/4613630072/
"Adapt what is useful, reject what is useless, and add what is specifically your own."
- BRUCE LEE
Check out my blog: www.prettyagile.com
• A Perspective on the Scaled Agile Framework
• What Happens to Project Managers When You Implement SAFe?
• The Power of Haka
• Book Clubs at Work – Are You Serious?
• Leading Through Vulnerability
• The "Bubble Up" Approach to Scaling Retrospectives
• Bookshelf
Want to Know More About the EDW Agile Release Train?
PARTNERS Mobile App
InfoHub Kiosks
teradata-partners.com
Twitter: @
Email:
PrettyAgile