continuous integration with business intelligence and analytics
TRANSCRIPT
![Page 1: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/1.jpg)
1
Continuous IntegrationFor
Business IntelligenceAnd Analytics
Alex MeadowsPrincipal ConsultantCSpring
![Page 2: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/2.jpg)
2
Agenda
● About Me
● Waterfall vs Agile
● Continuous Integration
● Software Release Process
● References
Feel free to ask questions at any point in the presentation!
![Page 3: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/3.jpg)
3
Alex Meadows
● Worked in various industries (manufacturing, social/email marketing, consulting, software, agriculture)
● Open source agilest
● Loves data :)
![Page 4: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/4.jpg)
4
Let's Talk Releases...
●What's your team's release candance?●How many bug releases go out afterwards?●How much new development gets cut?
![Page 5: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/5.jpg)
5
Waterfall Approach
Source: http://covalentmarketing.com/blog/2013/02/25/software-development-lifecycle-waterfall-vs-agile/
![Page 6: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/6.jpg)
6
Agile Approach
Source: http://www.mountaingoatsoftware.com/system/asset/file/17/ScrumLargeLabelled.png
![Page 7: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/7.jpg)
7
Continuous Integration
● Release Early, Release Often!
● Bundle work together into cohesive “builds”
– Server Configuration
– Code
– Database
– Tests, tests, and more tests
– Approval Process
![Page 8: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/8.jpg)
8
Continuous Integration
● Agile XP principal since 1994
● Used in software everywhere*
– Google● Tests 100 million test cases every day● Entire code base changes 50% monthly
– Amazon, Yahoo, Microsoft, etc.
* Used in mature agile software shops
![Page 9: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/9.jpg)
9
Software Release Process
Source: http://www.360logica.com/blog/2014/06/continuous-integration.html
![Page 10: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/10.jpg)
10
Plan
● Break tasks out into manageable chunks
– Can that ETL code be broken down by:● Source/Target● Table● Process
– Can that dashboard be broken down by:● Object● Type of data/visual/interaction
● Business must prioritize, team must commit
![Page 11: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/11.jpg)
11
BI Challenges
● Work is difficult to break apart
– Most tools are designed for waterfall
– Complex moving components
![Page 12: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/12.jpg)
12
Code
● Team works and develops “artifacts”
– “Code” - Reports, ETL, Visualizations, Database changes, etc.
– Documentation
– Research
![Page 13: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/13.jpg)
13
Build
● Developed work is
– Run through unit tests
– Peer reviewed
– “Compiled” - tested for readiness for QA
– Pushed to QA
![Page 14: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/14.jpg)
14
BI Challenges
● Coordination across disparate tools, components
● Version control doesn't coordinate between tools
● Database version control is hard to introduce
![Page 15: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/15.jpg)
15
Test
● Verify all functionality, edge cases, and user requirements
● 10% manual, 90% automation work
● Automated gate keeper (pass/fail the build)
![Page 16: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/16.jpg)
16
BI Challenges
● Test tools not written for many components
– Database – exists, but minimalist
– ETL – database tools and etlTest (shameless plug ;)
– Visualization – Selenium
– Reports – mix of the above
– Analytics – mix of the above
![Page 17: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/17.jpg)
17
Release/Deploy
● Coordinate and push to next environment
● Approvals act as gate keeping mechanisim
● Cycle starts back and the code phase as needed
![Page 18: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/18.jpg)
18
BI Challenges
● Coordination is possible, but challenging
● Tools might not provide integration points
![Page 19: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/19.jpg)
19
Operate/Monitor
● Release into Production
● Monitor processes, logs, etc for aberrations
● Bugs and other issues are added into backlog for future releases
![Page 20: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/20.jpg)
20
References
![Page 21: Continuous integration with business intelligence and analytics](https://reader033.vdocuments.us/reader033/viewer/2022052400/55a78d8e1a28ab206e8b48e2/html5/thumbnails/21.jpg)
21
Sources
● http://www.eclipsecon.org/2013/sites/eclipsecon.org.2013/files/2013-03-24%20Continuous%20Integration%20at%20Google%20Scale.pdf
●