integrate v&v within scrum: how does that work?
TRANSCRIPT
Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin
Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. SAND NO. 2011-XXXXP
V&V and Scrum V&V has to be Agile too!
Agenda
Introduction
How to Fold V&V into Scrum
Demo of V&V Navigator
Currently Government Use tool
2
Introduction
Julie V&V (software and M&S)
Analyst, Model Developer
Kathy Certified PO, Certified SM, Scaled Agile Framework Agilist)
Software Development, V&V
Sandia National Laboratories Albuquerque, NM
3
Introduction
Why we are doing this?
We are passionate about Agile and Scrum.
We are passionate about V&V and especially M&S V&V.
It is a bad idea to develop using Scrum and then throw V&V in at the end.
We want the application of V&V, which is traditionally very “waterfall-ish” to be agile.
4
What V&V Is and Isn’t
IS Required for development of systems,
software, and models
Based on best engineering practices
Iterative – Plan, Act, Assess
ISN’T Prescriptive
One size fits all
Easy
5
Our definitions Verification – ensuring that the software is implemented correctly
Validation – ensuring that model assumptions, theory, algorithms and results are correct/reasonable
Note: This talk isn’t a forum for argument about the Vs.
Likewise, we won’t be explaining Scrum terms.
So… How do we make V&V Agile?
Plan Development Work
Plan V&V Work
Merge both plans Into Scrum Roadmap, Epics & Stories
Execute both incrementally Perform development activities
Create development artifacts, e.g. code, documentation
Perform V&V activities
Create V&V artifacts and evidence
6
So… How do we make V&V Agile?
Merge Scrum and V&V planning and execution
Use an approach to merge the two Approach 1 – Define V&V planning/execution as
stories in the product backlog
Approach 2 – Define V&V planning/execution as tasks in stories
Combine approaches 1 & 2
Other approaches
We will focus on approaches 1 & 2 in this presentation.
7
How do we merge Scrum and V&V?
Approach #1 Add V&V planning work as Epics & Stories
Add V&V activities as Epics & Stories
Backlog has embedded V&V plans and activities
8
Approach #1 Overview
Identify V&V Activities
Use a tool (V&V Navigator) or other analysis
“Front-plan” V&V Epics, as needed
Approach V&V planning incrementally
9
Example questions from
Navigator:
Activity: Review Design
• Is design consistent and
implementable?
• Are requirements
traceable to the design
and adequately
addressed by the
design?
Approach #1 Overview
Add V&V activities to the Roadmap as Epics
Will be Epics for “Do It” e.g., Data Planning
Will be Epics for “V&V” e.g., V&V Data Collection Plan
Add Stories for V&V work
Either in a separate V&V BL or merge with Product BL
Stories for “Do It” and “V&V”
10
Approach #1 - Example Select Stories for a Sprint
11
Story: Design Collection for
accumulation of Results.
Story: Develop potential new
Use Case.
Sprint Backlog
Story: Design V&V approach
for Input and Results Data
Epic Backlog
V&V of Data Analysis & Processing
Planning for Data
Use/Needs/Collection
Design Database
V&V of Data Collection Plan
Requirements and Use Cases
Other Epics
V&V of Data Verification Plan
User Interface Implementation
Story: Implement Error Path
for User Interface.
Legend
V&V stories
Other stories
Approach #1 – Example Roadmap
12
Epic Sprint 1 Sprint 2 Sprint 3 Sprint 4 to 6 Quarter 2 Quarter 3 Quarter 4
V&V of Data Analysis & Processing
Design V&V Approach for Input and Results Data Story 6 Story 9
Adopt V&V Approach for Input Data Story3
Adopt V&V Approach for Results Data Story 7
Story 8
V&V of Data Characterization Plan
Story 1
Design Database
Story 1
User Interface Implementation
Story 1 Story 2 Story 3-5
V&V of Data Collection Plan
Story 1 Story 2
Requirements and Use Cases
Develop Potential New Use Case
Planning for Data Use/Needs/Collection
Story 1 Story 2-4
Legend
V&V stories
Other stories
Approach #1 – Another Example
Tie V&V activities to Development Activities all in one combined EPIC
Application: Develop Schedule Optimization Model
V&V approach: V&V Owner determines that V&V will include Expert Review of Optimization Algorithm
Integration testing
Epic: Develop Schedule Optimization Model Story – Select Optimization Algorithm (Dev)
Story – Prototype Optimization Algorithm - noncomputational model (Dev)
Story – Expert Review of Optimization Algorithm (V&V)
Story – Develop Code for Optimization Algorithm (Dev)
Story – Integration Test Optimization Algorithm Code (V&V)
13
How do we merge Scrum and V&V?
Approach #2 Embed V&V planning into Story Acceptance Criteria
Embed V&V activities into Story Tasks
Stories have embedded
V&V plans, activities, and Acceptance Criteria
14
Approach #2 Overview
As part of Planning, Identify the V&V Activities
Include each V&V Activity as a task in a Story e.g., Peer Review of Design (task) would be included in Design (story)
Drive V&V activities via Story Acceptance Criteria
15
Approach #2 - Example Acceptance Criteria In Stories
16
Story #: Design Collection for accumulation of Results Acceptance Criteria: Design Checked-in V&V Criteria: Peer Review of Design Complete V&V Owner Review
Story #: Meeting to discuss potential new Use Case Acceptance Criteria: Meeting held, notes distributed and in repository. V&V Criteria: none
Epic Backlog
Planning for Data
Use/Needs/Collection
Design Database
Requirements and Use Cases
Other Epics
User Interface Implementation
Story #: Implement Error Path for User Interface Acceptance Criteria: Implementation Checked-in; User Guide Updated V&V Criteria: Unit Test Pass, Regression Tests Pass, Test Plan Updated
Approach #2 - Acceptance
• Product Owner
Accepts completion of story according to the Acceptance Criteria when V&V tasks are included
Accepts completion of story including V&V Criteria met when V&V Criteria is included as part of the Acceptance Criteria
17
Considerations For Each Approach
Approach #1 – V&V embedded in the BL as stories V&V stories are visible
PO is responsible for V&V stories
Story Acceptance Criteria is explicit for V&V activities
V&V activities could be overlooked due to there not being an enforced 1:1 relationship between development and V&V
Approach #2 – V&V embedded in story tasks Team is responsible for tasking V&V activities
V&V is directly related to development activities
Coordination/effort required to embed the right activities as tasks in a story
V&V tasks could become box-checking-exercise instead of a useful task.
18
Consider a V&V Champion
Identify a person responsible for Ensuring V&V is addressed throughout development
Understanding how to merge Scrum and V&V
Assisting with selection of V&V methods (V&V Navigator lives here)
The V&V Champion can be the Product Owner The roles require different skill sets
V&V could be driven by the organization, not the customer
A designated V&V Champion is especially important with Approach 2 to ensure that V&V isn’t overlooked
19
V&V Navigator
Assists planning for V&V of Modeling and Simulation (M&S)
Any portion of the model includes theory or unknown science
Some output results are intentionally variable (e.g., optimization, prediction of future)
Where all answers/outputs are not known or agreed upon
The Process Step flow identifies what to plan for and what to do
21
Demonstration
V&V Navigator v. 2.0 can be distributed for Government Use.
Navigator
Final Thoughts
Plan Development Work
Plan V&V
Combine both plans Into Scrum Roadmap, Epics & Stories
Execute plans as per Roadmap Perform development activities
Perform V&V activities
Assess
Consider involving a V&V Champion!
22
QUESTIONS?
23
