findability through traceability - a realistic application of candidate trace links?
Post on 13-Sep-2014
131 views
DESCRIPTION
Conference presentation from ENASE 2012 in Wroclaw, Poland. Abstract: Since software development is of dynamic nature, the impact analysis is an inevitable work task. Traceability is known as one factor that supports this task, and several researchers have proposed traceability recovery tools to propose trace links in an existing system. However, these semi-automatic tools have not yet proven useful in industrial applications. Based on an established automation model, we analyzed the potential value of such a tool. We based our analysis on a pilot case study of an impact analysis process in a safety-critical development context, and argue that traceability recovery should be considered an investment in ndability. Moreover, several risks involved in an increased level of impact analysis automation are already plaguing the state- of-practice workflow. Consequently, deploying a traceability recovery tool involves a lower degree of change than has previously been acknowledged.TRANSCRIPT
Findability through Traceability
- A Realistic Application of Candidate Trace Links?
Markus Borg
Software Engineering Research GroupDept. of Computer Science
Lund University / [email protected]
• MSc. Computer science and engineering
• Software engineer (2007 - 2010)• PhD Student Software Engineering (2010- )• Research interests
• Large-scale development• Traceability• Information management
Lund University / [email protected]
Outline of the Presentation
• Industrial case– Work task in safety-critical development
• Previously proposed solution (>100 papers)– Decision support using semi-automated trace
recovery• Impact on the case
– Automation Analysis– Work flow: Pre auto. vs. post auto.
• Conclusion
Lund University / [email protected]
Definitions
A trace link is a specified association between a pair of software artifacts.
Trace recovery is the process of establishing trace links among existing software artifacts.
Cleland-Huang et al. (2012)
Lund University / [email protected]
Definitions (2)
Findability is the degree to which a system or environment supports navigation and retrieval.
Morville (2005)
Lund University / [email protected]
Definitions (3)
Information retrieval (IR) is finding material (usually documents) of an unstructured nature (usually text) that satisfies an information need from within large collections.
Manning et al. (2008)
Lund University / [email protected]
Case Company
• Large mutli-national company– Industrial automation
• Plan-driven development– Iterative gate-model– ~300 software engineers (Sweden and India)
• Safety-critical– Regulations mandate dev. process– V&V is vital to certify product
Lund University / [email protected]
Impact analysis
• Software changes should be analyzed before implementation
• Task integrated in issue tracker
Lund University / [email protected]
• For changes that have impact on safety parts– A question template must be filled in by the
engineer– 6 out of 13 questions require trace links
Impact analysis (2)
”List modified code files”
“Which documents need to be modified?”
“Which requirements and functions need to be retested?”
Lund University / [email protected]
Document spacePre-study
SRS
Func. spec.
System design
Detailed design
Source code
Design test spec.
Test scripts
Func. test spec.
System test spec.
IA Report
Lund University / [email protected]
Manual trace link seeking
Search similar reports
Search project docs.
Ask team
member
Q1. ?Q2. ?
.
.Qn. ?
Q1. ! Q2. !
.
.Qn. !
Impact analysis template
Impact analysis template
Lund University / [email protected]
IR-based Trace Recovery
• Information need
To what should this artifact be traced?
• Solution ideaSuggest textually similar artifacts
• AssumptionProvides a useful starting point for tracing
Lund University / [email protected]
IR-based Trace Recovery (2)
IR-based trace
recovery tool
==
Search tool with
automatically executed queries
Not very different from state-of-practice search solutions…
Lund University / [email protected]
Semi-automated trace link seeking
Search similar reports
Search project docs.
Ask team
member
Q1. ?Q2. ?
.
.Qn. ?
Q1. ! Q2. !
.
.Qn. !
Impact analysis template
Impact analysis template
Lund University / [email protected]
• Model of types and levels of human interaction with automation– 4 types– 10 levels
Analysis method (Parasuraman et al., 2000)
Lund University / [email protected]
Types of Automation
Acquire info
Analyze info
Decide action
Implement action
Not meaningful to distinguish for IR-
systems (Parasuraman et al., 2000)
Lund University / [email protected]
Levels of Automation
• From no assistance at all to a fully autonomous solution
(Parasuraman et al., 2000)Manual work
Decision support
Automated action, informs human
Automated action, not informing human
Level 1
Level 2-5
Level 6-8
Level 9-10
Lund University / [email protected]
Types and levels of automation
Acquire AnalyzeDecide
Implement
Leve
l
1
2
3
4
5
6
7
8
9
10
Lund University / [email protected]
Types and levels of automation (2)
Acquire
Analyze DecideImplement
Lev
el
1
2
3
4
5
6
7
8
9
10
Lund University / [email protected]
Type: Acquire + analyze information
• Software artifacts stored in different systems, e.g., – Requirements Databases– Test Management Systems– Source Code Repositories
• Challenges– Distributed information– Poor tool interoperability (i.e., information silos)– Finding relevant information
Lund University / [email protected]
Acquire + analyze information (2)
Pre-automation• Engineers must access
different information systems
• Engineer manually enters queries in search tools
• Possibly support for Boolean search operations, filtering etc.
Post-automation• Trace recovery tools must
access different information systems
• Tool automatically executes search queries
• Possibly support for filtering
Lund University / [email protected]
Acquire + analyze information (3)
IA Report
Reqs. DB
Test DB
DMS
Lund University / [email protected]
Acquire + Analyze information (4)
Acquire
Analyze DecideImplement
Lev
el
1
2
3
4
5
6
7
8
9
10
Post auto
Pre auto
Lund University / [email protected]
Type: Decide on Action
• Candidate trace links found• Decide which to report in the impact analysis report
Pre-automation• Engineers decide which
trace links to report
Post-automation• Engineers decide which
trace links to report
Lund University / [email protected]
Decide on action (2)
Acquire
Analyze DecideImplement
Lev
el
1
2
3
4
5
6
7
8
9
10
Post auto
Pre auto
Lund University / [email protected]
Type: Action implementation
• Report trace links in impact analysis report
Pre-automation• Engineers manually fill in
impact analysis report using free text
Post-automation• Engineers establish trace
links in trace recovery tool• Tool represents artifacts
and trace links as a network
Q1. ! Q2. !
.
.Qn. !
Lund University / [email protected]
Action implementation (2)
Acquire
Analyze DecideImplement
Lev
el
1
2
3
4
5
6
7
8
9
10
Post auto
Pre auto
Lund University / [email protected]
Deploying a trace recovery tool
• An IR-based trace recovery tool would increase the level of automation...
Instead, other types of automation are addressed
- but not mainly targeting decision making
Lund University / [email protected]
Deploying a trace recovery tool (2)
• Consequently– less change involved than has been
acknowledged– risks related to decision making already present– should be combined with manual searching
• An IR-based trace recovery tool should mainly be considered a way to increase findability– A complementary search tool rather than a
semi-automatic trace tool
