Download - Findability through Traceability - A Realistic Application of Candidate Trace Links?

Findability through Traceability

- A Realistic Application of Candidate Trace Links?

Markus Borg

Software Engineering Research GroupDept. of Computer Science

Lund University / [email protected]

• MSc. Computer science and engineering

• Software engineer (2007 - 2010)• PhD Student Software Engineering (2010- )• Research interests

• Large-scale development• Traceability• Information management


Outline of the Presentation

• Industrial case– Work task in safety-critical development

• Previously proposed solution (>100 papers)– Decision support using semi-automated trace

recovery• Impact on the case

– Automation Analysis– Work flow: Pre auto. vs. post auto.

• Conclusion


Definitions


Definitions

A trace link is a specified association between a pair of software artifacts.

Trace recovery is the process of establishing trace links among existing software artifacts.

Cleland-Huang et al. (2012)


Definitions (2)

Findability is the degree to which a system or environment supports navigation and retrieval.

Morville (2005)


Definitions (3)

Information retrieval (IR) is finding material (usually documents) of an unstructured nature (usually text) that satisfies an information need from within large collections.

Manning et al. (2008)


Description of the Case


Case Company

• Large mutli-national company– Industrial automation

• Plan-driven development– Iterative gate-model– ~300 software engineers (Sweden and India)

• Safety-critical– Regulations mandate dev. process– V&V is vital to certify product


Impact analysis

• Software changes should be analyzed before implementation

• Task integrated in issue tracker


• For changes that have impact on safety parts– A question template must be filled in by the

engineer– 6 out of 13 questions require trace links

Impact analysis (2)

”List modified code files”

“Which documents need to be modified?”

“Which requirements and functions need to be retested?”


Document spacePre-study

SRS

Func. spec.

System design

Detailed design

Source code

Design test spec.

Test scripts

Func. test spec.

System test spec.

IA Report


Manual trace link seeking

Search similar reports

Search project docs.

Ask team

member

Q1. ?Q2. ?

.

.Qn. ?

Q1. ! Q2. !

.

.Qn. !

Impact analysis template



Trace Recovery based on Information Retrieval


IR-based Trace Recovery

• Information need

To what should this artifact be traced?

• Solution ideaSuggest textually similar artifacts

• AssumptionProvides a useful starting point for tracing


IR-based Trace Recovery (2)

IR-based trace

recovery tool

==

Search tool with

automatically executed queries

Not very different from state-of-practice search solutions…


Semi-automated trace link seeking

Search similar reports

Search project docs.

Ask team

member

Q1. ?Q2. ?

.

.Qn. ?

Q1. ! Q2. !

.

.Qn. !




Automation Analysis:Method


• Model of types and levels of human interaction with automation– 4 types– 10 levels

Analysis method (Parasuraman et al., 2000)


Types of Automation

Acquire info

Analyze info

Decide action

Implement action

Not meaningful to distinguish for IR-

systems (Parasuraman et al., 2000)


Levels of Automation

• From no assistance at all to a fully autonomous solution

(Parasuraman et al., 2000)Manual work

Decision support

Automated action, informs human

Automated action, not informing human

Level 1

Level 2-5

Level 6-8

Level 9-10


Types and levels of automation

Acquire AnalyzeDecide

Implement

Leve

l

1

2

3

4

5

6

7

8

9

10


Types and levels of automation (2)

Acquire

Analyze DecideImplement

Lev

el

1

2

3

4

5

6

7

8

9

10


Automation Analysis:Results


Type: Acquire + analyze information

• Software artifacts stored in different systems, e.g., – Requirements Databases– Test Management Systems– Source Code Repositories

• Challenges– Distributed information– Poor tool interoperability (i.e., information silos)– Finding relevant information


Acquire + analyze information (2)

Pre-automation• Engineers must access

different information systems

• Engineer manually enters queries in search tools

• Possibly support for Boolean search operations, filtering etc.

Post-automation• Trace recovery tools must

access different information systems

• Tool automatically executes search queries

• Possibly support for filtering


Acquire + analyze information (3)

IA Report

Reqs. DB

Test DB

DMS


Acquire + Analyze information (4)

Acquire


Lev

el

1

2

3

4

5

6

7

8

9

10

Post auto

Pre auto


Type: Decide on Action

• Candidate trace links found• Decide which to report in the impact analysis report

Pre-automation• Engineers decide which

trace links to report

Post-automation• Engineers decide which

trace links to report


Decide on action (2)

Acquire


Lev

el

1

2

3

4

5

6

7

8

9

10

Post auto

Pre auto


Type: Action implementation

• Report trace links in impact analysis report

Pre-automation• Engineers manually fill in

impact analysis report using free text

Post-automation• Engineers establish trace

links in trace recovery tool• Tool represents artifacts

and trace links as a network

Q1. ! Q2. !

.

.Qn. !


Action implementation (2)

Acquire


Lev

el

1

2

3

4

5

6

7

8

9

10

Post auto

Pre auto


Conclusion


Deploying a trace recovery tool

• An IR-based trace recovery tool would increase the level of automation...

Instead, other types of automation are addressed

- but not mainly targeting decision making


Deploying a trace recovery tool (2)

• Consequently– less change involved than has been

acknowledged– risks related to decision making already present– should be combined with manual searching

• An IR-based trace recovery tool should mainly be considered a way to increase findability– A complementary search tool rather than a

semi-automatic trace tool


Thanks for your attention!

Download - Findability through Traceability - A Realistic Application of Candidate Trace Links?

Top Related