how do software architects consider non-functional requirements - an exploratory study

How do Software Architects consider Non-Functional

RequirementsAn Exploratory Study

D. Ameller, C. Ayala, J. Cabot, X. Franch

2

Outline

• Motivation and objective

• Background

• The study

• Observations

• Conclusions and related work

3

MotivationNFRs SA

“the rationale behind each architecture decision is mostly about achieving certain NFRs”

“[NFRs] play a critical role during system development, serving as selection criteria for choosing among myriads of alternative designs”

“quality attribute requirements strongly influence a system’s architecture”

Little evidence from empirical studies support these statements

4

Objective

To conduct an exploratory study to investigate the following research question:

How do software architects deal with NFRs in practice?

5

Background

RE activities

Reqs. structure

NFR types

Arch. design Analysis Companies

Prj. leades

Prd. managers

Programmers

Users

Sw. architects

Interviews 5Interviews 11Interviews 2e-Survey 25e-Survey ?Questionnaire 15Questionnaire ?Group discus. 10Interviews 12

+ SLR in [Svensson et al. 2010]

6

BackgroundObservations:• No clear view in NFR elicitation• Cost-driven NFR quantification• Role-dependent view on NFR type’s importance• Lack of knowledge about NFR management• Vagueness of NFRs• Difficulty to test NFRs

• FRs still prevale• Need for more empirical studies

7

The Study• Type:

Exploratory study using qualitative research Seven RQs

• Target population: Software architects (or practitioners with that role) We invited 21 software organizations (Spain) We recruited 12 of them

13 interviews made

8

The Study

SCC: Software Consulting Company; SH: Software House; ITD: IT Department

9

The Study• Data collection:

Semi-structured interviews Focus: one single project Duration: ~1 hour

• 7 questions about architect profile and development methodology used in the project

• 10 questions about decision-making and NFRs Audio-taped Transcribed Checked

10

The Study• Analysis:

Two authors coded data independentlytabulation

Categories generationNVivo Software

ConsolidationGroup meetings

PresentationCounting technique

11

The Study• Limitations and mitigations:

Evaluation apprehension Confidentiality Understandability Piloting (4) Influential factors Single project Bad memory Questionnaire sent in advance The best project Ask for the most familiar Omitting data Results overviewed by respondants Researcher bias Two separated interpretations Generalize the findings We do not attempt to make

universal generalizations

12

Observations• RQ1: What is the role of the software

architect? Nobody had the “software architect” position Nomination: (experience, tec. knowledge) > architect skills 12 played other roles played in the project:

3 Project

Manager

Developer 5 Both

4

13

Observations• RQ2: Are there terminological confusions on

NFRs? Inability. “availability”, “accuracy”, “sustainability” Confusion. “ergonomic” instead of “easy to use” Incorrectness. “Maintainability is very important, because

when something is working, we can’t make changes” Worsened by the Spanish translation

14

Observations• RQ3: What types of NFRs are relevant to

software architects?

Perfo

rman

ce

Usabil

ity

Secur

ity

Availa

bility

Inte

rope

rabil

ity

Main

tena

nce

Accur

acy

Fault

Tolera

nce

Reusa

bility

Scalab

ility

Mod

ularit

y

Mon

itorin

g

Porta

bility

0

1

2

3

4

5

6

7

8

9

10

49 Software qualities

DomainTacit

15

Observations• RQ3: What types of NFRs are relevant to

software architects?

Licen

sing

issue

s

Techn

ologic

al po

lices

Client

's NT re

quire

men

ts

Costs

Exter

nal r

egula

tions

Availa

bility

of s

uppo

rt

Organ

izatio

nal p

olicie

s0

1

2

3

4

5

6

7

8

9

10

33 Non-technical reqts.

16

Observations• RQ4: How are NFRs elicited?

All architects considered elicitation as a gradual process… also never-ending

10

3 Mainly architect

User and architect

OutsourcedProjects

17

Observations• RQ5: How are NFRs documented?

9

No documentation

3 Some kind of strategy

1 Plain text

“I rarely document my projects because it costs money”

VolereISO/IEC 9126-basedAd-hoc

Documentation not always evolved!

18

Observations• RQ6: How are NFRs validated?

NFRs had been met by the end of the project?

11

2 Yes Not “yes”

Very vague justifications

“We wait for the client to complain”

19

Observations• RQ6: How are NFRs validated?

What NFRs were validated?

8

Efficiency Accuracy Usability Reliability 4 Unknown

1 None

20

Observations• RQ7: What type of tool support for NFRs is

used? Architects did not use any specific tool support for

NFR management

21

Observations• RQ7: What type of tool support for NFRs is

used? What about NFR-driven MDD (cf. [Ameller RE’10])

5 Not at all

4 Not viable

2 Just support

2 Not answer

“I would not trust”

22

Conclusions• Contributions

Corroborating previous evidences

Architects…performed other activities simultaneouslydid not share a common vocabularyshowed some misunderstandings and lack of knowledgeconsidered performance and usability as most importantmainly elicited NFRs iterativelymore often than not, didn’t documentvalidated just a few typeswere not enthusiastic about advanced tool support

23

Conclusions• Contributions

Finding previously unreported observationsArchitects…

considered NTRs almost as important as quality reqts.mainly elicited NFRs by themselvesclaimed that NFRs were satisfied at the end of projectdid not use any specific tool for NFR management

24

Conclusions• Contributions

Finding some misalignments or even contradictions with previous studies

Architects…did not exist as a differentiated rolequantification of NFRs was poor

25

Conclusions• Future work

Replication; consolidation with other studies• E.g., study on OSS domain [REFSQ’12]

Changing the conditions• E.g., influence of an starting architecture (Ferrari et al, REJ10)

• More research needed on: Cost-benefit analysis (e.g., documentation) Highly customizable NFR management (e.g., existence

of architect role) Exploring other communication channels (e.g., blogs)

Hope youliked it!

Contact: Xavier Franch<franch@essi.upc.edu>