Supporting Quantitative Reasoning of Non-functional Requirements:

A Process-Oriented Approach

Amy Affleck and Aneesh Krishna

Department of Computing

Curtin University

Perth, Australia

amy.affleck@student.curtin.edu.au, a.krishna@curtin.edu.au

Abstract—A long standing problem in software engineeringis inadequate requirements elicitation, analysis, specification,validation and management. The lack of well defined require-ments is one of the major causes of project failure. Severalwell-known techniques and frameworks have been developedto deal with the functional aspect of requirements engineering.Recent years have also seen the emergence of frameworks thatincorporate non-functional requirements.

The Non-Functional Requirements (NFR) Framework mod-els non-functional requirements and associated implementationmethods. This paper presents a process-orientated, lightweight,quantitative extension to the NFR Framework; focusing onproviding quantitative support to the decision process and howdecisions affect the system.

Keywords-Requirements Engineering; Non-Functional Re-quirements; NFR framework

I. INTRODUCTION

Requirements engineering is defined by [1] as:

“. . . the branch of software engineering that is

concerned with the real world goals for, functions

of, and constraints on software systems. It is also

concerned with the relationship of these factors to

precise specifications of software behaviour, and

to their evolution over time and across software

families.”

[2] furthers this definition by specifying several cate-

gories of requirements, including non-functional require-

ments which are claimed to generally be more difficult to

express, analyse and verify. Research on project and system

success rates have discovered that the improper management

of non-functional requirements is a leading cause of failure

[3], [4].In an attempt to deal with this problem several frameworks

have been developed that aim to assist developers in dealing

with non-functional requirements. The NFR Framework [3]

presents a well documented, systematic approach to decision

making (in regard to non-functional requirements) that can

be applied during late requirements specification or early

design.The NFR Framework bridges the gap between require-

ments and design with respect to non-functional require-

ments. However, this framework lacks any form of quan-

titative support that preserves the ideals of the framework.

This paper proposes such a method.

Table ISTEPS OF THE EXTENDED NFR FRAMEWORK

Step Section Extension

1. Identify Softgoals N/A O2. Decompose Softgoals II-A E3. Assign Leaf-Softgoal Weights II-B A4. Identify Operationalizations II-C E5. Calculate Operationalization Scores II-D A6. Select Operationalizations II-E R7. Calculate Leaf-Softgoal Scores II-F R8. Calculate Softgoal Scores II-G R9. Calculate Attainment II-H A

The remainder of this paper is structured as follows.

Section II proposes a lightweight quantitative extension to

the NFR Framework and demonstrates its application on an

exemplar. Section III then evaluates the proposed extension

based on its completeness and ability to maintain the NFR

Framework’s ideals, this section also presents the findings

from the evaluation. Section IV outlines existing work of

a similar nature before Section V concludes the paper and

outlines future work that can be undertaken to improve the

extension.

II. EXTENDED NFR FRAMEWORK

The proposed extended NFR Framework involves several

additions, modifications and replacements to the original

framework. Table I defines the steps of the proposed ex-

tension, marked as being either original (O), addition (A),

extension (E) or replacement (R) to the NFR Framework.

Due to space restrictions readers are directed to [3] for

details on the NFR Framework. For comparative reasons

and due to its understandability the Bank System exemplar

[3, Chapter 2] has been reworked throughout the following

section in order to demonstrate the proposed extension. The

quantitative values are based on the original case study

description and are comparative in nature.

A. Decompose Softgoals

The identification and decomposition process is extended

so the contribution a child has on its parent can be weighted

according to Table II. The assigned weight should reflect

the percent to which a child contributes to its parent [5],

[6]. This contribution is hereafter referred to as, SGcontrib.

978-1-4673-2352-9/12/$31.00 c© 2012 IEEE ICSSP 2012, Zurich, Switzerland88

Table IICONTRIBUTION RELATIONSHIPS FOR THE SIG

Symbol Name Contribution

AND 1NumChild

OR 1

++Make 1

+Help (0,1)

--Break -1

-Hurt (-1,0)

Bank System Exemplar: Figure 3 shows there is only

one softgoal decomposition that requires weighting: the

relationship between the leaf-softgoal Accurate[Accounts]

and its parent.

B. Assign Leaf-Softgoal Weights

Next the leaf-softgoals are assigned a weight in the range

[0.0, 1.0]; zero is a non-critical softgoal with no impact on

decisions while 1 is a critical softgoal with a significant

impact on decisions. This weight is referred to as LSGweight

and allows quantitative priorities, it is based on [5], [7].Bank System Exemplar: The priority softgoal Accurate

[Accounts] is assigned a weight of 1 while the remaining

softgoals are assigned weights within the allowable range

(see Figure 3).

C. Identify Operationalizations

The relationships between leaf-softgoals and operational-

izations are weighted in a similar manner to the contributions

defined in Table II. In this case a weight of zero implies

no relationship exists. This relationship is referred to as

(impactLSG×OP ) and should reflect the percentage to which

the operationalization satisfices the softgoal, based on [5]–

[7].Bank System Exemplar: The identification of opera-

tionalizations only requires each impact be associated with a

weight (see Figure 3). The assigned weights are based on the

previous qualitative relationships and information presented

as claims in the original study.

D. Calculate Operationalization Scores

Equation 1 is applied top down through the operationaliza-

tions and calculates the individual operationalization scores

(based on [5], [7]).

OPscore = OPparent.score +∑

LSG impactLSG×OP × LSGweight

(1)

OPscore represents the degree to which the operational-

ization would affect the system should it be implemented.

Allowing the weight of a leaf-softgoal to affect the score

allows priorities to influence the final score (see Figure 1).

OPparent.score is included as a child indirectly impacts the

system in the same manner as its parent. In cases where an

operationalization has no parent a score of zero should be

substituted.

Figure 1. Effect different leaf-softgoal weight and impact combinationshave on the operationalization’s score

And(A) Direct(D) Or(O)

Figure 2. Overlap of operationalization classifications

Bank System Exemplar: Figure 3 results from applying

the score calculation, an example of the calculation for

Authorize access to account information is shown below,

the resulting score of 0.72 indicates it has a positive effect

on the system.

OPscore = 0.8× 0.9

= 0.72

E. Select Operationalizations

The OPscore is used in determining if the operationaliza-

tion should be accepted or rejected. The OPscore can fall

into three ranges; a score of 0 implies there is no effect on

the system, a score in the range (0,∞) implies it is beneficial

to the system, and a score in the range (-∞, 0) implies it is

detrimental to the system. For this stage of the process there

are three classifications:

• Direct (D) — These are operationalizations that di-

rectly impact a leaf-softgoal.

• And (A) — These are operationalizations that are clas-

sified as children and form part of an AND contribution.

• OR(O) — These are operationalizations that are clas-

sified as children and form part of an OR contribution.

These classifications are treated as sets and while they are

complete they are not necessarily disjoint (see Figure 2).

Additionally a set Siblings (S) is defined - siblings are a

part of the same contribution and form complete and disjoint

coverage of the A and O sets.1) Direct (D) Operationalizations : Direct (D) opera-

tionalizations are accepted if they are beneficial to the system

as defined by the scores relation to zero (see Equation 2)

∀OP ∈ D s.t. (OPscore ≥ 0 → OPaccept = true)∪(OPscore < 0 → OPaccept = false) (2)

89

2) And (A) Operationalizations : And (A) operationaliza-tions are accepted the same as D operationalizations with

one key difference. As the rejection of an A operationaliza-

tion means the accepted parent is denied and a contradiction

has occurred hence the developer must step in and make a

decision as signified by dalert (see Equation 3). The most

likely cause of this is negative implicit relationships.

∀OP ∈ A s.t. (OPscore ≥ 0 → OPaccept = true)∪(OPscore < 0 → dalert) (3)

3) Or (O) Operationalizations : Due to the nature of an

OR contribution, two equations are required to determine

selection. Equation 4 ensures accepted children are of addi-

tional benefit to the system. This applies tighter restrictions

on selection and hence selected operationalizations could

become rejected.

∀OP ∈ O : (OPscore ≥ OPparent.score → OPaccept = true)

∪ (OPscore < OPparent.score → OPaccept = false) (4)

Equation (5) requires the use of S, for each subset the

maximum OPscore is determined and accepted in order

to maintain the OR relationship. As before if the score is

negative the developer must take action.

∀S ⊆ O, ∃OP ∈ S, s.t. OP = maxscore(x)x∈S

,

(OPscore ≥ 0 → OPaccept = true)∪(OPscore < 0) → dalert) (5)

Bank System Exemplar: The operationalization selec-

tion is shown in Figure 3, the first step in achieving this

is to accept the D operationalizations. Based on Equation

2 if the score is positive the operationalization is accepted.

The next step is to address A, the parent (Authorize Access

to Account Information) has been accepted so its children

must also be accepted. As there are no negative scores no

developer input is needed. Finally the O relationship can be

dealt with, as only the child with the maximum score need be

accepted, and the maximum is positive Compare Signature

is accepted. When there are multiple maximums which one

is chosen would be a matter of algorithm implementation.

F. Calculate Leaf-Softgoal Scores

In order to calculate the leaf-softgoal score (LSGscore)

the method used by [5] has been adapted to allow for a

single impactor (contribution).

LSGscore = max(min(∑

OPaccept=true impactLSG×OP , 1),−1)

(6)

The score is capped at [-1.0, 1.0] as it reflects the percentage

to which the softgoal has been satisficed. This cap is

necessary as there could be several positive (or negative)

contributions to a leaf-softgoal.

Bank System Exemplar: The final leaf-softgoal scores

can be seen in Figure 3, following is a demonstration of the

score calculation for Response Time [Accounts].

LSGscore = max(min(1.0, 0.7 + 0.8 + (−0.1)),−1.0)

= max(min(1.0, 1.4),−1.0)

= 1.0

By achieving a score of 1.0, Response Time [Accounts] can

be said of be fully satisficed. This process is repeated for

each leaf-softgoal in the SIG.

G. Calculate Softgoal Scores

The leaf-softgoal scores are propagated to calculate the

score for each individual softgoal. The propagation algo-

rithm has been influenced by the logic of [3], [4], [6].

SGscore = max(min(∑

SGchild × SGchild.contrib, 1),−1)(7)

Each child contributes to a percentage of the parent’s score,

hence the child contributes x% of their own score towards

the parent’s attainment. The value depicted by x% was

introduced in Table II. Due to make, break, help and hurt

contributions the range of the softgoal’s score is capped at

[-1, 1].Bank System Exemplar: Figure 3 shows the final cal-

culated scores, below is as example of the calculation for

Good Performance [Accounts].

SGscore = (−0.5× 1

2) + (1.0× 1

2)

= −0.25 + 0.5

= 0.25

H. Calculate Attainment

Finally the optimal and actual attainment scores can be

calculated. Equation 8 calculates the optimal attainment

score, that is the non-functional requirements have been

satisficed to their full extent (based on the weight/attainment

relationship used by [5]).

attainmentoptimal =∑

LSG

LSGweight (8)

Equation 9 calculates the actual attainment based on the

percentage to which the leaf-softgoals have been attained.

Limiting contribution to [0, LSGweight] prevents a leaf-

softgoal from contributing more than was contributed in

Equation 8.

attainmentactual =∑

LSGmax(LSGscore × LSGweight, 0)(9)

There are several potential uses for the optimal and actual

attainment scores; the most obvious being the ability for

developers to see the system wide repercussions of their

decisions. Alternatively, the attainment scores for different

implementations can be compared in order to determine the

best system wide implementation. Additionally, the scores

provide developers with metrics that can be used for decision

support or verification.

90

GoodPerformance[Accounts]

Space[Accounts]

ResponseTime[Accounts]

Secure[Accounts]

Integrity[Accounts]

Complete[Accounts]

Accurate[Accounts]

Accurate[Accounts]

1

Confidentiality[Accounts]

Availability[Accounts]

w:0.3 w:0.5

w:0.7

w:1.0

w:0.9

w:0.6

User-friendly[AccountAccess]

w:0.8

UseUncom-pressedFormat

-0.5 0.7

Use Indexing

0.8

AuthorizeAccess toAccountInformation

0.8

Validate ac-cess againsteligibilityrules

-0.10.8

IdentifyUsers

AuthenticateUser Access

Use P.I.NCompareSignature

Require Addi-tional ID

-0.9

DuplicateServer

0.8

MirrorDatabase

0.5

0.2 0.4

0.72

1.47

0.720.72

0.72 0.720.0

0.480.3

-0.5 1.0

0.0

0.8

0.8

1.0

0.00.25

0.8

0.4

0.73

Figure 3. Final SIG for the Bank System exemplar

Figure 4. Comparison of Opera-tionalization score and selection

Bank System Exemplar: In order to demonstrate the

attainment calculation, the required data is presented in

Table III. This table outlines that the optimal score is 4.8

and the actual score is 2.62. Such a large discrepancy is

due to the fact that only a small portion of the system

was examined, hence operationalizations for several key

softgoals have not been considered.

Table IIILEAF-SOFTGOAL WEIGHTS, SCORES, CAPPED SCORES AND TOTALS

USED IN ATTAINMENT CALCULATIONS

Leaf-Softgoal Weight Score Weight × CappedScore Value

Space[Accounts] 0.3 -0.5 -0.15 0.0Response Time[Accounts] 0.5 1.0 0.5 0.5Complete[Accounts] 0.7 0.0 0.0 0.0Accurate[Accounts] 1.0 0.8 0.8 0.8Confidentiality[Accounts] 0.9 0.8 0.72 0.72Availability[Accounts] 0.6 1.0 0.6 0.6User Friendly 0.8 0.0 0.0 0.0[Account Access]

Total 4.8 2.62

III. SIMULATION AND EVALUATION

In order to evaluate the proposed framework it was

assessed on its ability to comprehensively and quantitatively

extend the NFR Framework. To this end a simulation

was created and the results analysed for discrepancies and

outliers [6]. Five sets of test data were generated based

on possible relationships and the valid range of values

maximising the number of combinations explored.

The results of the simulation were analysed according to

two criteria. Were the operationalizations correctly accepted

or rejected given their impact on the system and decompo-

sition relationships? How were these decisions reflected in

the leaf-softgoal scores?

A. Operationalization Selection

Figure 4 compares the selection of each operationalization

with its score; discrepancies can be easily detected as accept-

ing an operationalization with a negative score or rejecting

one with a positive score (the shaded areas). Comparison

of the results and the original SIG discovered that all the

discrepancies were the result of one of two situations.

The first situation is an accepted operationalization having

a negative score. In these cases the operationalization in

question was accepted to satisfice accepted parents. While

this would normally be a developer decision for ease of

use the simulation was designed to automatically accept

such situations. The second situation applies to a rejected

operationalization having a positive score. In these cases

the operationalization in question were part of the O set

and rejected according to Equation 5.

B. Leaf-Softgoal Scores

Figure 5 outlines the frequency of leaf-softgoal scores,

demonstrating the a significant number of leaf-softgoals

achieved their maximum score. This presents an interesting

issue in that if a softgoal has already been satisficed, addi-

tional resources and effort should not be expended to over-

satisfice it. This over satisfaction could cause negative effects

on other softgoals and subsequently the system, methods of

addressing this are proposed in Section V.

91

Figure 5. Leaf-softgoal score distributions for the five test cases

C. Simulation Findings

The test cases also assessed if the proposed extension

would continue to function correctly given any possible SIG.

The analysis of the results found that any discrepancies

were the result of satisfying the rules of the extension or

due to developer decisions enacted by the simulation. While

the pattern observed from leaf-softgoal score distributions

presents an interesting possibility for future work, it does not

invalidate the soundness or completeness of the reasoning.

D. Comparison with the Original NFR Framework

In undertaking several exemplars it was garnered that the

proposed extension was comparative in decision results until

the application of the propagation algorithm. However, as

outlined in Section II the propagation algorithms are focused

on achieving different goals and that the quantitative data

was more useful.

E. Findings

While the proposed extension can potentially be applied

to any system that the original framework can be applied

to, it may not always be appropriate to do so. Further

exemplars discovered that one-to-one mapping of softgoals

and operationalization had little use of the extension as no

decisions were made or compromises needed. Additionally

the higher number of trade-off’s present in the SIG (many-

to-many mappings) the more useful the extension became

in assisting developers.

IV. BACKGROUND

[7] integrates the concepts of softgoals and opera-

tionalizations into the KAOS framework and proposes a

lightweight quantitative solution. However, while it opti-

mises usability the solution does not calculate a requirements

attainment level or support all available decompositions.

[6] introduced quantitative reasoning with goal models,

based on first order logic and mathematical reasoning.

While comprehensive the solution requires strong back-

ground knowledge.

Defect Detection and Prevention (DDP) is a quantitative

risk management framework that bridges the gap between

requirements specification and early design [5]. It is based

on identifying relationships between requirements, failures

and prevention methods allowing the development team

to measure the effect of design decisions. DDP presents

several concepts that are worth consideration, including

requirements satisfaction and attainment level calculations.

V. CONCLUSIONS AND FUTURE WORK

This paper proposed a quantitative extension to the Non-

Functional Requirements Framework and applied the ex-

tension to the Bank System exemplar. The extension was

evaluated and the results of the extension were found to

be consistent and acceptable across a range of randomly

generated test cases. Finally, an evaluation of exemplars

discovered no differences in decisions between the original

framework and the proposed extension. Furthermore, propa-

gation discovered that the proposed extension provided more

detailed feedback then its qualitative alternative.

The success of any framework can be determined by

the effort it takes to apply in real world situations, hence

frameworks are often accompanied by C.A.S.E tools. As

such the algorithms developed for the simulation should

be integrated into one of the existing open-source C.A.S.E

tools. In order to deal with the issues presented in Section III

the operationalization selection process could be modified

based on optimising leaf-softgoal, softgoal or attainment

scores; this would have a positive benefit on effort, time

and cost, all factors which directly effect project success

[3], [4].

REFERENCES

[1] P. Zave, “Classification of research efforts in requirementsengineering,” ACM Comput. Surv., vol. 29, pp. 315–321, De-cember 1997.

[2] B. Nuseibeh and S. Easterbrook, “Requirements engineering: aroadmap,” in Proceedings of the Conference on The Future ofSoftware Engineering, ser. ICSE ’00. New York, NY, USA:ACM, 2000, pp. 35–46.

[3] L. Chung, B. A. Nixon, E. Yu, and J. Mylopoulos, Non-Functional Requirements in Software Engineering. KluwerAcademic Publishers, 2000.

[4] E. Letier and A. van Lamsweerde, “Reasoning about partialgoal satisfaction for requirements and design engineering,”SIGSOFT Softw. Eng. Notes, vol. 29, pp. 53–62, October 2004.

[5] M. Feather and S. Cornford, “Quantitative risk-based require-ments reasoning,” Requirements Engineering, vol. 8, pp. 248–265, 2003.

[6] P. Giorgini, J. Mylopoulos, E. Nicchiarelli, and R. Sebastiani,“Reasoning with goal models,” in Conceptual Modeling ER2002, ser. Lecture Notes in Computer Science, S. Spaccapietra,S. March, and Y. Kambayashi, Eds. Springer Berlin /Heidelberg, 2003, vol. 2503, pp. 167–181.

[7] A. van Lamsweerde, “Reasoning about alternative require-ments options,” in Conceptual Modeling: Foundations and Ap-plications, ser. Lecture Notes in Computer Science, A. Borgida,V. Chaudhri, P. Giorgini, and E. Yu, Eds. Springer Berlin /Heidelberg, 2009, vol. 5600, pp. 380–397.

92