Many discussions of security risk analysis methodologies mention a distinction between quantitative and qualitative risk analysis, but virtually none of those discussions clarify the distinction in a rigorous way. The purpose of this 3-part series is to clarify that distinction and then show why it matters.

Definition of TermsRisk Analysis (RA) is the identification and estimation of risks. Risk identification is the process whereby one identifies the sources of risk. (In an information security risk analysis, risk identification is the identification of hazards.) Risk estimation is the process whereby one estimates the probability and utility of prospective risks. In an information security risk analysis, the probabilities of threats are often measured conditionally-conditional upon the vulnerabilities present in the asset.In other words, risk analysis answers three questions:

(1) What can happen? (In information security risk analysis, this could be reworded as, “What can go wrong?”, since information security risks are usually associated with negative outcomes.)(2) How likely is it?(3) What are the consequences? (Again, since information security only recognizes risks with negative outcomes, this question could be reworded as, “How bad could it be?)

In addition to the above standard three questions, Steven Long has convinced me that a fourth question should be added to the list:

(4) How much uncertainty is present in the analysis? (In other words, how reliable are the answers to questions 1-3?)

There are two fundamental types of risk analyses: quantitative and qualitative. Each method has pros and cons, and there is significant controversy over which approach is superior. In what is perhaps an indicator of the controversy surrounding this issue, even the definitionsof the two approaches is somewhat controversial. I have attempted to offer as neutral a definition of these approaches as possible.Many authors make the distinction between the two types of risk analyses very complicated, but the difference is really very simple.Quantitative Risk Analyses assign fixed numerical values (within a margin of error) to both the probability and utility (business impact) of an outcome; Qualitative Risk Analyses don’t. Instead, they represent both the probability and utility of an outcome using an interval scale, where each interval includes a range of numerical values

(beyond the margin of error) and each interval is typically represented by a non-numerical label (such as the words “High”, “Medium”, “Low”), not the ranges of values those labels represent.While we may draw a distinction between quantitative and qualitative RA (and in fact most security professionals do), I believe that we would be hard pressed to defend its significance, for the reasons usually given. In virtually every discussion of information security RA that I have seen, other writers assume that quantitative RA is objective and numerical while qualitative RA is subjective and non-numerical. As I argue below, however, this common view is mistaken. Both types of RA are numerical and both types are compatible with objective and non-objective estimates of probability. Moreover, within the scope of a single RA project, different methods can be used for different risks. The distinction between quantitative and qualitative RA is significant, but not due to the reasons that are typically offered.

Quantitative & Objective vs. Qualitative & SubjectiveMany authors associate quantitative methods with objectivity and qualitative methods with subjectivity. This is a false dichotomy. Consider quantitative risk analysis first. It is objective if and only if the probabilities and utilities are objective. Suppose someone subjectively assigns a probability of zero to an outcome they regard as impossible. The value of zero was subjectively assigned, but it is a precise numerical value, not a range of values, and hence is consistent with a quantitative RA. Similarly, although qualitative RA is usually associated with subjectivity, it is fully compatible with objective estimates of probability. Suppose someone uses published actuarial data about infant mortality to determine the probability of death due to Sudden Infant Death Syndrome (SIDS). Then, in a dumbed-down speech to an audience of non-experts, the researcher declares that the probability of death due to SIDS is “low.” The probability value is objective because it is based upon facts that are independent of the opinions or beliefs of persons-in this case, facts about the frequency of infant mortality. Yet because the researcher “converted” a precise numerical value into a category (“low”) that includes a range of values, the result is consistent with a qualitative RA.

Quantitative & Numerical vs. Qualitative & Non-NumericalSimilarly, many authors associate quantitative risk analysis with numerical methods and qualitative risk analysis with non-numerical methods. This distinction is not genuine, however. Both quantitative and qualitative methodologies are numerical. That qualitative risk assessments represent probability and utility with a range of numerical values is sometimes obscured by methodologies that employ scales with seemingly non-numerical labels. For example, in information security risk management, it is quite common for qualitative risk assessments to represent the probability of an outcome as either high, medium, or low-often without any attempt to define what ranges of probability values these intervals represent! Nevertheless, in order for words like “high,” “medium,” and “low” to be used meaningfully as an interval scale for all

possible probability values, they have to represent ranges of numerical values that make it possible to say that one interval (say, the “Medium” interval) is greater than another interval (“Low”).

An analogy should make this point clear. Many if not most or all people will use common expressions like, “It is hot today,” or, “It is cold outside,” usually without knowing the exact numerical temperature. But even if they don’t know the exact numerical temperature, they feel comfortable making comparisons between different temperatures (“It sure is a lot warmer outside today than it was yesterday”). Nevertheless, temperature is a numerical value, even when the person using interval labels like “freezing” or “blazing hot” doesn’t know the temperature and may not even know the exact ranges those labels represent! Along the same lines, qualitative RAs are numerical, even if their numerical nature is obscured in practice.