qualitative analysis pros & cons by: kevin zarem, mai appraisal methods

QUALITATIVE ANALYSIS

PROS & CONS

by: Kevin Zarem, MAI

Appraisal Methods

http://www.rybacki.com/metro/index.html

What is qualitative analysis?

05/14/2013 2Appraisal Methods


Qualitative adjustments are purely relative (inferior, similar and superior).

Quantitative adjustments use specific numbers (% or dollar amounts).


Note: percentages as dollar amounts may look very different to the lay reader

Comp 2: (50% of $1.54/SF = $0.77/SF)

Comp 3: (-20% of $3.69/SF = -$0.74/SF)




Most quantitative adjustments appraisers make are simply qualitative adjustments presented as percentages. They are not directly supported by market data.

We should quantify adjustments based on market data whenever possible.


No difference exists in terms of research or valuation between qualitative and quantitative analysis.

The two techniques simply represent two forms of presentation.



As appraisers, we should select the presentation style that best communicates our analysis without being misleading. The best technique may be different from one data set to the next.



In 1993, in the case, Daubert v. Merell Dow Pharmaceuticals, Inc., the Supreme Court instructed trial judges to serve as “gatekeepers” in determining whether the opinion of a proffered expert is based on scientific reasoning and methodology.

Daubert Case


The words “scientific reasoning and methodology” arguably do not apply to the typical appraiser’s sales comparison technique. We work with industry standards and accepted analysis techniques.

Pertinent Daubert Criteria for Discussion



One of the specific criteria coming out of the Daubert case is whether or not the technique the expert used is generally accepted in the relevant scientific community.

Qualitative sales comparison analysis is clearly an accepted technique in the appraisal community.



The Appraisal Institute supports the use of qualitative analysis.

My informal survey results also clearly highlight that qualitative analysis is a relatively common and accepted technique – range from 0% to 90% with 33% average use reported.


Selected Quotes from the Survey

Question: In your opinion, what are the primary strengths of qualitative analysis?

“More intellectually honest, since the percentage adjustments are artificial and imply more precision than is really the case.”

“The strength of qualitative analysis is that it more accurately models the market…”



“It is a way to make more credible comparisons of otherwise subjective information. It may help if the report is challenged under the new Daubert standard as not being sufficiently reliable, and it avoids the appraiser having to admit that his 20% adjustment is based simply on ‘my 25 years of experience.’”



“A more honest reflection of what we know (i.e., limitations on “science”). When explained to lay people, makes more sense as to how markets actually operate. Best used where we have many and bracketing comps – allows us to expand the range of the comps and so show how our subject fits into a wider context.”



“When an appraiser is completing a string of condemnation appraisals all in a row, this method allows the appraiser to make the appropriate adjustments without having to justify specific percentage adjustments in exact relation to all the other subject appraisals for adjacent property owners.”



“I feel that grids that use the concept of superior/similar/inferior are not used as often as they should be. Many appraisers make outlandish adjustments that have no quantifiable source. I also believe that there are few pure quantifiable aspects to an appraisal. Some of those might be market adjustments, expenditures immediately after acquisition, conditions of sale, or cash equivalency. Others are quantified for sake of associating a percent as a relative term of comparison.”



“More simplistic, which may be of benefit if an appraisal is used for trial purposes.”

“It reflects the most typical method that buyers use. Only appraisers analyze and quantify adjustments. I like to use it for a property where there are no true comparables, and I can then paint a range of values and explain where the subject property fits within the range which may be of benefit if an appraisal is used for trial purposes.”


Question: In your opinion, what are the primary WEAKNESSES of qualitative analysis?

“It makes how the appraiser reached the SF value appear to be a greater mystery than it really is. The adjustment grid gives the appearance of a reasoned, mathematical approach to comparable sales adjustments. The qualitative analysis lacks this veneer.”

“Because of the above, when used in trial, a qualitative analysis will require more explanation through testimony to justify the SF conclusion.”


“Pretty difficult to explain to a jury, and the grid that results is capable of being ridiculed a bit as having no basis in data by an effective attorney; bottom line is that I would rather have some data to support the adjustments, than the qualitative analysis, but it’s better than the old ’25 years of experience’ methodology.”

Question: In your opinion, what are the primary weaknesses of qualitative analysis?



“Jurors may give more weight to the implied additional precision of percentage analyses. The percentage adjustments make a cleaner transition to the ultimate opinion of value…”

“The big issue is whether the jury trusts the appraiser. If so, then a qualitative system makes it likely that the jury will go with, rather than tweak, the appraiser’s number. If the jury doesn’t trust the appraiser, it may make it more likely that they will disregard the opinion entirely.”



“Requires a lot of thought and discussion from the appraiser to properly conclude a value based on the range… A real problem is that a comp may be a little inferior for one factor and highly superior for another which is hard to show in the grid…”



“The weakness of qualitative adjustments is that some are going to say it’s lazy and a squishy way to arrive at a value. I would answer that the specious accuracy that come from applying quantitative adjustments to unquantifiable attributes like location, condition, building size, results in a misleading value opinion. It’s misleading because we all know that matched pairs really don’t exist and if you use a regression model to quantify percentage or dollar adjustments then I say good luck trying to explain a coefficient of determination to a reviewer, judge, or jury. The best we can do is gather impressions about the market, filter those impressions through our education and experience, and communicate using the tools at our disposal (quantitative and qualitative) to arrive at an objective and supported opinion of value.”

05/14/2013 25• Appraisal Methods


“The appraiser is instructed or directed by USPAP to provide enough information: reasonable, logical, and defensible (my terms) for the user of the report to make a decision. Sometimes it may appear that the determination of value is too vague for some readers of the report. People tend to want numbers even though the quantified percent adjustments are derived by the appraiser’s subjective inference.”



“Can be difficult to weight one factor over another, or at least to explain weighting (readers want to add up the superiors/inferiors to a net overall comparison – maybe some factors are more important than others. Some users of appraisals are so accustomed to % or $ adjustments that they don’t believe we’re actually doing our job when we present qualitative comparison – I was once grilled very hard on the stand by an attorney who insisted that I had not done a sales comparison approach when I used a relative comparison table.”


What does Appraisal Institute say about qualitative analysis?

“Qualitative analysis recognizes the inefficiencies of real estate markets and the difficulty in expressing adjustments with mathematical precision. It is essential therefore, that the appraiser explains the analytical process and logic applied in reconciling the value indications…”


All quotes from The Appraisal of Real Estate, published by Appraisal Institute



“Relative comparison analysis is the study of relationships indicated by market data without recourse to quantification. Many appraisers use this technique because it reflects the imperfect nature of real estate markets.”



“In relative comparison analysis, one key issue concerns comparable properties with a similar highest and best use that compete in a similar market as the subject. Significant differences in highest and best use may exclude a potential comparable sale from consideration in qualitative analysis, just as it would be excluded in the quantitative adjustment process.”




“Reliable results can usually be obtained by bracketing the subject between comparable properties that are superior and inferior to it… The appraiser must search the market diligently to obtain and analyze sufficient pertinent data to bracket the value of the subject.”



My Position on Qualitative Analysis

No right or wrong answer.



Main Strength

Qualitative analysis is more natural. It best matches what we really do as appraisers to gauge value when no good, quantifiable, market data support for adjustments exists. Percentage adjustments often imply more precision than is really there.



Main Weakness

In qualitative analysis, specific values are harder to hone in on for the reader of the report or jury. People like to see numbers. More narrative and subjective weighting like “slightly superior” or “significantly inferior” may help this problem.



Historically, I’ve used qualitative analysis with fair or poor data sets and percentage adjustments for everything else. As appraisers, we should select the presentation style and accompanying narrative which best communicates our analysis without being misleading.


qualitative analysis pros & cons by: kevin zarem, mai appraisal methods

Documents

qualitative adjustments

use of qualitative analysis

mai appraisal methods

appraisal community

appraisal institute

quantitative analysis

qualitative analysis

accepted analysis techniques