Selection Methods
Rev: Nov, 2013
Euiho (David) Suh, Ph.D.
POSTECH Strategic Management of Information and Technology Laboratory(POSMIT: http://posmit.postech.ac.kr)
Dept. of Industrial & Management EngineeringPOSTECH
Contents1 Conjoint Analysis
2 Q-Sort
3 DEA
4 Brainstorming
5 Delphi Method
6 Majority Voting
7 AHP
8 Financial Options vs. Real Options
3
Conjoint Analysis
■ What is Conjoint Analysis?– Statistical technique used in market research to determine how people value different
features that make up an individual product or service
– The objective of conjoint analysis• To Determine what combination of attributes is the most influential on respondents’ utilities
■ How to use Conjoint Analysis
Decide the important attributes and their levels
Construct the product’s profile
Collect the data (e.g., individuals rate each in terms of desirability or rank them)
Estimate the partial value using multiple regression
Apply the result to strategies (e.g., forecast market share of each product based on estimated coefficients)
Step 1
Step 2
Step 3
Step 4
Step 5
A1 A2
B1 B2
A1 A1
B1 B2
A2
B1
A2
B2
4
Conjoint Analysis – Example (1/2)
■ Organizing a Package tour product
■ Step 1: Decide the important attributes and their levels– Extract important factors through survey• Place: Southeast Asia, Australia, Guam• Price: 1.5 million won, 1.8 million won, 2.0 million won• Value added tax: Include, Exclude
■ Step 2: Construct the product’s profile– Make all possible conjoint profiles– Use fractional factorial design to reduce the number of profiles
3 X 3 X 2 X (1/2) = 9 Profiles– Profile 1: Guam, 2.0 million won, included VAT
Profile 2: Southeast Asia, 1.8 million won, included VATProfile 3: Guam, 1.5 million won, included VATProfile 4: Southeast Asia, 2.0 million won, excluded VATProfile 5: Australia, 2.0 million won, included VATProfile 6: Guam, 1.8 million won, excluded VATProfile 7: Australia, 1.8 million won, included VATProfile 8: Australia, 1.5 million won, excluded VATProfile 9: Southeast Asia, 1.5 million won, included VAT
5
Conjoint Analysis – Example (2/2)
■ Step 3: Collect the data– Let individuals rank the profile
■ Step 4: Estimate the partial value using multiple regression– Use statistic package software to estimate importance
of each factors (e.g. SPSS)– The most important attribute: Place– The most preferred place: Guam
■ Step 5: Apply the result to strategies
idpref1
pref2
pref3
pref4
pref5
pref6
pref7
pref8
pref9
1 8 3 7 5 1 6 9 2 4
2 6 9 1 3 7 5 8 2 4
3 1 6 3 9 4 2 5 8 7
4 1 8 3 9 4 2 5 6 7
5 8 9 3 7 2 6 5 4 1
6 5 2 7 9 4 8 6 3 1
7 8 6 7 4 2 3 5 9 1
6
Q-Sort (1/2)
■ What is Q-Sort?– Method that offers you systematic and quantitative means for examining human sub-
jectivity– Method that gives you insight into understanding consensus or disagreement on ideas
among your group
– Key factor• 1st. Q-samples
Statements or ideas which are needed to be ranked
statements or ideas which are needed to be ranked
• 2nd. Q-Sorting Filling the blanks of the distribution-box with Q-Samples
• 3rd. Analysis Understanding the preferences for the ideas
Disagree(-4)
Neutral(0) Agree(+4)
7
Q-Sort (2/2)
■ How to use Q-Sort
Design Q-sort model
1. Identify a topic
2. Make Q-samples to represent the spectrum of opinions on a given topic
3. Select response distribution (e.g., -4 ~ +4 or -5 ~ +5)
Execute Q-sort
1.Categorize each statement into three groups (disagreement, neutrality, and agreement)
2.Place statements to the distribution: The higher degree of agreement of statement, the more the statement is placed at right
Analyze the result
1.Analyze correlations between respondents2.Group statements - 1st. Calculate factor loadings about each factor - 2nd. Classify respondents into factors by checking the factor loadings3.Analyze normalized factor score to find significant statements
8
Q-Sort – Example (1/2)
■ Step 1: Design Q-sort model– Topic: Ability ph.D in MIS must have– Q-samples: Table 1– Response distribution
■ Step 2: Execute Q-sort– Categorize each statement into three groups
(disagreement, neutrality, and agreement)– place each statement to
the distribution model(Agree Right, Disagree Left)
– e.g.
-2 -1 0 +1 +2
5
2 8 11
7 3 6 12 14
4 10 1 9 13
-2 -1 0 +1 +2
9
Q-Sort – Example (2/2)
■ Step 3: Analyze the result– Analyze correlations between respondents– Classify respondents by calculating factor loadings– Calculate normalized score for each type of respondents– Identify characteristics of responses from a certain type
(Check the absolute value of the normalized score)
10
DEA
■ What is Data Envelopment Analysis (DEA)?– Linear programming methodology to measure the efficiency of multiple decision-
making units (DMU) when the production process presents a structure of multiple in-puts and outputs
– Nonparametric method in operations research and economics for the estimation of production frontiers
– Advantages of DEA• No need to explicitly specify a mathematical form for the production function• Proven to be useful in uncovering relationships that remain hidden for other methodologies• Capable of handling multiple inputs and outputs• Capable of being used with any input-output measurement• The sources of inefficiency can be analyzed and quantified for every evaluated unit
– How to use DEAUtilize some variable to establish the efficiency frontier
Define the efficiency frontier
A numerical coefficient is given to each firm, defining its relative efficiency
Step 1
Step 2
Step 3
11
DEA – Example
■ A, B, C, D, E: Companies producing product y with raw material x1 and x2
– Efficiency of A and D: 1– Relative efficiency of B, C, E: 0.82, 0.78, 0.75 respectively (Excessive input of x1 and
x2)– B: Benchmarking A– C: Benchmarking A and D with weights 0.64 and 0.36 respectively
Required x2 to makeone y product
Efficiency Frontiers
Required x1 to makeone y product
12
Brainstorming
■ What is brainstorming?– Group creativity technique by which efforts are made to find a conclusion for a specific
problem by gathering a list of ideas spontaneously contributed by its members
– How to use Brainstorming
Find a comfortable meeting environment, and set it up ready for the brainstorming
Appoint one person to record the ideas. These should be noted in a format than everyone can see and refer to. Depending on the approach you want to use, you may want to record ideas on flip charts, whiteboards, or computers with data projectors
If people aren’t already used to working together, consider using an appropriate warm-up exer-cies
Define the problem you want solved clearly, and lay out any criteria to be met. Make it clear that the objective of the meeting is to generate as many ideas as possible
Give people plenty of time on their own at the start of the session to generate as many ideas as possible
Ask people to give their ideas, making sure that you give everyone a fair opportunity to contrib-ute
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
In a long brainstorming, take plenty of breaks so that people can continue to concentrateStep 7
13
Brainstorming – Example
■ Colab– Computer-supported conference system
by Xerox’s Palo Alto Research Center(PARC)1) “Bordnoter” : an electronic blackboard2) “Cognoter” : brainstorming, organizing, evaluating information3) “Argnoter”: presenting, arguing, and evaluating ideas
– Design principle: information sharing,“WYSIWIS” (What you see is what I see)
■ KJ method– Card-based knowledge-creation technique– Computerization of the KJ method• Prototype example: workstation-based GrIPS by Fujitsu
Divergent thinking phase:pick up data such as key words, phrases, picturesfrom Keyword Associater/Picture Library put into “cards” on the “Shared Board”
Convergent thinking phase:Using “Card Editor”, organize the cards into pieces of information, related into a diagram using “D-ABDUCTOR”
Communicate through “Miniature meeting Room”
KJ method
Colab
14
Delphi Method
■ What is Delphi Method?– Communication technique based on a structured process for collecting and synthesiz-
ing knowledge from a group of experts by means of a series of questionnaires accom-panied by controlled opinion feedback
– Key characteristics• Structuring of information flow• Regular feedback• Anonymity of the participants
■ How to use Delphi Method
Define the problemIdentifying the problem(s) in various forms from a questionnaire to a broad and open question
Give everyone the prob-lem
Recruiting experts to the Delphi group, sending the problem(s) to everyone in the group and asking them to respond
Taking the responses that experts send back and collating these into a single anonymous list or sets of listsCollate the response
Sending the collation back out to everyone with request to score each item on a given scale (typically 1 to 5) and may allow them to add further items or commentsGive everyone the colla-
tionRepeating the rounding until a certain stopping condition meets(Number of iterations, a specific level of agreement)Repeat as necessary
Analyzing the findings and putting plans in place to deal with future risks and opportunities in the projectAct on the findings
15
Delphi Method – Example (1/2)
■ Example– Choosing the next strategy for the company with 5 experts (Stopping Condition: 3
rounds)• 1st round (Questionnaire & Scoring result)
• 2nd round (Questionnaire & Scoring result)
Strategy + CommentsScore (1 is low, 7 is high)
E1 E2 E3 E4 E5
Organizing a task force team 2 6 4 5 7
Promoting a new supervisor for the project 5 3 4 6 2
Proceeding the project as now 3 2 1 4 3
Organize a new department for the project 6 2 1 3 1
Strategy + CommentsScore (1 is low, 7 is high)
E1 E2 E3 E4 E5
Organizing a task force team 4 6 5 6 7
Promoting a new supervisor for the project+ It can be hard to find a right person for the job
3 3 4 2 2
Proceeding the project as now+ It is inefficient
3 2 2 3 3
Organize a new department for the project+ Reorganizing will be needed after the project is over+ It is premature to make a new department+ The future of project is a little gray
2 1 1 2 1
16
Delphi Method – Example (2/2)
■ Example– Choosing the next strategy for the company with 5 experts (Stopping Condition: 3
rounds)• 3rd round (Questionnaire & Scoring result)
The company choose “Organizing a task force team” strategy
Strategy + CommentsScore (1 is low, 7 is high)
E1 E2 E3 E4 E5
Organizing a task force team+ It would be a good chance to make proper people in-volved to the project regardless of their departments
6 6 5 7 7
Promoting a new supervisor for the project+ It can be hard to find a right person for the job+ It can be a threat to the current supervisor and make him upset
2 3 2 2 1
Proceeding the project as now+ It is inefficient+ The project is too big to be proceeded with the current temporary process
3 2 2 1 3
Organize a new department for the project+ Reorganizing will be needed after the project is over+ It is premature to make a new department+ The future of project is a little gray
1 1 1 2 1
17
Majority voting
■ What is Majority Voting?– A decision rule that selects alternatives which have a majority, that is, more than half
the votes
– Advantage• Being able to produce a prompt and clear decision
– Limitation• Disregarding the basic rights of the minority
– Properties of Majority Voting• Fairness
Anonymity» The decision rule treats each voter identically» When using majority rule, it makes no difference who casts a vote; indeed the voter's identity need not even
be known. Neutrality» The decision rule treats each alternative equally
• Decisiveness The decision rule selects a unique winner.
• Monotonicity The decision rule would always, if a voter were to change a preference, select the alternative that the voter
preferred, if that alternative would have won before the change in preference. Similarly, the decision rule would never, if a voter were to change a preference, select a candidate the voter did not prefer, if that alterna-tive would not have won before the change in preference
18
Majority voting – Example
■ Should the university set curfew for students living in the dormitory?
The statement will be rejected,but the opinion of the one who agreed to the statement will be ignored
19
AHP
■ What is AHP(Analytic Hierarchy Process)?– A modeling structure for representing multicriteria (multiple goals, multiple objectives)
problems – with sets of criteria and alternatives (choices) – commonly found in busi-ness environments
– Benefits• Helping capture both subjective and objective evaluation measures• Providing a useful mechanism for checking the consistency of the evaluation measures and al-
ternatives suggested by the team
■ How to use AHP
Defining policy issues and establishing policy-making mem-bers
Constructing the hierarchy layer structure of the problems
Conducting questionnaire surveys and expert preference in-tegration
Establishing pair-wise comparison matrices
Conducting consistency test
Selecting the most optimal option
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
Goal1.00
Criterion 1
0.75
Criterion 2
0.25
SubC 1
0.15
SubC 2
0.2
SubC 3
0.4
SubC 3
0.1
SubC 3
0.15
Hierarchy layer structure
20
AHP – Example
■ Example
Defining policy issues and estab-lishing policy-making members
Constructing the hierarchy layer structure of the problems
Conducting questionnaire surveys and expert preference integration(1-9 scale preference question-naire)
Establishing pair-wise comparison matrices( if i≠j; if i=j)(Weight: normalization of geo-metric means of each row; ∑Weight = 1)
Conducting consistency test
Selecting the most optimal optionby comparing each alternative’s weighted sum of criteria values
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
• Issue: Which car is the best car for you?
Choice of a Car
FunctionLevel (FL)
PriceLevel (PL)
SafetyLevel (SL)
DesignLevel (DL)
PL9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8
9SL
FL PL SL DL Weight
FL 1 5 6 9 0.641
PL 1/5 1 2 7 0.198
SL 1/6 1/2 1 5 0.123
DL 1/9 1/7 1/5 1 0.038
• FL>PL, PL>SL, FL>SL → Consis-tency
• FL>PL, PL>SL, SL>FL → Inconsis-tencyFL PL SL DL Score
Car A 80 60 60 90 73.96
Car B 60 90 80 50 68.02
21
Financial Options vs. Real Options
■ What is Financial Options?
■ How to valuate Financial Options
■ What is Real Options?
■ How to valuate Real Options
An instrument that conveys the right,but not the obligation, to engage in a future transaction with fixed price
E.g.) Call option, Put option
Financial Options The right, but not the obligation, to
undertake certain business initia-tives
To reduce uncertainty of future planes
Real Options
Prepare a number of
scenarios
Put plan into ac-tion immediately
One of the sce-
nario happen
Expand
Mathematical model of a financial mar-ket containing certain derivative in-vestment instruments
To calculate “future option value” using stock price, volatility, risk free interest
Black–Scholes
A suite of option pricing tools to quan-tify the embedded strategic value for a range of proposed or existing in-vestment scenarios
Real Options Valuation (ROV)
22
Financial Options – Example
■ Call option– Right to buy certain stocks at the fixed strike price in the future– In-the-money: Future price > Strike price• Use the call option if the difference is bigger than the price of the call option
– At-the-money: Future price = Strike price– Out-of-the-money: Future price < Strike price
■ Put option– Right to sell certain stocks at the fixed strike price
in the future– In the money: Future price < Strike price• Use the put option if the difference is bigger than
the price of the put option
– At-the-money: Future price = Strike price– Out-of-the-money: Future price < Strike price
Use the call optionGive up the put option
Give up the call optionUse the put option
23
Real Option – Example (1/3)
■ Real Option Valuation (ROV) in Mining Industry– Factors: mineral commodity price (major), uncertainty of size of the orebody,
proper timing for exploration, cut-off grade, development of the mine Uncertainty will increase the risk of investment Estimate the project by ROV
– Value of operating mine• V = V (S, Q, t, j, φ)
24
Real Option – Example (2/3)
■ Real Option Valuation (ROV) in Mining Industry (Cont’d)– Free Cash Flow to the Firm (FCFF)• Cash that a company is able to generate after laying out the money required to maintain or ex-
pand its asset base
•
25
Real Option – Example (3/3)
■ Real Option Valuation (ROV) in Mining Industry (Cont’d)– Discounted Cash flow Method (DCF)
•
• Method of valuing a project, company, or asset using the concepts of the time value of money• All future cash flows are estimated and discounted to give their present values (PVs) – the sum
of all future cash flows, both incoming and outgoing, is the net present value (NPV), which is taken as the value or price of the cash flows in question
– ROV
•
• It is more appropriate than DCF to deal with changing price of commodity by considering closing mine permanently, temporary and reopen case
26
■ Alternative Selection Methods
Conjoint Analysis
Q-Sort
DEA
Brainstorming
Delphi Method
Majority Voting
AHP
Financial OptionsReal Options
27
Discounted Cash Flow (DCF)
■ Commonly used quantitative methods include discounted cash flow methods and real options.
■ Discounted Cash Flow (DCF)– Net Present Value (NPV): Expected cash inflows are discounted and compared to out-
lays.
28
Discounted Cash Flow (DCF) Cont’d
■ Internal Rate of Return (IRR): The discount rate that makes the net present value of investment zero.– Calculators and computers perform by trial and error.– Potential for multiple IRR if cash flows vary
■ Strengths and Weaknesses of DCF Methods:– Strengths• Provide concrete financial estimates• Explicitly consider timing of investment and time value of money
– Weaknesses• May be deceptive; only as accurate as original estimates of cash flows.• May fail to capture strategic importance of project
29
Real Options
■ Real Options: Applies stock option model to nonfinancial resource investments. E.g.,with respect to R&D:
■ The cost of the R&D program can be considered the price of a call option.
■ The cost of future investment required to capitalize on the R&D program (such as the cost of commercializing a new technology that is developed) can be considered the exercise price.
■ The returns to the R&D investment are analogous to the value of a stock pur-chased with a call option.
30
The Aggregate Project Planning Framework
■ The Aggregate Project Planning Framework– Managers map their R&D projects according to levels of risk, resource commitment
and timing of cash flows