Quantitative Methods

Varsha Varde

Quantitative Methods

Models for Data Analysis & Interpretation: Regression Analysis

Varsha Varde 3

Cause and Effect

The Present Contains Nothing More Than The Past, and What Is Found In The Effect Was Already In The Cause.

- Henri Bergson (19th Century French Philosopher)

Varsha Varde 4

Regression Model

• A Statistical Model which Depicts the Influence of One Cardinal Variable (The Cause) on Another Cardinal Variable (The Effect).

• These Models Have a Wide Variety of Forms and Degrees of Complexity.

Varsha Varde 5

Regression

• The Step Logically Next To Correlation.

• Situation: Usually, Correlation Between Two Variables Is Not Mere Benign Association. But, It Is In Fact Causation.

• It Is a Cause and Effect Relationship, Where X Influences Y.

• X is the Cause Variable.

• Y is the Effect Variable.

Varsha Varde 6

Some Examples

Cause Effect

Movie Ticket Price Multiplex Occupancy

Machine Downtime Production

Rainfall at Night Absenteeism Next Day

R&D Expenditure Gross Profit

? ?

Varsha Varde 7

Regression

• Dictionary Says: The Act of Returning or Stepping Back to a Previous Stage.

• Query: Do Quantitative Methods Force Us to Regress instead of Progress?

• Or, Is It Back to the Future?• Answer: Statistics, Like Any Other Field,

Adopts Crazy Names Arising from Some Important Historical Events.

• Soap Opera.

Varsha Varde 8

Story of Regression

• Sir Francis Galton Studied the Heights of the Sons in Relation to the Heights of Their Fathers.

• His Conclusion: Sons of Tall Fathers Were Not So Tall and Sons of Short Fathers Were Not So Short as their Fathers.

• Path Breaking Finding: Human Heights Tend To Regress Back To Normalcy.

Varsha Varde 9

Evolution of the Term ‘Regression’

• Since Then (1880), Similar Studies on Nature and Extent of Influence of One or More Variables on Some Other Variable Acquired the Name ‘Regression Analysis’.

• In Quantitative Methods, Regression Means a ‘Cause and Effect Relationship’.

• Cause Variable = Independent Variable

• Effect Variable = Dependent Variable

Varsha Varde 10

Scatter PlotHorizontal Axis: Reasoning Scores

Vertical Axis: Creativity Scores

Varsha Varde 11

Scatter PlotHorizontal Axis: Cause Variable: Reasoning Scores

Vertical Axis: Effect Variable: Creativity Scores

Varsha Varde 12

Regression Curve Horizontal Axis: Cause Variable: Reasoning Scores

Vertical Axis: Effect Variable: Creativity Scores

Varsha Varde 13

Regression Analysis

• A Quantitative Method which Tries to Estimate the Value of a Cardinal Variable (the Effect) by Studying Its Relationship with Other Cardinal Variables (the Cause).

• This Relationship is Expressed by a Custom-Designed Statistical Formula Called Regression Equation.

Varsha Varde 14

Purpose of Regression Analysis

1. To Establish Exact Nature of Influence of Cause Variable on Effect Variable.

2. To Determine the Quantum of Influence.

3. To Estimate an Unknown Value of Effect Variable from Value of Cause Variable.

4. To Forecast Future Values of Effect Variable from Info about Cause Variable

Varsha Varde 15

Patterns of Regression Curves

• Pattern: Upward Sloping Straight Line

• Mathematical Model: Y = a + bX (b > 0)

Varsha Varde 16

Estimating Regression Parameters a & b

• Formula for Regression Coefficient b :

Mean of Products of Values – Product of the Two Means= -------------------------------------------------------------------------- Variance of Cause Variable

• Formula for Regression Constant a :

a = Mean of Effect Variable Minus b times Mean of Cause Variable

• Don’t Worry. This is the Job of SPSS.

Varsha Varde 17

Estimating Correlation Coefficient

• Recall the Formula for Correlation Coeff.

• Pearson’s Correlation Coefficient

• Formula:

Mean of Products of Values – Product of the Two Means= -------------------------------------------------------------------------- Product of the Two Standard Deviations

• Spot the Similarity and the Difference.

Varsha Varde 18

A Simple Example

Empl. No. Yrs in Co. Salary (‘000) Product

1 2 25 50

2 3 30 90

3 5 37 185

4 7 38 266

5 8 40 320

Total 25 170 911

Arith Mean 5 34

Std. Dev. 2.3 5.6

Varsha Varde 19

Regression Model

• Formula for Regression Coefficient b : Mean of Products of Values – Product of the Two Means= -------------------------------------------------------------------------- Variance of Cause Variable

(911 / 5) – (5 x 34) 182.2 – 170 12.2= ----------------------- = ------------- = ----------- = 2.30

2.3 x 2.3 5.3 5.3

• Formula for Regression Constant a : a = Mean of Effect Variable Minus b times Mean of Cause Variable = 34 – 2.3 x 5 = 22.5

• Regression Model: Y = 22.5 + 2.3 X

Varsha Varde 20

Check Goodness of the Model

Empl. No. Yrs in Co. Salary (‘000) Estimate

1 2 25

2 3 30

3 5 37

4 7 38

5 8 40

Total 25 170

Arith Mean 5 34

Std. Dev. 2.3 5.6

Varsha Varde 21

Check Goodness of the Model

Empl. No. Yrs in Co. Salary (‘000) Estimate

1 2 25 27.1

2 3 30 29.4

3 5 37 34.0

4 7 38 38.6

5 8 40 40.9

Total 25 170 170

Arith Mean 5 34

Std. Dev. 2.3 5.6

Varsha Varde 22

Concept: Error of Estimation

• Note the Difference Between the Actual Values of Effect Variable (Salary) and the Values Estimated by the Regression Model

• This is the Error of Estimation

• Less the Error, Better the Model. Ideally 0.

• Statistical Model: Y = a + b X + e

• If Correlation is Perfect (+1 or -1), e = 0.

Given below are five observations collected in a regression study on two variables, x (independent variable) and y (dependent variable).

x y2 43 44 35 26 1a. Develop the least squares estimated regression

equation.b. Estimate value of y for x=7. Varsha Varde 23

Q5.

Varsha Varde 24

Exercise: Fit a Regression Model to Reasoning & Creativity Scores

Apl No, RsnSc CrvSc Apl No, RsnSc CrvSc

01 15.2 11.9 11 8.1 6.8

02 9.9 13.1 12 15.2 13.0

03 7.1 8.9 13 10.9 13.9

04 17.9 17.4 14 17.2 19.1

05 5.1 6.9 15 8.2 10.1

06 10.0 8.8 16 10.8 15.9

07 7.2 14.0 17 12.0 12.1

08 17.1 15.8 18 13.1 16.0

09 15.2 9.7 19 17.9 19.2

10 9.2 12.1 20 7.1 11.9

Varsha Varde 25

Exercise

• Does Your Model Look Like What I Got?:Creativity Scores = 5.23 + 0.65 x Reasoning Scores + e

• Test the Goodness of Your Regression Model

• How Bad are the Errors?

Varsha Varde 26

Other Patterns of Regression Curves

• Pattern: Downward Sloping Straight Line

• Statistical Model: Y = a - bX + e (b > 0)

Varsha Varde 27

Other Patterns of Regression Curves

• Pattern: Simple ExponentialModel: Log Y = a + bX + e (b > 0)

• Pattern: Negative ExponentialModel: Log (1/Y) = a + bX + e (b > 0)

• Pattern: Upward CurvilinearModel: Y = a + b Log X + e (b > 0)

• Pattern: Downward Curvilinear• Pattern: Logistic or S Curve

Varsha Varde 28

Your Role as a Manager

• Grasp the Situation Thoroughly. (Qualitative)• Identify Related Cardinal Variables. (DIY)• Obtain Quantitative Data on Them.• Draw Scatter Plot. Your Asstt Will Do It For You• If It Shows a Pattern, Compute Correlation

Coefficient. (Use SPSS or YAWDIFY)• If It Is High (+ or -), Draw a Free Hand Curve

and Identify the Pattern of Regression Curve.• Compute Regression Parameters for the Pattern

and Fit Regression Model. (SPSS or YAWDIFY)

Varsha Varde 29

A Word of Caution

• Undertake Regression Analysis Only For Cardinal Variables.

• Select the Variables Only If You Logically Suspect Influence of One Over the Other.

• Carry Out Regression Analysis Only After Completing Correlation Analysis AND Only If The Selected Cause and Effect Variables Are Highly Correlated.

Varsha Varde 30

Simple and Multiple Regression

• Simple Regression: One Cause Variable Influences the Effect Variable.

• This is What We Focused On So Far.

• Regression Models Have a Wide Variety of Forms and Degrees of Complexity.

• Multiple Regression: Several Cause Variables Jointly Influence Effect Variable.

Varsha Varde 31

Multiple Regression

• Multiple Regression Analysis is a Method to Analyze the Effect of Joint Influence of Many Cause Variables on Effect Variable.

• Multiple Regression Model:

Y = a + b1X1 + b2X2 + - - - - +bnXn + e

• Caution: Cause Variables X1, X2, - - - -, Xn Should Not Be Inter-Correlated.

• Otherwise You Face Multicollinearity.

Varsha Varde 32

Exercise: Select Cause Variables

Cause # 1 X1

Cause # 2 X2

Cause # 3 X3

Effect Y

Machine Downtime

Labour Absenteeism

Power Outage

Monthly Production

EPS ? ? BASF Share Price

? ? ? MRP

? ? ? ManpowerRequiremt