Applied Regression

Applied RegressionChapter 1

An brief introduction of regression analysis

Hongcheng Li

April, 6, 2013

Outline

1 Requirement

2 Introduction concepts in Regression Analysis

3 Formula

4 Steps in Regression Analysis

5 Data structure

6 Classification

Applied Regression

Requirement

1 Requirement

2 Introduction concepts in Regression Analysis

3 Formula

4 Steps in Regression Analysis

5 Data structure

6 Classification

Applied Regression

Requirement

Textbooks

Required1. Text: Regression Analysis by Example(3rd Ed), Samprit,Ali S. Hadi2. Software: SPSS or R

Optional1. Sanford Weisberg,Applied Linear Regression, Nov., 2005 byWiley2. An R Companion to Applied Regression, Dr. John Fox,Harvey Sanford Weisberg, 2010, Sage Publications.3. Web site: teach.minewin.com

Applied Regression

Requirement

List of content

1 Chapter 1 Introduction

2 Chapter 2 Simple LR

3 Chapter 3 Multiple LR

4 Chapter 4 Regression-Diagnostics

5 Midterm

6 Chapter 5 Qualitative Variables as predictors

7 Chapter 6-11 Regression related topics

8 Chapter 12 Logistic Regression

9 Final

Applied Regression

Introduction concepts in Regression Analysis

1 Requirement

2 Introduction concepts in Regression Analysis

3 Formula

4 Steps in Regression Analysis

5 Data structure

6 Classification

Applied Regression

Introduction concepts in Regression Analysis

What is Regression Analysis I

Regression analysis is a conceptually simple method forinvestigating functional relationships among variables.

1 A real estate appraiser may wish to relate the sale price of ahome from selected physical characteristics of thebuilding(like,area of the lot,area of the house, location, age ofthe house, number of bedrooms, number of bathrooms, andso on.)

Applied Regression

Introduction concepts in Regression Analysis

What is Regression Analysis II

2 whether cigarette consumption is related to varioussocioeconomic and demographic variables such as age,education, income, and price of cigarette.

3 Whether the implementation of a specific law affects the costof living for a four-person family

Applied Regression

Introduction concepts in Regression Analysis

Why Regression Analysis

Regression analysis is one of the mostly used subject in realapplications, either in real world problems or in research.

1 Almost all subjects in economics or finance that needquantitative analysis

2 Further study in Time series Analysis, advancedeconometrics(M.S or PHD) courses

3 Consulting companies

4 Big data analysis

5 cdots more

Applied Regression

Formula

1 Requirement

2 Introduction concepts in Regression Analysis

3 Formula

4 Steps in Regression Analysis

5 Data structure

6 Classification

Applied Regression

Formula

Formula IThe relationship is expressed in the form of an equation.

Y = f (X1,X2, · · · ,Xp) + ε

The function f (·) describes the relationship between Y andX1,X2, · · · ,Xp. An example is the linear regression model:

Y = β0 + β1X1 + β2X2 + · · ·+ βpXp + ε

where β0, β1, · · · , βp are called the regression parameters orcoefficients, which are TBD. X1,X2, · · · ,Xp are knownnon-random; Y ’s are random variables.

Applied Regression

Formula

Formula II

1 Y : response or dependent output variable

2 X1,X2, · · · ,Xp: explanatory, predictor, input, independentvariables, descriptive variables

3 ε the random error,which represents the discrepancy betweenthe true value and the predicted values.

4 p: the number of predictor variables.

Applied Regression

Steps in Regression Analysis

1 Requirement

2 Introduction concepts in Regression Analysis

3 Formula

4 Steps in Regression Analysis

5 Data structure

6 Classification

Applied Regression

Steps in Regression Analysis

Steps in Regression Analysis

1 Statement of the problem

2 Selection of potentially relevant variables

3 Data collection

4 Model specification

5 Choice of fitting method

6 Model fitting

7 Model validation and criticism

8 Using the chosen model(s) for the solution of the posedproblem

Applied Regression

Data structure

1 Requirement

2 Introduction concepts in Regression Analysis

3 Formula

4 Steps in Regression Analysis

5 Data structure

6 Classification

Applied Regression

Data structure

Data Structure in Regression Analysis

Observation Response PredictorsNumber Y X1 X2 · · · Xp

1 y1 x11 x12 · · · x1p

2 y2 x21 x22 · · · x2p

3 y3 x31 x32 · · · x3p...

......

......

...n yn xn1 xn2 · · · xnp

Applied Regression

Data structure

Model Specification I

The form of the model that is thought to relate the responsevariable to the set of predictor variables must be specifiedexplicitly. i.e., select the exact form of thefunctionf (X1,X2, · · · ,Xp), it can be classified into two types.

Applied Regression

Data structure

Model Specification II

1 linearWhich means the regression parameters enter linearly, forexample:

Y = β0 + β1X1 + ε

Y = β0 + β1 log X + ε

2 nonlinear like

Y = β0 + eβ1X1 + ε

Applied Regression

Classification

1 Requirement

2 Introduction concepts in Regression Analysis

3 Formula

4 Steps in Regression Analysis

5 Data structure

6 ClassificationModel fittingModel criticisum

Applied Regression

Classification

Classification I

1 Simple regressionIf a regression equation contains only one predictor variable

Y = β0 + β1X

2 Multipe regressionIf an equation containing more than one predictors

Y = β0 + β1X1 + β2X2 + · · ·+ βpXp + ε

Applied Regression

Classification

Classification II

1 Univariate regressionIf a regression equation contains only one response variable

2 Multivariate regressionIf an equation containing more than one response variables

Applied Regression

Classification

Model fitting

Model fitting

Suppose we specify the form the function of f (X1,X2, · · · ,Xp) as:

Y = β0 + β1X1 + β2X2 + · · ·+ βpXp + ε

Then the process of estimate β′s is called model fitting. Denotethe estimated β’s as β′s. Then the fitted model is :

Y = β0 + β1X1 + β2X2 + · · · βpXp

Y ’s: the fitted values

Applied Regression

Classification

Model criticisum

Model Criticism and Selection

1 Assumptions about the data and the model met ?

2 What are the required assumptions?

3 For each of these assumptions, how do we determine whetheror not the assumption is valid.

4 What can be done in cases where one or more of theassumptions does not hold?

Applied Regression

Classification

Model criticisum

A schematic illustration of the cyclic nature of the RAprocess

subject matter theories

Model

Data

Statistical techniques

Auxiliary assumptions

Parameter estimates

Confidence regions

Test Statistics

Graphical Display

Applied Regression

Classification

Model criticisum

The simple linear regression model

⇐⇒diagostics∑Calculate

Applied Regression

Classification

Model criticisum

Homework

1 H.w. 2.1, 2.2

2 Are the following equations linear models or not? Explain it.

Y = β0 + β1X + β2X2 + β3X

3 + εY = 5 + β1X

31 + β2X

22 + ε

Y = β0 + β1X1 + β2X1X2 + β3X1X2 + εY = β0 + β1X1β2 + εY = β0X

β1X1

1 ε