ECO 362

Regression Analysis:

Model Building

General Linear Model

Determining When to Add or Delete

Variables

Variable Selection Procedures

Residual Analysis

Multiple Regression Approach to Analysis

of Variance and Experimental Design

Chapter 16

Regression Analysis:

Model Building

School of Business and Economics

SUNY Plattsburgh

Dr. Kameliia Petrova

Slide 1

Dr. Kameliia Petrova

Linear models: models in which all parameters

(β 0, β 1, . . . , β p ) have exponents of one.

General linear model with p independent

variables:

The simplest case is when z1 = x1. We want

to estimate y by using a straight-line

relationship.

Simple first-order model with one predictor

(independent) variable.

y = β 0 + β1 z1 + β 2 z2 + L + β p zp + ε

Each of the independent variables z is a

function of x1, x2,..., xk (the variables for

which data have been collected).

School of Business and Economics

SUNY Plattsburgh

Slide 2

General Linear Model

General Linear Model

Dr. Kameliia Petrova

School of Business and Economics

SUNY Plattsburgh

y = β 0 + β 1 x1 + ε

0

Slide 3

Modeling Curvilinear

Relationships

Dr. Kameliia Petrova

School of Business and Economics

SUNY Plattsburgh

Slide 4

Interaction

To account for a curvilinear relationship we

set z1 = x1 and z2 = x12

Second-order model with two predictor

variables.

2

2

y = β0 +β1x1 +β2x2 +β3x1 + β4x2 +β5x1x2 + ε

Second-order model with one predictor

variable:

Variable z5 = x1x2 is added to account for the

potential effects of the two variables acting

together.

2

y = β 0 + β 1x1 + β 2 x1 + ε

This type of effect is called interaction.

Dr. Kameliia Petrova

School of Business and Economics

SUNY Plattsburgh

Slide 5

Dr. Kameliia Petrova

School of Business and Economics

SUNY Plattsburgh

Slide 6

1

Nonlinear Models That Are

Intrinsically Linear

Transformations Involving the

Dependent Variable

The non-constant variance can be corrected by

transforming the dependent variable to a

different scale.

Models in which the parameters (β 0, β 1, . . . ,

β p ) have exponents other than one are called

nonlinear models.

Logarithmic transformations: using either the

base-10 (common log) or the base e =

2.71828... (natural log). Log(y) or Ln(y) instead

of y

Reciprocal transformation: use 1/y as the

dependent variable instead of y.

Perform a transformation of variables and use

regression analysis with the general linear

model.

x

E( y ) = β 0 β 1

Exponential model:

Dr. Kameliia Petrova

School of Business and Economics

SUNY Plattsburgh

Slide 7

Determining When to Add or

Delete Variables

Transform to a linear model by taking the

logarithm of both sides.

Dr. Kameliia Petrova

School of Business and Economics

SUNY Plattsburgh

Variable Selection Procedures

To test whether the addition of x2 to a model

involving x1 (or the deletion of x2 from a model

involving x1 and x2) is statistically significant we

can perform an F Test.

Determine the amount of reduction in the SSE

resulting from adding one or more

independent variables to the model.

Stepwise Regression

Forward Selection

Backward Elimination

Best-Subsets Regression

(SSE(x1 )-SSE(x 1 ,x 2 ))/1

F=

(SSE(x1 , x2 ))/(n − p − 1)

Dr. Kameliia Petrova

School of Business and Economics

SUNY Plattsburgh

Slide 9

Variable Selection:

Stepwise Regression

Add the most significant variable not in the

model because its F value is greater than the

user-specified or default Alpha to enter.

If no variable can be removed and no

variable can be added, the procedure stops.

School of Business and Economics

SUNY Plattsburgh

Dr. Kameliia Petrova

Iterative; one

independent

variable at a

time is added or

deleted based on

the F statistic

Different subsets

of the

indep. variables

are evaluated

School of Business and Economics

SUNY Plattsburgh...