Pages: 29 (2637 words) Published: February 6, 2013
ECO 362

Regression Analysis:
Model Building
General Linear Model
Determining When to Add or Delete
Variables
Variable Selection Procedures
Residual Analysis
Multiple Regression Approach to Analysis
of Variance and Experimental Design

Chapter 16
Regression Analysis:
Model Building

SUNY Plattsburgh

Dr. Kameliia Petrova

Slide 1

Dr. Kameliia Petrova

Linear models: models in which all parameters
(β 0, β 1, . . . , β p ) have exponents of one.
General linear model with p independent
variables:

The simplest case is when z1 = x1. We want
to estimate y by using a straight-line
relationship.
Simple first-order model with one predictor
(independent) variable.

y = β 0 + β1 z1 + β 2 z2 + L + β p zp + ε

Each of the independent variables z is a
function of x1, x2,..., xk (the variables for
which data have been collected).
SUNY Plattsburgh

Slide 2

General Linear Model

General Linear Model

Dr. Kameliia Petrova

SUNY Plattsburgh

y = β 0 + β 1 x1 + ε
0

Slide 3

Modeling Curvilinear
Relationships

Dr. Kameliia Petrova

SUNY Plattsburgh

Slide 4

Interaction

To account for a curvilinear relationship we
set z1 = x1 and z2 = x12

Second-order model with two predictor
variables.
2
2
y = β0 +β1x1 +β2x2 +β3x1 + β4x2 +β5x1x2 + ε

Second-order model with one predictor
variable:

Variable z5 = x1x2 is added to account for the
potential effects of the two variables acting
together.

2
y = β 0 + β 1x1 + β 2 x1 + ε

This type of effect is called interaction.
Dr. Kameliia Petrova

SUNY Plattsburgh

Slide 5

Dr. Kameliia Petrova

SUNY Plattsburgh

Slide 6

1

Nonlinear Models That Are
Intrinsically Linear

Transformations Involving the
Dependent Variable
The non-constant variance can be corrected by
transforming the dependent variable to a
different scale.

Models in which the parameters (β 0, β 1, . . . ,
β p ) have exponents other than one are called
nonlinear models.

Logarithmic transformations: using either the
base-10 (common log) or the base e =
2.71828... (natural log). Log(y) or Ln(y) instead
of y
Reciprocal transformation: use 1/y as the

Perform a transformation of variables and use
regression analysis with the general linear
model.
x
E( y ) = β 0 β 1
Exponential model:

Dr. Kameliia Petrova

SUNY Plattsburgh

Slide 7

Delete Variables

Transform to a linear model by taking the
logarithm of both sides.
Dr. Kameliia Petrova

SUNY Plattsburgh

Variable Selection Procedures

To test whether the addition of x2 to a model
involving x1 (or the deletion of x2 from a model
involving x1 and x2) is statistically significant we
can perform an F Test.
Determine the amount of reduction in the SSE
resulting from adding one or more
independent variables to the model.

Stepwise Regression
Forward Selection
Backward Elimination
Best-Subsets Regression

(SSE(x1 )-SSE(x 1 ,x 2 ))/1
F=
(SSE(x1 , x2 ))/(n − p − 1)
Dr. Kameliia Petrova

SUNY Plattsburgh

Slide 9

Variable Selection:
Stepwise Regression

Add the most significant variable not in the
model because its F value is greater than the
user-specified or default Alpha to enter.
If no variable can be removed and no
variable can be added, the procedure stops.
SUNY Plattsburgh

Dr. Kameliia Petrova

Iterative; one
independent
variable at a
deleted based on
the F statistic
Different subsets
of the
indep. variables
are evaluated

SUNY Plattsburgh...