Linear Algebra

Lecture 12

Projections

Problem

Given a vector 1. 2.

a and a vector b, find p p ∈ span({a}) (b − p) ⊥ a b

such that

a

p =xa

Projections

Problem

Given a vector 1. 2.

a and a vector b, find p p ∈ span({a}) (b − p) ⊥ a b

such that

a · (b − p) = 0 aT b x= T a a

a

p =xa

Projections

Problem

Given a vector

a and a vector b, find p such that 1. p ∈ span({a}) aT b p=a T 2. (b − p) ⊥ a a a a · (b − p) = 0 b a aT b x= T a a p =xa

Projections

Problem

Given a vector 1. 2.

a and a vector b, find p such that p ∈ span({a}) aT b p=a T (b − p) ⊥ a a a Orthogonal b projection of b onto p ∈ span({a}). a p =xa

Projections

Problem

Given a vector

a and a vector b, find p such that 1. p ∈ span({a}) aT b p=a T 2. (b − p) ⊥ a a a Orthogonal What matrix P b projection of b onto p ∈ span({a}). projects b a onto p ∈ span({a})? aaT P= T a a p =xa

Projections

Problem

Given a vector

a and a vector b, find p such that 1. p ∈ span({a}) aT b p=a T 2. (b − p) ⊥ a a a Orthogonal Projection matrix b projection of b onto p ∈ span({a}). that projects vectors a onto p ∈ span({a}). p =xa

aaT P= T a a

Projections

Problem

Given a vector 1. 2.

a and a vector b, find p such that p ∈ span({a}) aT b p=a T (b − p) ⊥ a a a Orthogonal projection of b onto p ∈ span({a}).

Projection matrix that projects vectors onto p ∈ span({a}).

aa P= T a a

T

Remark: Remark:

P=P P2 = P . Check !

T . Check !

Projections

Problem

Given vectors

a1, a2 and a vector b, find p such that 1. p ∈ span({a1 , a2 }) 2. (b − p) ⊥ a1, (b − p) ⊥ a2 b a2 p = x1 a1 + x2 a2 a1

Projections

Problem

Given vectors

. . . A = a1 . . .

a1, a2 and a vector b, find p such that 1. p ∈ span({a1 , a2 }) 2. (b − p) ⊥ a1, (b − p) ⊥ a2 b p = Ax x = x1 x2 a2 . . . a2 . . .

p = x1 a1 + x2 a2 a1

Projections

Problem

a1, a2 and a vector b, find p such that 1. p ∈ span({a1 , a2 }) 2. (b − p) ⊥ a1, (b − p) ⊥ a2 b p = Ax a2 T a1 (b − Ax) = 0 T a2 (b − Ax) = 0 p = x1 a1 + x2 a2 a1 Given vectors

Projections

Problem

a1, a2 and a vector b, find p such that 1. p ∈ span({a1 , a2 }) 2. (b − p) ⊥ a1, (b − p) ⊥ a2 b p = Ax a2 T A (b − Ax) = 0 Given vectors

p = x1 a1 + x2 a2 a1

Projections

Problem

a1, a2 and a vector b, find p such that 1. p ∈ span({a1 , a2 }) 2. (b − p) ⊥ a1, (b − p) ⊥ a2 b p = Ax a2 T A (b − Ax) = 0 Given vectors

A Ax = A b

T

T

p = x1 a1 + x2 a2 a1

Projections

Problem

Given vectors

a1, a2 and a vector b, find p such that 1. p ∈ span({a1 , a2 }) 2. (b − p) ⊥ a1, (b − p) ⊥ a2 If A then

T

p = Ax A (b − Ax) = 0 A Ax = A b

Projection Matrix

A is invertible,

T −1

T

T

T

x = (A A) A b T −1 T p = A(A A) A b T −1 T P = A(A A) A

T

Projections

p ∈ span({a1 , a2 , . . . , an }) 2. (b − p) ⊥ ai , 1 ≤ i ≤ n . . . . ··· . . p = Ax where A = a1 a2 · · · . . . . ··· and . . 1. In general, given vectors a1 , a2 , . . . , an and a vector b , find p such that

AT (b − Ax) = 0

. . . an . . .

Projections

In general, given vectors a1 , a2 , . . . , an and a vector b , find p such that 1. 2.

p ∈ span({a1 , a2 , . . . , an }) (b − p) ⊥ ai , 1 ≤ i ≤ n p = A(A A) T −1

If AT A is invertible, then

A b

T −1

T

The Projection Matrix,

P = A(A A)

A

T

Projections

Check: 1. P

=P

T

Exercise : Show that if an invertible matrix is symmetric, its inverse is also symmetric.

The Projection Matrix,

P = A(A A)

T

−1

A

T

Projections

Check: 1. P

=P 2 2. P = P

T

The Projection Matrix,

P = A(A A)

T

−1

A

T

Projections

Check: 1. P

=P 2 2. P = P

T

b Pb

Projection matrix P projects vectors onto C(A) .

C(A)

The Projection Matrix,

P = A(A A)

T

−1

A

T

Projections

Check: 1. P

=P 2 2. P = P

T

b − Pb = (I − P)b b Pb

Projection matrix P projects vectors onto C(A) .

C(A)

The...