Bernard Cornet January 18, 2011

Contents

Notation 1 Euclidean Spaces 1.1 Scalar Product and Associated Norm . . . . . . . . . . . . 1.1.1 Scalar Product . . . . . . . . . . . . . . . . . . . . 1.1.2 Norm Associated to a Scalar Product . . . . . . . . 1.1.3 Convergence in a Normed Space . . . . . . . . . . . 1.1.4 Euclidean Spaces and Hilbert Spaces . . . . . . . . 1.2 Matrices and Scalar Product . . . . . . . . . . . . . . . . . 1.2.1 Generalities on Matrices . . . . . . . . . . . . . . . 1.2.2 Matrices and Scalar Product . . . . . . . . . . . . . 1.2.3 Positive (Negative) Deﬁnite Matrices . . . . . . . . 1.3 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Orthogonal Vectors . . . . . . . . . . . . . . . . . . 1.3.2 Orthogonal Space . . . . . . . . . . . . . . . . . . . 1.3.3 Gram-Schmidt Orthogonalization Process . . . . . 1.4 Orthogonal Projectors . . . . . . . . . . . . . . . . . . . . 1.4.1 Linear Projectors . . . . . . . . . . . . . . . . . . . 1.4.2 Orthogonal Projections . . . . . . . . . . . . . . . . 1.4.3 Symmetric Endomorphisms and Matrices . . . . . 1.4.4 Gram Matrix of a Family of Vectors . . . . . . . . . 1.4.5 Orthogonal Projections and Matrices . . . . . . . . 1.4.6 Least Squares Problem . . . . . . . . . . . . . . . . 1.4.7 Hyperplanes . . . . . . . . . . . . . . . . . . . . . . 1.5 Symmetric Matrices and Endomorphisms . . . . . . . . . . 1.5.1 Deﬁnitions . . . . . . . . . . . . . . . . . . . . . . . 1.5.2 Diagonalization of a Symmetric Matrix . . . . . . . 1.5.3 Positive and Negative Deﬁnite Matrices, continued 1 3 5 5 5 7 8 9 10 10 14 15 16 16 17 18 19 19 20 21 23 24 25 25 28 28 28 29

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS 1.6 Dual of a Euclidean Space . . . . . . . . . . . . . . . . . 1.6.1 Dual of a Linear Space . . . . . . . . . . . . . . . 1.6.2 Representation of the Dual Space . . . . . . . . . 1.6.3 Gradient and Frechet Derivative . . . . . . . . . . 1.6.4 Hyperplanes, continued . . . . . . . . . . . . . . . Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . 1.7.1 Deﬁnitions . . . . . . . . . . . . . . . . . . . . . . 1.7.2 Positive Deﬁnite Quadratic Forms . . . . . . . . . 1.7.3 Scalar Product Associated with a Quadratic Form . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 30 30 32 33 34 35 35 36 36

1.7

Notation

When it is not basic, a precise deﬁnition for the terminology introduced below is to be found in the course of the book (with the help of the index). Given a positive integer n, Rn := n-dimensional Euclidean space; Rn := the nonnegative orthant of Rn ; + R := R1 ; R+ := R1 ; + R := R ∪ {−∞} ∪ {+∞}. For any x, y ∈ Rn , and r > 0 in R, xi := the ith coordinate of x, i = 1, . . . , n; x · y := √ n xi yi denotes the Euclidean inner product of x and y; i=1 x := x · x denotes the Euclidean norm of x; x ≥ y means xi ≥ yi for every i = 1, . . . , n; x > y means x ≥ y and x = y; x y means xi > yi for every i = 1, . . . , n; n + x := x ∨ 0 = max{xi , 0} i=1 denotes the positive part of x; n x− := (−x) ∨ 0 = max{−xi , 0} i=1 denotes the negative part of x; B(x, r) := {z ∈ Rn | z − x < r} denotes the open ball with center x and radius r; B(x, r) := {z ∈ Rn | z − x ≤ r} denotes the closed ball with center x and radius r; [x, y] denotes the closed line segment joining x and y; ]x, y[ or (x, y) denote the open line segment joining x and y. For any subsets A, B of Rn , for any λ ∈ R, A + B := {x + y : x ∈ A, y ∈ B}; A − B := {x − y : x ∈ A, y ∈ B}; 3

Notation

4

λA := {λx : λ ∈ R, x ∈ A}. The algebraic concepts A + B and A − B should not be confused with the set-theoretic concepts A ∪ B and A \ B. For any subset A of Rn , coA denotes the convex hull of A; AﬀA denotes the aﬃne hull of A; cone(A) denotes the cone generated by A; clA denotes the closure of A; int(A) denotes the interior of A; ∂A denotes the boundary of A; clcoA...