MACM 316

1/9

Numerical Differentiation

® Suppose we have a list of points

x0 < x1 < x2 < · · · < xn

and corresponding function values

f (x 0 ), f (x 1 ), f (x 2 ), . . . , f (x n )

® A natural question is whether we can use the data above to approximate f ′ (x) at some point x ∈ [x0, xn].

® The answer is easy provided the points xi are equally spaced so that xi − xi−1 = h (constant) and x.

The easiest way to motivate derivative formulas is using the definition of derivative f (x + h) − f (x )

f ′ (x) = lim

h→0

h

which suggests many possible difference formulas:

f ′ (x ) ≈

f (x + h) − f (x )

h

f (x ) − f (x − h)

f ′ (x ) ≈

h

f (x + h) − f (x − h)

f ′ (x ) ≈

2h

f (x + 2 h) − f (x − 2 h)

′

f (x ) ≈

4h

(forward difference)

(backward difference)

(centered difference)

(wide centered difference)

···

® These formulas are accurate only if h is “small enough.” October 30, 2008

c Steven Rauch and John Stockie

Numerical Diﬀerentiation

MACM 316

2/9

Example

Suppose we’re approximating the derivative of f (x) = 2 sin(3x) using the equally-spaced data

x 0.3000 0.3250 0.3500 0.3750 0.4000 0.4250 0.4500 0.4750 0.5000 f (x) 1.5667 1.6554 1.7348 1.8045 1.8641 1.9131 1.9514 1.9788 1.9950

The approximations of f ′ (0.4) with h = 0.1 are:

1. f ′ (x) ≈

f (x+h)−f (x)

h

f ′ (0.4) ≈

1.9950−1.8641

0. 1

= 1.3090

(40%)

2. f ′ (x) ≈

f (x)−f (x−h)

h

f ′ (0.4) ≈

1.8641−1.5667

0. 1

= 2.9740

(37%)

3. f ′ (x) ≈

f (x+h)−f (x−h)

2h

f ′ (0.4) ≈

1.9950−1.5667

0. 2

= 2.1415

(1.5%)

where the relative errors are computed using the exact value f ′ (0.4) = 6 cos(3 × 0.4) = 2.17414652686004

2.1

2

y

1.9

1.8

1.7

1.6

1.5

0.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

x

Figure 1: f (x) = 2 sin(3x)

October 30, 2008

c Steven Rauch and John Stockie

Numerical Diﬀerentiation

MACM 316

3/9

Example (cont’d)

Investigate what happens when h is decreased to 0.05:

x 0.3000 0.3250 0.3500 0.3750 0.4000 0.4250 0.4500 0.4750 0.5000 f (x) 1.5667 1.6554 1.7348 1.8045 1.8641 1.9131 1.9514 1.9788 1.9950

The approximations of f ′ (0.4) with h = 0.05 are:

1. f ′ (x) ≈

f (x+h)−f (x)

h

f ′ (0.4) ≈

1.9514−1.8641

0.05

= 1.7460

(20%)

2. f ′ (x) ≈

f (x)−f (x−h)

h

f ′ (0.4) ≈

1.8641−1.7348

0.05

= 2.5860

(19%)

3. f ′ (x) ≈

f (x+h)−f (x−h)

2h

f ′ (0.4) ≈

1.8641−1.7348

0. 1

= 2.1660

(0.4%)

4. f ′ (x) ≈

f (x+2h)−f (x−2h)

4h

f ′ (0.4) ≈

1.9950−1.5667

0. 2

= 2.1415

(1.5%)

Notice that:

• The forward and backward difference formulas (1 and 2) have similar accuracy.

• The centered difference (3) is much more accurate than the one-sided differences.

• Decreasing h increases the accuracy of the approximation. Question: Can this be explained?

October 30, 2008

c Steven Rauch and John Stockie

Numerical Diﬀerentiation

MACM 316

4/9

Error Analysis

To analyse the error in ﬁnite difference formulas, use Taylor series approximations. Example 1: Forward difference formula

• Write the Taylor polynomial of degree n = 1, with error term: ′

f (x + h) = f (x ) + f (x ) h +

f ′′ (c)

2

h2

• Then

f (x + h) − f (x )

h

′

= f (x ) +

f ′′ (c)

2

h

= f ′ (x ) + O (h)

• Decreasing h clearly reduces the error.

Example 2: Centered difference formula

• Taylor polynomials for f (x + h) and f (x − h) to O (h4): f ′′(x) 2 f ′′′(x) 3

f (4) (x) 4

f (5) (c1 ) 5

f (x + h) = f (x) + f (x) h +

h+

h+

h+

h

2

6

4!

5!

′

f (x − h) = f (x) − f ′(x) h +

f ′′ (x)

2

h2 −

f ′′′(x)

6

h3 +

f (4) (x)

4!

h4 −

f (5) (c2 )

5!

h5

• Subtract the second equation from the ﬁrst and divide by 2h: f ′′′(x) 2

f (x + h) − f (x − h)

′

= f (x) +

h + O (h4 )

2h

6

• Error in centered formula is smaller (as expected)!

October 30, 2008

c Steven Rauch and John Stockie...