CHAPTER 5. RECURSION AND RECURRENCES

5.2

The Master Theorem

Master Theorem

In the last section, we saw three diﬀerent kinds of behavior for recurrences of the form aT (n/2) + n if n > 1 d if n = 1.

T (n) =

These behaviors depended upon whether a < 2, a = 2, and a > 2. Remember that a was the number of subproblems into which our problem was divided. Dividing by 2 cut our problem size in half each time, and the n term said that after we completed our recursive work, we had n additional units of work to do for a problem of size n. There is no reason that the amount of additional work required by each subproblem needs to be the size of the subproblem. In many applications it will be something else, and so in Theorem 5.1 we consider a more general case. Similarly, the sizes of the subproblems don’t have to be 1/2 the size of the parent problem. We then get the following theorem, our ﬁrst version of a theorem called the Master Theorem. (Later on we will develop some stronger forms of this theorem.) Theorem 5.1 Let a be an integer greater than or equal 1. Let c be a positive real number and d a nonnegative form aT (n/b) + nc T (n) = d then for n a power of b, 1. if logb a < c, T (n) = Θ(nc ), 2. if logb a = c, T (n) = Θ(nc log n), 3. if logb a > c, T (n) = Θ(nlogb a ). Proof: In this proof, we will set d = 1, so that the bottom level of the tree is equally well computed by the recursive step as by the base case. It is straightforward to extend the proof for the case when d = 1. Let’s think about the recursion tree for this recurrence. There will be logb n levels. At each level, the number of subproblems will be multiplied by a, and so the number of subproblems at level i will be ai . Each subproblem at level i is a problem of size (n/bi ). A subproblem of size n/bi requires (n/bi )c additional work and since there are ai problems on level i, the total number of units of work on level i is ai (n/bi )c = nc ai bci = nc a bc i

to 1 and b be a real number greater than real number. Given a recurrence of the if n > 1 if n = 1

.

Recall from above that the diﬀerent cases for c = 1 were when the work per level was decreasing, constant, or increasing. The same analysis applies here. From our formula for work on level i, we see that the work per level is decreasing, constant, or increasing exactly when ( ba )i c

5.2. THE MASTER THEOREM

171

is decreasing, constant, or increasing. These three cases depend on whether ( ba ) is 1, less than c 1, or greater than 1. Now observe that ( ba ) = 1 c ⇔ ⇔ a = bc logb a = c ⇔ logb a = c logb b

Thus we see where our three cases come from. Now we proceed to show the bound on T (n) in the diﬀerent cases. In the following paragraphs, we will use the facts (whose proof is a straightforward application of the deﬁnition of 1ogartihms and rules of exponents) that for any x, y and z, each greater than 1, xlogy z = z logy x and that logx y = Θ(log2 y). (See Problem 3 at the end of this section and Problem 4 at the end of the previous section.) In general, we have that the total work done is logb n

nc

i=0

a bc

i

logb n

= nc

i=0

a bc

i

In case 1, (part 1 in the statement of the theorem) this is nc times a geometric series with a ratio of less than 1. Theorem 4.4 tells us that logb n

n

c i=0

a bc

i

= Θ(nc ).

Exercise 5.2-1 Prove Case 2 of the Master Theorem. Exercise 5.2-2 Prove Case 3 of the Master Theorem. In Case 2 we have that logb n a bc

= 1 and so

i logb n

nc

i=0

a bc

a bc

= nc

i=0

1i = nc (1 + logb n) = Θ(nc log n) .

In Case 3, we have that

> 1. So in the series

logb n c

n

i=0

a bc

i

logb n

=n

c i=0

a bc

i

,

a logb n bc

the largest term is the last one, so by Theorem 4.4,the sum is Θ nc a bc logb n

. But

nc

alogb n (bc )logb n nlogb a = nc log bc n b nlogb a = nc nc logb a = n . = nc

172 Thus the solution is Θ(nlogb a ).

CHAPTER 5. RECURSION AND...